Ansible Task Execution and Parallelism Guide

In this guide, we'll cover how to manage task execution in Ansible, including parallel execution, async tasks, modular playbook management, and best practices. The goal is to ensure that your Ansible automation is efficient, scalable, and maintainable.

1. Sequential vs Parallel Execution in Ansible

Ansible typically runs tasks sequentially within a playbook. By default:

• Tasks within a play run one after another.
• Plays in a playbook also run sequentially.

To manage larger infrastructures or parallelize tasks, you can tweak this behavior.

1.1. Sequential Task Execution (Default)

Tasks are run in the order they appear in the playbook. This is easy to manage and understand.

Example:

---
- hosts: all
  tasks:
    - name: Create groups
      group:
        name: "ansible"
        state: present
 
    - name: Create user and assign to groups
      user:
        name: "ansible_user"
        groups: "ansible,sudo"
        state: present
        create_home: yes

Best Practice: Use sequential execution for tasks that depend on the completion of earlier steps (e.g., user creation after group creation).

2. Parallelism with Forks

Ansible can run tasks in parallel across multiple hosts by configuring the number of "forks." This controls how many hosts are handled simultaneously, while tasks still run sequentially on each host.

2.1. Setting Forks in ansible.cfg

Configure the number of forks globally by modifying ansible.cfg.

# ansible.cfg
[defaults]
forks = 10  # Run tasks on 10 hosts in parallel

Alternatively, you can specify the forks option when running a playbook:

ansible-playbook playbook.yml -f 10  # Forks 10 parallel host executions

Best Practice: Adjust the forks number based on the number of hosts you are managing and the available system resources. Too many forks can overwhelm the control node.

3. Running Tasks in the Background (Async Tasks)

For tasks that might take a long time (e.g., database backups, file transfers), you can use Ansible’s async feature to run tasks asynchronously. This prevents long-running tasks from blocking the execution of subsequent tasks.

3.1. Async and Poll Example

Run a task in the background using async and check the status later with poll.

---
- hosts: all
  tasks:
    - name: Long-running task
      command: /path/to/long_task.sh
      async: 600  # Run for up to 10 minutes
      poll: 0  # Do not wait for task to finish
 
    - name: Check task status
      async_status:
        jid: "{{ ansible_job_id }}"
      register: job_result
      until: job_result.finished
      retries: 5
      delay: 10

• async: 600: Task will run for up to 10 minutes.
• poll: 0: No need to wait for it to finish before moving on.
• async_status: Poll the status of the job at a later point.

Best Practice: Use async for time-consuming or non-blocking tasks. Keep retries and delays reasonable to avoid too many status checks.

4. Modular Playbooks with import or include

Instead of putting everything in one large playbook, you can split tasks into smaller playbooks. This makes it easier to manage and reuse code. Ansible offers two options for this:

• import_playbook: Imports another playbook at the time the playbook is read.
• include_tasks: Dynamically includes tasks during runtime.

4.1. Using import_playbook

You can create multiple playbooks and organize them into a master playbook for sequential execution.

Example Structure:

site.yml
create_groups.yml
create_users.yml

site.yml:

---
- import_playbook: create_groups.yml
- import_playbook: create_users.yml

Each playbook (create_groups.yml and create_users.yml) will contain their specific tasks.

4.2. Using include_tasks

You can include tasks dynamically within a play, allowing finer control over which tasks are executed.

---
- hosts: all
  tasks:
    - name: Include task to create groups
      include_tasks: create_groups.yml
 
    - name: Include task to create users
      include_tasks: create_users.yml

Best Practice: Modularize your playbooks using import_playbook or include_tasks to enhance reusability, readability, and maintenance.

5. Delegate Tasks to Other Hosts

If you need a task to run on a specific host but triggered by another, you can use delegation.

5.1. Delegation Example

This allows a host to delegate a task to another host.

---
- hosts: web
  tasks:
    - name: Create database on the DB server
      mysql_db:
        name: myapp_db
        state: present
      delegate_to: db_host

This task is delegated to db_host from the web host.

Best Practice: Use delegation for tasks that must be executed on other hosts but require triggers from a central playbook.

6. Best Practices for Efficient Ansible Playbooks

6.1. Structure Your Playbooks

• Use roles for large playbooks to modularize tasks, variables, handlers, and files.
• Split complex playbooks using import_playbook or include_tasks.

6.2. Limit Forks Based on Environment

• Monitor your system’s resources (CPU, memory) when running Ansible with many forks.
• Keep the forks number aligned with your infrastructure’s capacity.

6.3. Use Tags for Selective Execution

• Apply tags to tasks to selectively run them without executing the whole playbook.

Example:

---
- name: "Install web server"
  hosts: all
  tasks:
    - name: "Install Nginx"
      apt:
        name: nginx
        state: present
      tags: nginx

You can run only the tagged tasks:

ansible-playbook playbook.yml --tags nginx

6.4. Optimize Inventory Management

• Use dynamic inventory for cloud or dynamic infrastructures.
• Maintain separate inventory files for different environments (e.g., production, staging).

6.5. Leverage Handlers and Notifications

• Use handlers to trigger actions like restarting a service after configuration changes.
• Handlers improve performance by only running when notified.

Example:

---
- hosts: all
  tasks:
    - name: Update web server config
      copy:
        src: /path/to/config
        dest: /etc/nginx/nginx.conf
      notify:
        - Restart nginx
 
  handlers:
    - name: Restart nginx
      service:
        name: nginx
        state: restarted

7. Conclusion

This guide outlines various techniques to manage task execution in Ansible effectively. Whether running tasks in parallel with forks, using async tasks for long-running processes, or splitting playbooks into modular components, Ansible provides flexible mechanisms for handling complex automation.

Key Takeaways:

• Use forks to parallelize across hosts.
• Use async for background task execution.
• Modularize your playbooks with import_playbook and include_tasks.
• Delegate tasks when appropriate, and apply best practices like using tags, handlers, and roles for better scalability and readability.