Nobody enjoys patching servers – or at least I don't know anyone who claims to – but everybody agrees that security patches are essential, at least for front-line systems such as web servers, firewalls, load balancers, mail servers, and so on. However, even in 2025, I still hear people saying (in essence): "You don't need to worry, you know. These machines are inaccessible from the Internet and hidden right at the back of the LAN."

This setup could even work for a while, until someone right at the back of the LAN has the bright idea of plugging the fancy USB stick they found earlier in the lounge into their work PC or of opening what appears to be a totally harmless email attachment – after all, the company has a painfully expensive state-of-the-art endpoint security solution in place. In these cases, even the most attractive, fancy software box on the shelf in the admin's office is of no use whatsoever. Do you have highly segmented networks in your back office? Are your LAN sockets protected by MAC filters? Do you disable USB ports or even lock away your hardware in metal boxes? The thought of users opening up a path for malware right into the heart of the company is likely to make many an admin break out in a cold sweat.

Against the backdrop of these horror scenarios, it will not hurt to look at some totally banal but still very real issues, including rolling out mass updates, which plays a particularly important role if you don't have access to powerful, but potentially expensive, tools like SUSE Multi-Linux Manager [1] or Uyuni [2]. In such a case, it's worth considering Ansible [3].

Inventory

Ansible always begins with inventory, irrespective of whether you are starting with an existing infrastructure or setting up a greenfield deployment.

Ideally, the Ansible inventory will include all of your hosts at some point. I am deliberately not talking about all the critical hosts. If you take a closer look at this kind of automation, you will quickly understand that the less critical systems do not cause any additional overhead. A typical inventory file (Listing 1) can look relatively simple. Whether structured in YAML [4] or the INI format, it ultimately has no influence on the results. I opted for YAML.

01 ---
02 servers:
03   children:
04     debian_servers:
05       hosts:
06         # Pi-hole
07         pihole:
08           ansible_ssh_private_key_file: /home/treuss/.ssh/pihole
09           ansible_user: thomas
10         # Gitea
11         raspi03:
12           ansible_ssh_private_key_file: /home/treuss/.ssh/raspi03
13           ansible_user: thomas
14         # Backup
15         raspi06:
16           ansible_ssh_private_key_file: /home/treuss/.ssh/raspi06
17           ansible_user: thomas
18     dns_servers:
19       hosts:
20         # Pi-hole
21         pihole:
22           ansible_ssh_private_key_file: /home/treuss/.ssh/pihole
23           ansible_user: thomas

As you can see, I defined my hosts in the file (Figure 1). In this specific case, the inventory is named servers.yml. As the name suggests, it only contains servers. How you break down your hosts is up to you. In this example, I distinguished between debian_servers (line 4) and dns_servers (line 18). Although I have only been running Debian servers up to now, I would like to keep open the option of setting up a SUSE- or Gentoo-based machine.

Figure 1: The schematic of an Ansible inventory showing the hosts and connections.

Another reason for classifying by (Layer 7) services is that I use Pi-hole [5] as an ad blocker. It has its own update management, which has its own life outside the distribution package manager. If I want to update Pi-hole, I can't just call apt update && apt upgrade.

I defined two variables for each host: ansible_user specifies the user Ansible will use to log in to the target system, and ansible_ssh_private_key_file stipulates private key-based SSH login.

Playbooks

Besides inventory, the Ansible world has other important files. I'm talking about playbooks, of course. Although the inventory determines the subject of the operation, playbooks define what will happen.

The sample playbook in Listing 2 for updating Debian systems follows a relatively simple structure, much like the inventory example. Line 2 contains the playbook's name, and line 3 ensures that Ansible only executes the tasks that follow on my Debian servers.

01 ---
02 - name: Patch Debian servers
03   hosts: debian_servers
04   tasks:
05   - name: Use apt to update all Debian servers
06     ansible.builtin.apt:
07       force_apt_get: yes
08       upgrade: dist
09     become: yes
10     become-method: sudo

Experience has shown that the YAML format is rarely self-explanatory for newcomers. For deeper insights, please refer to the documentation [5]. For my purposes, a couple of brief details should suffice: Ansible interprets lines beginning with a dash as list elements. Lines of the form <key>: <value> are the key-value pairs of a dictionary or a hash table. In contrast to long-established formats such as XML or JSON, whitespace is very important in YAML. As in Python, the indentation depth determines to which element a line belongs. Complex structures can be created with just these few syntax elements.

This playbook, named Patch Debian servers in line 2 targets all the hosts in the inventory that belong to the debian_servers host group. Ansible applies the specified tasks to each of these hosts. This example has just one task, ansible.builtin.apt, and its assigned name in line 5 is Use apt to update all Debian servers . Line 6, ansible.builtin, already suggests something from the tool's standard repertoire: the built-in Apt module.

Linux users do not laboriously have to gather the tools needed for their choice of operating system. Ansible comes with a massive collection out of the box. The force_apt_get switch forces Apt to update the package sources again. The upgrade switch tells Apt to install the available updates. Finally, I used the --become and --become-method parameters to tell Ansible that a context switch to root is required, which relies on sudo.

If everything is correct, and the indentations in the YAML files match, the output you can see in Figure 2 is gradually generated when the code

$ ansible-playbook --inventory inventory/servers.yml --ask-become-pass debian_patch.yml

Figure 2: If the YAML file has no errors – including indentation – Ansible successfully runs the playbook.

is called. Ansible always determines the facts for all hosts first, which in this case means extensive information, including all the IP addresses, hostnames, and much more.

In the Use apt to update all Debian servers task, Ansible just says ok for two hosts, raspi03 and raspi06 . Therefore, no updates were available by Apt, so Ansible did not have anything to do. No changes occurred in the case of Pi-hole. In Ansible parlance, a change means a deviation from the target state, which the tool was able to restore.

The --ask-become-pass option tells software to request the password interactively for the context switch to root from the user. Ansible again offers tools for this use case: vaults. With the help of vaults, you can also run playbooks without password prompts.

Ansible is a tool for configuration management, so ultimately you specify a target state in the inventory and some other files later on; the Ansible modules then try to establish this state within the framework of playbooks. In other words, if Apt does not find any updatable packages, the Debian systems must already be in the target state, which is a good thing. In the case of raspi03 and raspi06, I activated unattended-upgrades for the hosts at setup. Both systems update automatically on a regular basis, which is why no updates needed to be installed when I called them in Figure 2.

Thinking Ahead

System updates are not all the work that needs to be done. If you run your own server, you will be aware that you cannot always simply download the software you need from your favorite distribution's package repository. Typical examples include ownCloud, Nextcloud, Gitea, Mastodon, and in this case, Pi-hole. Of course, I always want to have the latest version of the software with the latest ad lists containing URLs from advertisers, which means that it made sense to write my own playbook for updating Pi-hole [5].

Ultimately, just two commands are all it took, starting with

sudo pihole updatePihole
sudo pihole updateGravity

to update the software and the lists of advertisers to be blocked. Both commands require root privileges, as you can see from the call to sudo. Therefore, you again need the become parameter at this point.

In Listing 3, you can see the two calls defined as two tasks. As a reminder, the hyphens stand for list elements, which is why any number of tasks can be strung together. This playbook tells Ansible to run the two commands on all dns_servers (line 3). I used the ansible.builtin.command (lines 6 and 9) to call shell commands without any serious configuration overhead.

01 ---
02 - name: Update Pi-hole server
03   hosts: dns_servers
04   tasks:
05   - name: Update Pi-hole software
06     ansible.builtin.command: pihole updatePihole
07     become: yes
08   - name: Update Gravity (AdLists)
09     ansible.builtin.command: pihole UpdateGravity
10     become: yes

However, keep in mind that Ansible interprets a task as successful if the return code (RC) of a program or shell script is zero. Experienced shell users therefore do not need to change anything; the logic remains the same.

Figure 3 shows the output from the playbook. At first glance, you can see that Ansible always uses the same, tidy-looking output regardless of the tasks at hand. Ultimately, it outputs one line per host containing the number of successful tasks (ok ), the number of changes (i.e., deviations from the target state), and so on.

Figure 3: Ansible after updating Pi-hole. In the example, the tool successfully completed three tasks.