Monitoring daemons with shell tools

Watching the Daemons

Article from ADMIN 01/2010

By Harald Zisler

Administrators often write custom monitoring programs to make sure their daemons are providing the intended functionality. But simple shell tools are just as well suited to this task, and not just for systems that are low on resources.

Unix daemons typically go about their work discreetly in the background. The process table, which is output with the ps command, only shows you that these willing helpers have been launched, although in the worst case they could just be hanging around as zombies. Whether or not a daemon is actually working is not something that the process table will tell you. In other words, you need more granular diagnostics. The underlying idea is to write a "sensor" script for each service that performs a tangible check of its individual functionality.

Because almost every program outputs standardized exit codes when it terminates, you can use Unix conventions. 0 stands for error-free processing, whereas 1 indicates some problems were encountered. This value is stored in the $? shell variable, which a shell script evaluates immediately after launching the sensor.

Various programs are suitable for automated, "unmanned" access to the service provided by a given daemon; all of them will run in the shell without a GUI. These programs often provide an option (typically -q) that suppresses output, and this is fine for accessing the exit code. Error logs are obtainable by redirecting the error output to a file or, if available, by setting the corresponding program option.

The only thing left to do is to find the matching client program test the functionality of each service.

Web Servers

To check a web server, you could use wget. The shell script command line for this would be:

wget --spider -q ip-address

The --spider option tells wget to check that the page exists but not to load it. Defining the IP address instead of the hostname avoids a false positive if DNS-based name resolution fails for some reason.

Almost

...

Use Express-Checkout link below to read the full article (PDF).