Writing Custom Check Modules for Nagios Using mysql and apache2 as Examples

Nagios is an open-source client-server platform for monitoring computer systems and networks. It’s built for observing and tracking the state of compute nodes and services, and notifies the administrator whenever a service stops (or resumes) working. (c)Wikipedia

I won’t go over the full installation - there are plenty of resources online that already cover it.

Nagios is an incredibly flexible monitoring system. There’s a huge number of plugins for nagios, shipped in the nagios-plugins packages, but sometimes the out-of-the-box plugins don’t fit for one reason or another. Either they output a lot of useless info, or they output nothing at all (just don’t work on a particular system). I usually don’t want to install a whole package with a dozen plugins just to check one system value.

A big plus of nagios is that nagios-checks, or check scripts, can be written in anything: bash, python, perl or powershell. I think you could even use a plain bat file to check the health of Windows systems. As one of my buddies used to say: With Nagios you can monitor the coffee in your cup

So, let’s look at writing your own check scripts, using status checks for Mysql-Server and Apache as an example.

First, let’s create a user that the check script (checker) will use to connect to mysql. Sure, you could use the root account, but to be safe I recommend creating a separate user:

grant select on *.* to nagios@localhost identified by 'nagios4mysql';

Next, in the plugins folder (/usr/lib/nagios/plugins/) create the custom_mysql_check file with the following content:

#!/bin/bash
user="nagios";
password="nagios4mysql";
info=$(echo "SHOW STATUS;" |mysql -u$user -p$password |grep "Open_tables\|Uptime\|Threads_run" |grep -v "flush"|awk '{printf $1":"$2", "}');
if ($info); then
	echo "OK: Service is running. $info";
	exit 0
else
	echo "Service is down";
	exit 2
fi

This script uses the standard mysql function to get the info it needs and shows the number of open tables, uptime and the number of mysql instances. You can look more closely at the SHOW STATUS output and add more info. The original point of this check is to make sure mysql server is running and responding to requests (exit 0). If mysql is unavailable - the script returns exit code = 2, which signals nagios that the server has a problem.

All that’s left is to describe this custom check on the nagios server. You can do that by following the pattern of existing checks. And don’t forget to add the same check to the /etc/nagios/nrpe_local.cfg file:

command[custom_mysql_check]=/usr/lib/nagios/plugins/custom_mysql_check

Example bash script for checking the apache http server:

#!/bin/bash
let "number = $(ps aux|grep apache2 |wc -l) -2 "
if [ -f /var/run/apache2.pid ]
then
	echo "OK: Service is up. Running $number child processes";
	exit 0
else
	echo "Service is down";
	exit 2
fi

The script checks for the pid file and counts the number of running apache child processes. In principle you could use the $number variable’s value as the condition:

if [ $number -gt "0" ]

When creating this check, you also need to declare it in the /etc/nagios/nrpe_local.cfg file