Welcome to docs.opsview.com

Using Event Handlers

Event handlers are optional commands that are executed whenever a service state change occurs. An obvious use for event handlers (especially with services) is the ability for Opsview to proactively fix problems before anyone is notified. However, you should still analyse what the root cause is!

Service event handler commands are executed when a service:

  • goes into a “soft” error state
  • each soft error state also invokes the handler
  • goes into a “hard” error state
  • recovers from a “soft” or “hard” error state

In Opsview, event handlers are called on the monitoring server that the host is monitored by. So if you have a slave monitoring HostA, the slave will have the event handler called, and not the master.

More documentation is available on the official Nagios® Core documentation site.

Writing event handler scripts

In most cases, event handler commands will be shell or perl scripts. You should use the available macros in the environment to process. The main variables are:

  • $NAGIOS_HOSTSTATE (UP, DOWN or UNREACHABLE)
  • $NAGIOS_HOSTSTATETYPE (SOFT or HARD)
  • $NAGIOS_HOSTATTEMPT (number, starts from 1)
  • $NAGIOS_SERVICESTATE (OK, WARNING, CRITICAL or UNKNOWN)
  • $NAGIOS_SERVICESTATETYPE (SOFT or HARD)
  • $NAGIOS_SERVICEATTEMPT (number, starts from 1)

Using your script logic, you can only execute action when service is in appropriate state, eg: state is CRITICAL and state type is HARD.

Other Nagios Core environment variables are also available to the script. See Macros as Environment Variables on the Using Macros In Commands reference page. However, only a subset of these are available for event handlers in Opsview. This list is:

  • CONTACTGROUPNAMES
  • CONTACTNAME
  • CONTACTPAGER
  • HOSTACKAUTHOR
  • HOSTACKCOMMENT
  • HOSTADDRESS
  • HOSTALIAS
  • HOSTATTEMPT
  • HOSTGROUPNAME
  • HOSTNAME
  • HOSTOUTPUT
  • HOSTSTATE
  • HOSTSTATETYPE
  • LASTHOSTSTATE
  • LASTHOSTSTATECHANGE
  • LASTSERVICESTATE
  • LASTSERVICESTATECHANGE
  • LONGDATETIME
  • LONGHOSTOUTPUT
  • LONGSERVICEOUTPUT
  • NOTIFICATIONTYPE
  • SERVICEACKAUTHOR
  • SERVICEACKCOMMENT
  • SERVICEATTEMPT
  • SERVICEDESC
  • SERVICEOUTPUT
  • SERVICESTATE
  • SERVICESTATETYPE
  • SHORTDATETIME
  • TIMET

When written and tested, the scripts should be placed in /usr/local/nagios/libexec/eventhandlers on the Opsview master server. Run send2slaves -p to send the eventhandler to all slaves.

Note: the eventhandler script is run on of the Opsview server, not the monitored host.

Using event handler scripts

To use an event handler, on the host configuration page select a service in the Monitors tab and then click on the 'Event Handlers' icon (on the right hand side of the page). Enter the event handler script name (you can also add arguments if your event handler supports it):

handle_service_failure

Then click on Save and then Reload the configuration.

Debugging event handler scripts

  • Check that Log Event Handlers is checked in System Preferences. Reload if changed.
  • Place your script in /usr/local/nagios/libexec/eventhandlers
  • Execute by passing any appropriate environment variables to simulate a Nagios Core execution:
NAGIOS_SERVICESTATE=CRITICAL NAGIOS_SERVICESTATETYPE=HARD NAGIOS_SERVICEATTEMPT=3 /usr/local/nagios/libexec/eventhandlers/handle_service_failure
  • Ensure it is working as expected
  • Run “send2slaves -p” on the Opsview master to send the file to all slaves
  • Configure in Opsview and reload
  • Simulate the failure (either by failing the actual service or by submitting fake results)
  • Check nagios.log on the appropriate server for the message:
[1224616751] SERVICE EVENT HANDLER: opsview:Pipe check;CRITICAL;SOFT;1;host1_service83_eh-hander_cmdpipe.sh
  • This means the event handler was called. If you want to check what environment variables were set by Nagios Core, dump the whole environment to a file for checking (see the example script). You can then invoke your event handler script with these variables set to see what your script would do
  • If you see the log message:
[1224616753] Warning: Attempting to execute the command "/usr/local/nagios/libexec/eventhandlers/event_handler_for_tcpip_on_slave" resulted in a return code of 127.  Make sure the script or binary you are trying to execute actually exists...
  • Then check permissions on the script.

Example script

#!/bin/bash
#
# SYNTAX:
#       handle_service_failure
#
# DESCRIPTION:
#       Template for creating event handlers
#
# AUTHORS:
#       Copyright (C) 2008 Opsera Limited
#
#    This file is part of Opsview
#
#    Opsview is free software; you can redistribute it and/or modify
#    it under the terms of the GNU General Public License as published by
#    the Free Software Foundation; either version 2 of the License, or
#    (at your option) any later version.
#
#    Opsview is distributed in the hope that it will be useful,
#    but WITHOUT ANY WARRANTY; without even the implied warranty of
#    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
#    GNU General Public License for more details.
#
#    You should have received a copy of the GNU General Public License
#    along with Opsview; if not, write to the Free Software
#    Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
#

# Uncomment below to get debug information about the environment variables set by Nagios Core
#{date; env | sort; echo; } >> /tmp/handler.log

# If Service State is CRITICAL (options are OK, WARNING, CRITICAL and UNKNOWN)
# and Service State Type is HARD (options are HARD and SOFT)
# then execute event handler action

if [[ "$NAGIOS_SERVICESTATE" = "CRITICAL" && "$NAGIOS_SERVICESTATETYPE" = "HARD" ]]
then
        echo "conditions match"
        # insert event handler action here...
fi
Navigation
Print/export
Toolbox