Welcome to docs.opsview.com

Opsview Slaves

Within large organisations where there are large numbers of devices to monitor or in a distributed datacenter that can span continents, countries or cities, a single Opsview server can get bogged down trying to process all the necessary device checks. As a result, it may fall behind leading to delayed checks and notifications.

Also, networks that make use of secured zones or firewall rules to segregate devices and services should not have that security compromised by having to open significant numbers of ports to enable monitoring.

Adding Opsview slaves to the monitoring hierarchy can help to spread the load and reduce latency, as well as reduce network administration issues by ensuring that all devices can be adequately monitored.

Note: All slaves must be available when Opsview reloads in order for configuration to be synchronised. If a slave is not working, then the reload will fail. It is possible to temporarily deactivate a slave system to allow the reload to continue.

Note: Each slave instance is a separate instance of Nagios® Core, but has their state synchronised with the master. See the documentation for some known limitations.

Note: It is not possible to have notification suppression based on parent/child relationships for hosts outside of a slave, because slaves only know about their own hosts. If notifications are sent from the slave, there may be more notifications than if it was sent from the master.

Note: The OS time zone of the slave should match the master. If not, there could be issues with freshness checking for services that only run at certain times, as the master will be expecting results which the slaves will not send.

Note: There is a queuing system implemented on the slave to cater for a master failure or a network connection error. Any results older than 5 minutes will be dropped and a log entry in /usr/local/nagios/var/log/opsview-slave.log will be created. This can be changed by the slave_results_max_cache variable.


The NRD mechanism is described here.


Slave servers currently must have the same architecture and OS build as the master server (including prerequisite software).

Slave servers must also have their date and time in sync with the master. Slave servers must also have an SSH server listening on port 22.


On all slave servers, you should not install the main Opsview packages. The Opsview code will be pushed from the master server to each slave during initial configuration and upgrades. When creating slaves for Red Hat 7 Enterprise please make sure that the optional and extra repos are enabled on the server, this will allow opsview to install all the dependencies required for opsview slave. The repos can be enabled using the following command(s).

 subscription-manager repos --enable=rhel-7-server-extras-rpms --enable=rhel-7-server-optional-rpms

Further information about prerequisites is based on the platform:

Pre-install Tasks

These steps are to be performed on the new slave server, unless otherwise stated.

1. Create the nagios and nagcmd groups:

groupadd nagios
groupadd nagcmd

2. Create the nagios user and set its password to a known and secure value:

useradd -g nagios -G nagcmd -d /var/log/nagios -m nagios
passwd nagios

3. Ensure nagios user has root access for specific commands via sudo (note also the nagios user should have sudo on its PATH to use this correctly):

# Comment out the following line if it is set
#Defaults requiretty
# add the following line
nagios ALL = NOPASSWD: /usr/local/nagios/bin/install_slave

4. On the master server, copy the nagios SSH public key from the master to the slave server:

su - nagios
ssh-keygen -t dsa  # Creates your SSH public/private keys if they do not currently exist
ssh-copy-id -i ~/.ssh/id_dsa.pub {slave_hostname}

The .ssh directory should be mode 0700 and the id_dsa file should be 0600.

You should be able to connect to the slave server from the Opsview master server without passwords:

root@master$ su - nagios
nagios@master$ ssh {slave_hostname}
# Should be logged into slave system

5. Copy the check_reqs and profile scripts from the master onto the slave as the nagios user. This should work without prompting for authentication:

nagios@master$ scp /usr/local/nagios/installer/check_reqs /usr/local/nagios/bin/profile {slave}:

On the slave as the nagios user, source the profile then run check_reqs:

nagios@slave$ . ./profile
nagios@slave$ ./check_reqs slave

If above fails, fix any dependency issues listed. Specifically verify opsview-slave package is installed

6. Set up the profile for the nagios user on the slave server to be sourced at login:

nagios@slave$ echo "test -f /usr/local/nagios/bin/profile && . /usr/local/nagios/bin/profile" >> ~nagios/.profile
nagiso@slave$ chown nagios:nagios ~nagios/.profile

Note: The /usr/local/nagios/bin/profile will be installed a bit later on - this sets it up for future logins.

7. Create the temporary drop directory on the slave.

On the slave, create the temporary directory to put the transfer files:

su - root
mkdir -p /usr/local/nagios/tmp 
chown nagios.nagios /usr/local/nagios /usr/local/nagios/tmp 
chmod 755 /usr/local/nagios /usr/local/nagios/tmp

8. Check SSH TCP Port Forwarding on the slave.

In order to communicate with the master server, port forwarding must be enabled in /etc/ssh/sshd_config on the slave server. Ensure that the following option is set to yes (default is yes):

AllowTcpForwarding yes

Restart the SSH server if this is changed.

Setup of the slave

1. Within the master web interface, ensure that the slave host is set up on the master server:

  • Configuration ⇒ Hosts ⇒ Create New Host
  • Make sure you assign the Application - Opsview Common host template against the host.

The host used for the slaves must have at least 1 service associated with it, otherwise a configuration reload will fail.

2. Within the master web interface, add the slave host as a monitoring server via settings ⇒ Monitoring Servers ⇒ Create New Monitoring Server [icon]

3. From the master server, test the connection to the slave host:

su - nagios
/usr/local/nagios/bin/send2slaves -t [opsview slave node name] # Connection test

If you get an error like:

Host key verification failed.
Error connecting to opslave.opsera.com

This is likely to be that the ssh host key is not setup. Test with:

ssh opslave.opsera.com

Accept any host keys if required.

4.From the master server, send all the program and configuration files to the slave host:

/usr/local/nagios/bin/send2slaves [opsview slave node name]

The slave node name is optional and can be used when multiple slaves have been defined. In the example above, the slave node name is opslave.

The connection test should show:

Connected to opslave...

The send code step should produce an error:

Errors requiring manual intervention: 1

and will detail running the commands in the next step, which should be….

5. As root on the slave server, run the setup program:

cd /usr/local/nagios/tmp && ./install_slave

Note: The opsview-slave startup script is only required if you have a reverse ssh connection setup to your slave systems.

6. Within the master web interface, reload the Opsview configuration on the master server:

settings → Configuration → Reload

When this completes, the slave will automatically start the nagios daemon and will start sending data back to the master.

Upgrading Slaves

Slaves are upgraded as part of the master upgrade (though slaves must be in a working state at the time of the master upgrade).

You can resend the latest master files to the slave by running:

su - nagios
/usr/local/nagios/bin/send2slaves {slavename}

Note that the Opsview Agent configuration, nrpe.cfg, on the master is not sent to the slave, so you might want to compare any changes that have been made to it with its equivalent on each slave. See, for example, our recent changes to the Agent in 4.6.3.

Monitoring Slaves

Some services will be automatically created on the master to monitor each slave. These are called Slave-node: {hostname}. This service will run a plugin called check_opsview_slave_node which checks:

  • if all slaves are contactable
  • if their time is synchronised
  • if NSCA has errored
  • if the nagios daemon is running correctly

You should make sure that you will get alerts from this service as it will be the first warning of problems with a slave.

Note: In Opsview 2.14, this slave checking functionality was provided by check_opsview_slave assigned to the slaves. This is now redundant, so you can remove this service.

Slave failures

When a slave fails, this is the sequence of actions:

  • The Slave-node: {hostname} will go into a critical state - make sure you get notifications for this service
  • After 30 minutes, all hosts and services on the failed slave will go into an UNKNOWN stale state with the text of UNKNOWN: Service results are stale. We think this is reflective of the situation as the service states are not up to date, and there is a single failure that needs resolving

Note: All hosts monitored by that slave will not change state. It is not possible to set a freshness value on the host state as hosts are only checked on demand and thus will not have regular results sent to the Opsview master.

Note: While a slave is down, a reload will fail. This is because the reload expects to be able to communicate with all slave nodes.

If the slave is expected to be offline for a long period, you can disable the slave by marking it as not activated. If the slave is restarted independently, the nagios daemon on the slave will continue to run checks but results will not be received by the master. You may still get notifications from this slave server - remember to stop nagios!

Slave clusters

If more than one server is selected during configuration of monitoring server, a 'slave cluster' will be created. This will provide automatic load balancing between nodes within the cluster and all checks will automatically fail over if one of the nodes in that cluster fails.

See slave clusters for more information on setting up and configuring slave clusters.

Using The Slave

To make use of the slave host, amend each host configuration and set the Monitored By value to the correct server

Configuration->Hosts->{Edit Host}->Monitored By->{slave_host}

Alternatively, you can drag and drop the host between servers on the Monitoring servers list page.

Slave Web Interface

Opsview slaves need to have a web interface if you want to view NMIS or MRTG detailed interface data. You can also have a standard Nagios Core web interface.

However the web interface is not enabled by default. To enable it:

1. Add the apache web server user (i.e. the user that apache runs as) to the nagcmd group, in order to read the htpasswd file. On Debian, this is normally www-data - please check your system.

sudo usermod -G nagcmd www-data

2. Copy the Apache configuration from the master onto the slave:

Master source file: /usr/local/opsview-web/etc/apache_opsview_slave.conf Slave target file: /etc/apache2/sites-available/opsview_slave.conf

Edit the file oni the slave and tailor as necessary for your site (if required) and then enable the new configuration (the default configuration should not be needed).

sudo a2ensite opsview_slave
sudo a2dissite default

3. stop and start Apache (not restart as the new group membership needs to be activated)

sudo /etc/init.d/apache2 stop
sudo /etc/init.d/apache2 start

You should now be able to access the web pages on the slave using the same login credentials as the Opsview master server.

Limitation: If you use LDAP on the master, then you cannot login on to the slave with any user that has the LDAP realm.

If you have any issues with accessing the web interface, check the Apache logs for error messages.

Using Slaves with Reversed SSH Tunnels

The reverse ssh tunnels are useful if security policy only allow slaves to initiate conversations to the master. A tunnel is started from slave to master, and then the master is able to start new communications with slave as required.

Note: Reverse SSH functionality applies to all nodes of selected monitoring cluster - default setting is to use forward SSH tunnels.

Create new monitoring node/cluster or edit the existing one and set the SSH tunnel configuration option to Reverse tunnel. On the list page, hover over the slave node - a popup will appear with the slave_port_number.

You will need to access slave in some way for the initial install. Create the nagios groups and user and exchange SSH keys: the slave needs the public key of the master and the master needs public key of slave.

On the slave, run as nagios user:

ssh -N -R {slave_port_number}: {opsviewmaster}

This process will not exit on slave.

On the master run the following as the nagios user:

/usr/local/nagios/bin/dosh -i -s {slavename} uname -a

This commands checks the connectivity to slaves.

Install Opsview with send2slaves {slave_name}. If this is a new slave you will need to manually intervene as root - check instructions onscreen. If this is a pre-existing slave, rerun send2slaves {slave-name} again.

When installed, on the slave, stop the earlier ssh command. Test that this has been cancelled by running on the master and getting the expected error:

$ /usr/local/nagios/bin/dosh -i -s {name} uname -a
ssh: connect to host port 25801: Connection refused

On the slave, create /usr/local/nagios/etc/opsview-slave.conf with contents of:


Test from the slave with:

/etc/init.d/opsview-slave test
/etc/init.d/opsview-slave start
/etc/init.d/opsview-slave status

Now repeat these steps for all slaves in Opsview.

On master, use dosh uname -a to test connectivity to all slaves.

All communications with master should now work correctly and a reload can now be performed.

Note: If you change a slave from reverse to forward, you must remove the opsview-slave.conf file so that Opsview will not attempt to restart the tunnel.

Changing the base port number

Reverse SSH requires starting from a specific TCP port number. However, this may clash with other applications, so this value is configurable. The default value is 28500 and the actual port used will be the base port + the slave nodes' id.

To change the base port, on the Opsview Master, edit opsview.conf and set:

$slave_base_port = 11100;

Restart opsview and opsview-web.

You will then need to change the port number in the opsview-slave.conf file on all your slave nodes.


When you run on the master dosh uname -a and you get the error:

Host key verification failed

Then you probably haven't created all the necessary host keys. For that particular slave, run:

/usr/local/nagios/bin/dosh -i -s {slavename} uname -a

This will prompt for password if the SSH key exchange hasn't been done yet. It will also automatically add the host key if not already done.


These are some common errors that may occur as part of installing slaves.

perl: undefined symbol

If you get this error while installing Opsview on the slave:

/usr/bin/perl: symbol lookup error: /usr/local/nagios/perl/lib/i486-linux-gnu-thread-multi/auto/Time/HiRes/HiRes.so: undefined symbol: Perl_Istack_sp_ptr

This can be due to different levels of perl between the master and the slave. There is a restriction that the OS distribution must be at the same level between master and slave.

ssh: Connection timed out during banner exchange

This appears if there is too much load on the slave. Try again. Keep an eye on resources as you do not want this to occur often.

Missing prerequisites - libexpat is not installed

When running check_reqs slave, your environment may be identified as missing the libexpat XML parsing libraries, even if you already have the libexpat or libexpat1 packages installed.

The solution to this is to also install the dev version of this package.

When using apt the package name is libexpat-dev or libexpat1-dev packages, depending upon your OS version.

When using yum the package name is expat-devel.

Plugin output is not preserved completely on Master view

Plugins run on the slaves are passed back to the master via NRD. The flow is:

  1. Plugin executed via slave
  2. Result stored as a Nagios performance result using the performance template
  3. File moved to /usr/local/nagios/var/slaveresults
  4. File picked up by import_slaveresultsd
  5. Results sent to NRD on the master
  6. NRD places the result into Nagios' checkresult directory
  7. Nagios reads these results from the checkresult directory

However, at step 2, the output is passed through a filter where certain characters are removed. Also linefeeds are changed to \n and all backslashes are converted to a double backslash. Because of this, the output will not be exactly the same between master and slave.

Suppressing Message of the Day/Login Banner

To suppress login banners while connecting from Opsview Master to Slave servers (which will be logged in opsviewd.log as they are printed to STDERR) suggested approach when the banners cannot be disabled is to create a ~nagios/.hushlogin:

touch ~nagios/.hushlogin