Monitoring remote host with nrpe

[Kevlys]

Hi,

I don't understand how I can monitor some remote host with nrpe. I have tried many things, but nothing seems to work for me. Note that I'm a beginner on Shinken or Nagios, sorry if the question is basic.

I created a file containing my remote host config in custom_configs/miscellaneous/my-remote-host.cfg :

define host {
    host_name            my-remote-host.com
    use                        generic-host
    alias                      my-remote-host.com
    address                 my-remote-host.com
}

But I don't know how I can define my services to be able to work with nrpe.

I tried something like this (with many variants)

define service {
    use             generic-service
    host_name           my-remote-host.com
    service_description     TEST REMOTE
    check_command            check_nrpe!check_load
}

But I have a not found error. check_nrpe is missing, and after search, it seems to be normal, I should use booster-nrpe instead.

Ok, so, I tried somethings like this :

define module{
     module_name       booster-nrpe
     module_type       nrpe_poller
}

define command {
   command_name   check_nrpe
   command_line  $NAGIOSPLUGINSDIR$/check_nrpe -H $HOSTADRESS$ -c $ARG1$ -a $ARG2$
   module_type    nrpe_poller
}

Source

But I have the same error. (Maybe I don't write these lines at the right location ?)

Am I missing something ?

Thanks.

[rohit01]

Yes. You need to install: nagios-nrpe-server & nagios-plugins on every host/client -- you want to monitor using shinken. For example, it will be 'my-remote-host.com' from the example above.

It is available for install via standard package managers, like apt, yum, etc. Sample deb package link: https://packages.debian.org/squeeze/nagios-nrpe-server

Once installed, start nagios-nrpe-server service. It will listen on TCP port 5666. The configuration file for NRPE is: /etc/nagios/nrpe.cfg. All NRPE commands are written here. It is the "-c $ARG1$ " part from your shinken command definition. The default installation directory for NRPE plugins is: /usr/lib/nagios/plugins/ (for most systems).

For adding your own commands, you can modify /etc/nagios/nrpe_local.cfg (there is a include statement for this file in the main nrpe.cfg file). This will keep your main configuration clean and help you track changes better.

[rohit01]

This is NRPE based monitoring, just like Nagios does (by installing an agent on all clients). Agentless monitoring is also possible using ssh. If you want to explore all these, visit:
http://shinkenlab.io/
It contains a bunch of cool online courses (screencast tutorials). You may be interested in this one: http://shinkenlab.io/online-course-5-monitoring-plugins/

Please note: https://www.monitoring-plugins.org/ is a fork of nagios NRPE plugins. It contains latest versions of plugins. However, easy to install linux packages are not available. You need to compile from source code.

[Kevlys]

First, thanks for your answer and your advices,

I have not mentioned that I had already installed nagios-nrpe-server & nagios-plugins on client, sorry.

it seems that my mistake was that I named the command check_nrpe, the "native" check_nrpe command (locate at /etc/shinken/commands/check_nrpe.cfg on the docker image) was called instead of mine. this explains the "not found" error.

I update my nrpe command like this :

define command{
  command_name     cmd_check_nrpe
  command_line     $NAGIOSPLUGINSDIR$/check_nrpe -H $HOSTADDRESS$ -n -c $ARG1$ -a $ARG2$ $ARG3$ $ARG4$
  module_type      nrpe_poller
}

now my server send command to my remote host (note I have to disable ssl with -n parameter, else nothing is happening, I do not know why yet), but I have new error : Empty response from the NRPE server

On me remote host syslog file, I have :

Host address is in allowed_hosts
Handling the connection...
Error: Could not complete SSL handshake. 1
Connection from xxx.xxx.xxx.xxx closed.

I saw on some forums that was a known problem, I have to look at it more closely.

I'm just wondering if this error is not caused by the -n parameter. do you have any idea about this ?

Thanks

[rohit01]

Is the IP of shinken server whitelisted at nagios-nrpe-server configuration: nrpe.cfg or nrpe_local.cfg?
Check for allowed_hosts parameter. By default it is configured to allow only localhost.
allowed_hosts=127.0.0.1

disable ssl with -n

I don't think this is required. I never used this.

[rohit01]

Oops, didn't saw that you mentioned 'Host address is in allowed_hosts'.

Try dropping 'module_type nrpe_poller' from command definition. It is used to enable booster-nrpe. You can try adding it later, once it works. Example configuration:

define command{
  command_name     cmd_check_nrpe
  command_line     $NAGIOSPLUGINSDIR$/check_nrpe -H $HOSTADDRESS$ -c $ARG1$ -a $ARG2$ $ARG3$ $ARG4$
}

[Kevlys]

I have [Errno 2] No such file or directory again on the Thruk interface.

it seemed normal to me because there is no check_nrpe plugin file in docker image, am I wrong ?

This is why I have tried to configured the booster-nrpe.

Thanks for help

[rohit01]

Humm... Added check_nrpe plugin in the docker image:
320f25f
Build triggered!

[rohit01]

Did a full test and found the issue. Possible reasons:

1. You are running docker on the same host which you want to monitor. The source IP of the docker container is private and dynamic. It reaches the nrpe host via docker0 network (check ifconfig output). Solution - Allow private source ip addresses. Something like this:

   allowed_hosts=127.0.0.1,172.16.0.0/12

2. Your sample configuration contains NRPE arguments. This is by default disabled in NRPE server. Solution - enable arguments:

   dont_blame_nrpe=1

---

On the server side, your shinken configuration can be something like this:

$ cat custom_configs/my_host_and_service.cfg

define command{
    command_name               cmd_check_nrpe
    command_line               $NAGIOSPLUGINSDIR$/check_nrpe -H '$HOSTADDRESS$' -c '$ARG1$' -a '$ARG2$' '$ARG3$' '$ARG4$'
}

define host {
    host_name                  myhost.example.com
    use                        generic-host
    alias                      myhost.example.com
    address                    myhost.example.com
}

define service {
    service_description        check_users
    use                        generic-service
    host_name                  myhost.example.com
    check_command              cmd_check_nrpe!check_users
}
define service {
    service_description        check_load
    use                        generic-service
    host_name                  myhost.example.com
    check_command              cmd_check_nrpe!check_load
}
define service {
    service_description        check_hda1
    use                        generic-service
    host_name                  myhost.example.com
    check_command              cmd_check_nrpe!check_hda1
}
define service {
    service_description        check_zombie_procs
    use                        generic-service
    host_name                  myhost.example.com
    check_command              cmd_check_nrpe!check_zombie_procs
}
define service {
    service_description        check_total_procs
    use                        generic-service
    host_name                  myhost.example.com
    check_command              cmd_check_nrpe!check_total_procs
}

---

Client side (myhost.example.com) changes:

$ sudo apt-get install nagios-nrpe-server 
$ sudo echo 'allowed_hosts=127.0.0.1,172.16.0.0/12' > /etc/nagios/nrpe_local.cfg
$ sudo echo 'dont_blame_nrpe=1' >> /etc/nagios/nrpe_local.cfg
$ sudo service nagios-nrpe-server restart

---

Let me know, if this solved the issue. Happy monitoring :)

[Kevlys]

It's perfect now,

after pulling the docker image, my command with 4 args worked correctly, but not the others. after removing args parameters, all works fine. I will define commands args on the remote host.

Now, It remains for me to learn more about Shinken / Nagios monitoring.

Thank you very much for your time.