check_update over nrpe #44
Comments
|
What happens if you run dnf check-update manually as "nrpe" and as "root"? |
|
As nrpe: [nrpe@mx1 ~]$ dnf check-update bpftool.x86_64 5.14.3-1.el8.elrepo elrepo-kernel As root: [root@mx1 ~]# yum check-update bpftool.x86_64 5.14.3-1.el8.elrepo elrepo-kernel |
|
I don't get it ... It seems that the process is not run as "nrpe" and not as "root" ... I don't use NRPE. What happens if you specify the |
|
Client side: [root@mon ssl]# /usr/lib64/nagios/plugins/check_nrpe -t 60 -H mx1.la1.clx.corp --key-file=mon.la1.clx.corp.key --client-cert=mon.la1.clx.corp.cer --ca-cert-file=/etc/pki/tls/certs/blah-chain.pem -c check_updates NRPE Server side: [1631704204] CONN_CHECK_PEER: checking if host is allowed: 192.168.30.76 port 48771 |
|
I just committed a new version with some more debugging output. Would it be possible to test again? |
matteocorti
added
the
need more info
|
[1631705217] CONN_CHECK_PEER: checking if host is allowed: 192.168.30.76 port 3204 |
|
I am experiencing this bug as well. Worth noting that it only happens to my CentOS/RHEL 8 systems, those on 7.x work as expected. check_updates works fine when run standalone, as root or the nrpe user. When run via check_nrpe from the local system or our monitoring server it bombs with the "CHECK_UPDATES UNKNOWN - Missing GPG key, run "dnf check-update" manually" error. Also, while troubleshooting I saw exactly the same 'Importing GPG key' line from @xxedgexx 's output. When I disabled the third-party repos using this key (remi-safe and remi-modular), the check_nrpe command started working. Obviously that's not a viable long-term solution, though. |
|
Thanks for the feedback ... I'll try to take a look again |
|
The warning about |
|
What happens when running as user |
|
It appears to work, as far as I can see. (This is the output via the nrpe user on RHEL 8.) Not root, Subscription Management repositories not updated |
|
I really why |
|
I get the same output as running from a bash prompt. |
|
Mmmm I really don't get why it's behaving differently when run through nrpe. I'll try to set it up with a server of mine (I usually check with ssh ...). |
|
@jonmilby I installed NRPE on a CentOS 8 machine with the following command And got no issues ... Even by the first run. And I have the GPG check set to 1 globally |
|
@matteocorti I tried this as well, and it works as you said. The issue only occurs once I install the Remi repo RPM (dnf -y install https://rpms.remirepo.net/enterprise/remi-release-8.rpm) to install other versions of PHP than those provided by the system repos. None of the other repos has repo_gpgcheck defined at all, but these do and that seems to be the trouble. Here's one of the repo definitions (/etc/yum.repos.d/remi-safe.repo): [remi-safe] On this fresh install, running as the nrpe user, both running check_updates directly and via NRPE fail once the repo is added. If I run 'dnf check-update' as the nrpe user, it asks me to accept the GPG keys for the repos. Once I do that, running check_updates directly works as expected. However, running via check_nrpe still throws the GPG key error. If I edit remi-safe.repo and remi-modular.repo to set repo_gpgcheck=0, the NRPE check works as it did with my other system. |
|
Thanks I installed the RPM and got the same error. Then: If I run But I get the message in Nagios (as you do ...) |
|
In And executing the command: |
|
The plugin is not (?) running as user |
|
SELinux is not the problem: |
|
Still puzzled: with No log file is generated... |
|
By running |
|
By manually removing a check in the plugin I was able to get |
|
|
As I really did not find out why the import has to be repeated when running with nrpe with this repository I added (fd4aa38 |
|
Possibly part of the problem is that when NRPE runs, because it runs as a daemon, the environment is empty, so PATH and HOME in particular are missing. This affects the use of the 'which' command line (i.e. get_path()), which won't find commands, and GPG won't find keys. Given that daemons are supposed to run without environment variables, it seems like either we simply can't run check_updates at all through NRPE, or we need a backup plan to set those environment variables if they're missing, and we expect there to be some. I'm having the current problem of check_updates being unable to find dnf or yum due to this difference. (Replicate on command line by prefixing your invocation with "env -i".) |
|
NRPE config: command[check_updates]=/usr/lib64/nagios/plugins/check_updates -t 60 --assumeyes On the Nagios server: /usr/lib64/nagios/plugins/check_nrpe -t 60 -H mx1 --key-file=/etc/nagios/ssl/mon.key --client-cert=/etc/nagios/ssl/mon.cer --ca-cert-file=/etc/pki/tls/certs/chain.pem -c check_updates On the client side: [1642759033] CONN_CHECK_PEER: checking if host is allowed: 192.168.30.76 port 30914 |
|
The only way I could get it to work is to run the command under sudo: command[check_updates]=sudo /usr/lib64/nagios/plugins/check_updates -t 60 --assumeyes Not ideal but maybe this lends to what the issue could be running it as the nrpe user? |
I applied your patch, but this is most likely not the problem. When the script is executed with NRPE the
|
|
I'm running RHEL 8, and the PATH is blank (RPM version 4.0.3 - nrpe-4.0.3-6.el8.x86_64) |
I'd recommend setting your timeout (both in nrpe.cfg and in the nagios client) to 120 or 150. That first time through can take quite a bit of time. Subsequent invocations are nicely quick. Also, make sure that your SElinux is either off or the context for the check_updates command is set to 'unconfined_u:object_r:nagios_unconfined_plugin_exec_t:s0' (the default on RHEL 8). |
Yes I implemented the standard 15s recommended as default for Nagios plugins. For check_updates it's too low. |
|
Mmm I am on CentOS 8 ... I will try to get a RHEL 8 machine to test |
Yes, unfortunately, Red Hat dropped continuing support for CentOS 8 at the end of 2021, which is why I've been working on getting this to work on RHEL 8 (Developer license). |
|
There’s also Rocky Linux, which is the new CentOS. That’s what I’m using.
… On Saturday, Jan 22, 2022 at 6:40 AM, Tilghman Lesher ***@***.*** ***@***.***)> wrote:
>
>
> Mmm I am on CentOS 8 ... I will try to get a RHEL 8 machine to test
>
Yes, unfortunately, Red Hat dropped continuing support for CentOS 8 at the end of 2021, which is why I've been working on getting this to work on RHEL 8 (Developer license).
—
Reply to this email directly, view it on GitHub (#44 (comment)), or unsubscribe (https://github.com/notifications/unsubscribe-auth/ADEA2IPIHNIGQZWZKUYXHP3UXK6VHANCNFSM5D2T5KOA).
You are receiving this because you were mentioned.Message ID: ***@***.***>
|
|
I am not sure if the issue should be followed up as nrpe is now deprecated (https://github.com/NagiosEnterprises/nrpe) |
|
Initial tests with ncpa give me good results. I didn't have to do anything special outside of making the check_updates command available in NCPA's plugins directory. Still playing around but I get out of date package results. |
|
No feedback. NRPE is deprecated. |
|
We see the same problem when running checks over SSH with Icinga, and generally when running the script under a separate user that isn't root ( |
|
The command needs --nogpgcheck added to the arguments. Once that's there, the command runs correctly and detects when new packages need to be installed, per the intended functionality. |
|
But then I can't check for real GPG key problems when a repository owner actually rotates a key. Broken GPG handling can't be intended behaviour. |
|
Why does a changed key matter to you, unless there are packages to be updated? When there are new packages, it alerts you to the fact, and you'll see the GPG key conflict when you login to the affected machine and run the update command. This is a check update command, not a run update command. |
|
It's a check update command that cannot properly check for authentic updates if it can't properly handle GPG keys. Why would I ever want to voluntarily disable that security check, even if it's "just" to check updates? If you disregard security that much I'll just replace your plugin with something safer. |
|
Sure, go ahead, replace my plugin. I won't hold my breath. |
|
We could compromise and add a command line option like |
|
That works for me |
|
I just remembered that YUM or DNF arguments can be specified with the The workaround would then be: |
Describe the bug
/usr/lib64/nagios/plugins/check_nrpe -t 60 -H mx1.la1.blah.corp --key-file=mon.la1.blah.corp.key --client-cert=mon.la1.blah.corp.cer --ca-cert-file=/etc/pki/tls/certs/BlahCA-chain.pem -c check_updates
CHECK_UPDATES UNKNOWN - Missing GPG key, run "dnf check-update" manually
NRPE server side:
[1631340898] CONN_CHECK_PEER: checking if host is allowed: 192.168.30.76 port 28316
[1631340898] Connection from 192.168.30.76 port 28316
[1631340898] is_an_allowed_host (AF_INET): is host >192.168.30.76< an allowed host >192.168.30.76<
[1631340898] is_an_allowed_host (AF_INET): host is in allowed host list!
[1631340898] Host address is in allowed_hosts
[1631340898] Host 192.168.30.76 is asking for command 'check_updates' to be run...
[1631340898] Running command: /usr/lib64/nagios/plugins/check_updates
[1631340898] WARNING: my_system() seteuid(0): Operation not permitted
System (please complete the following information):
Additional context/output
I'm able to run as the nrpe user. I'm assuming this is some kind of tty allocation permission thing? Maybe a sudo fix?
[nrpe@mx1 ~]$ /usr/lib64/nagios/plugins/check_updates -d -v
[DBG] check_updates version: 1.8.4
[DBG] system info: Linux mx1.la1.blah.corp 5.13.12-1.el8.elrepo.x86_64 #1 SMP Tue Aug 17 10:51:25 EDT 2021 x86_64 x86_64 x86_64 GNU/Linux
[DBG] NAME="Rocky Linux"
VERSION="8.4 (Green Obsidian)"
ID="rocky"
ID_LIKE="rhel fedora"
VERSION_ID="8.4"
PLATFORM_ID="platform:el8"
PRETTY_NAME="Rocky Linux 8.4 (Green Obsidian)"
ANSI_COLOR="0;32"
CPE_NAME="cpe:/o:rocky:rocky:8.4:GA"
HOME_URL="https://rockylinux.org/"
BUG_REPORT_URL="https://bugs.rockylinux.org/"
ROCKY_SUPPORT_PRODUCT="Rocky Linux"
ROCKY_SUPPORT_PRODUCT_VERSION="8"
Checking a linux system
Running on Rocky Linux release 8.4 (Green Obsidian)
[DBG] Updater: yum
Using Yum or DNF
running a Linux kernel: 5.13.12-1
kernel: running = 5.13.12-1, installed = 5.13.12-1
[DBG] Running "/usr/bin/dnf check-update --assumeno --errorlevel=0 -q 2>&1"
[DBG]
[DBG] bpftool.x86_64 5.14.2-1.el8.elrepo elrepo-kernel
[DBG] cabextract.x86_64 1.9-7.epel8.playground epel-playground
[DBG] certbot.noarch 1.18.0-2.el8 epel-testing
[DBG] clamav.x86_64 0.103.3-5.epel8.playground epel-playground
[DBG] clamav-data.noarch 0.103.3-5.epel8.playground epel-playground
[DBG] clamav-filesystem.noarch 0.103.3-5.epel8.playground epel-playground
[DBG] clamav-lib.x86_64 0.103.3-5.epel8.playground epel-playground
[DBG] clamav-update.x86_64 0.103.3-5.epel8.playground epel-playground
[DBG] clamd.x86_64 0.103.3-5.epel8.playground epel-playground
[DBG] epel-release.noarch 8-13.el8 epel-testing
[DBG] kernel.x86_64 4.18.0-305.17.1.el8_4 baseos
[DBG] kernel-core.x86_64 4.18.0-305.17.1.el8_4 baseos
[DBG] kernel-ml.x86_64 5.14.2-1.el8.elrepo elrepo-kernel
[DBG] kernel-ml-core.x86_64 5.14.2-1.el8.elrepo elrepo-kernel
[DBG] kernel-ml-modules.x86_64 5.14.2-1.el8.elrepo elrepo-kernel
[DBG] kernel-modules.x86_64 4.18.0-305.17.1.el8_4 baseos
[DBG] kernel-tools.x86_64 4.18.0-305.17.1.el8_4 baseos
[DBG] kernel-tools-libs.x86_64 4.18.0-305.17.1.el8_4 baseos
[DBG] libprelude.x86_64 5.2.0-1.epel8.playground epel-playground
[DBG] p7zip.x86_64 16.02-20.epel8.playground epel-playground
[DBG] p7zip-plugins.x86_64 16.02-20.epel8.playground epel-playground
[DBG] php-common.x86_64 7.4.23-1.el8.remi remi-modular
[DBG] php-fpm.x86_64 7.4.23-1.el8.remi remi-modular
[DBG] php-gd.x86_64 7.4.23-1.el8.remi remi-modular
[DBG] php-gmp.x86_64 7.4.23-1.el8.remi remi-modular
[DBG] php-imap.x86_64 7.4.23-1.el8.remi remi-modular
[DBG] php-intl.x86_64 7.4.23-1.el8.remi remi-modular
[DBG] php-json.x86_64 7.4.23-1.el8.remi remi-modular
[DBG] php-mbstring.x86_64 7.4.23-1.el8.remi remi-modular
[DBG] php-mysqlnd.x86_64 7.4.23-1.el8.remi remi-modular
[DBG] php-opcache.x86_64 7.4.23-1.el8.remi remi-modular
[DBG] php-pdo.x86_64 7.4.23-1.el8.remi remi-modular
[DBG] php-pecl-zip.x86_64 1.19.4-1.el8.remi.7.4 remi-modular
[DBG] php-pgsql.x86_64 7.4.23-1.el8.remi remi-modular
[DBG] php-xml.x86_64 7.4.23-1.el8.remi remi-modular
[DBG] python3-acme.noarch 1.18.0-1.el8 epel
[DBG] python3-certbot.noarch 1.18.0-2.el8 epel-testing
[DBG] python3-perf.x86_64 5.14.2-1.el8.elrepo elrepo-kernel
Security updates available
[DBG] Running "/usr/bin/dnf check-update --assumeno --errorlevel=0 -q --security 2>&1 2>&1"
[DBG]
[DBG] bpftool.x86_64 5.14.2-1.el8.elrepo elrepo-kernel
[DBG] kernel.x86_64 4.18.0-305.17.1.el8_4 baseos
[DBG] kernel-core.x86_64 4.18.0-305.17.1.el8_4 baseos
[DBG] kernel-modules.x86_64 4.18.0-305.17.1.el8_4 baseos
[DBG] kernel-tools.x86_64 4.18.0-305.17.1.el8_4 baseos
[DBG] kernel-tools-libs.x86_64 4.18.0-305.17.1.el8_4 baseos
[DBG] p7zip.x86_64 16.02-20.epel8.playground epel-playground
[DBG] p7zip-plugins.x86_64 16.02-20.epel8.playground epel-playground
[DBG] python3-perf.x86_64 5.14.2-1.el8.elrepo elrepo-kernel
Security updates available
no security plugin: every update could be a security problem
CHECK_UPDATES CRITICAL - 9 security updates and 29 non-security updates available | total_updates=38;0;0 security_updates=9;0;0
bpftool.x86_64 (security)
kernel.x86_64 (security)
kernel-core.x86_64 (security)
kernel-modules.x86_64 (security)
kernel-tools.x86_64 (security)
kernel-tools-libs.x86_64 (security)
p7zip.x86_64 (security)
p7zip-plugins.x86_64 (security)
python3-perf.x86_64 (security)
cabextract.x86_64
certbot.noarch
clamav.x86_64
clamav-data.noarch
clamav-filesystem.noarch
clamav-lib.x86_64
clamav-update.x86_64
clamd.x86_64
epel-release.noarch
kernel-ml.x86_64
kernel-ml-core.x86_64
kernel-ml-modules.x86_64
libprelude.x86_64
php-common.x86_64
php-fpm.x86_64
php-gd.x86_64
php-gmp.x86_64
php-imap.x86_64
php-intl.x86_64
php-json.x86_64
php-mbstring.x86_64
php-mysqlnd.x86_64
php-opcache.x86_64
php-pdo.x86_64
php-pecl-zip.x86_64
php-pgsql.x86_64
php-xml.x86_64
python3-acme.noarch
python3-certbot.noarch
The text was updated successfully, but these errors were encountered: