How to know if distributed pollers work ?

[sk4mi]

Hello !

I'm trying to install/deploy librenms with distributed pollers on docker.

my docker-compose.yml:

note that i work offline, so i'd need to pull yours images and push them into internal artifact.
I also delete traefik service, smtp and db who is install somewhere else.
the original docker-compose.yml from example works nice.

docker stats:

CONTAINER ID        NAME                 CPU %               MEM USAGE / LIMIT     MEM %               NET I/O             BLOCK I/O           PIDS
3a43fd9b417d        librenms-poller-1    0.12%               36.14MiB / 15.51GiB   0.23%               9.53kB / 4.89kB     0B / 98.3kB         270
f1d9815f8096        librenms_syslog      0.00%               9.488MiB / 15.51GiB   0.06%               9.58kB / 3.07kB     0B / 98.3kB         2
85769044c815        librenms_cron        0.00%               52KiB / 15.51GiB      0.00%               282MB / 203MB       0B / 164kB          1
000e45ddd26b        librenms             0.04%               64.01MiB / 15.51GiB   0.40%               7.54GB / 68.5MB     0B / 11.3MB         274
f706dacde751        librenms_memcached   0.02%               4.371MiB / 15.51GiB   0.03%               873kB / 625kB       0B / 0B             10
26b57f01dc44        librenms_rrdcached   0.00%               11.09MiB / 15.51GiB   0.07%               59.8MB / 7.42GB     0B / 0B             7

docker ps:

CONTAINER ID        IMAGE                                                                                   COMMAND                  CREATED             STATUS              PORTS                                                  NAMES
3a43fd9b417d        docker-interne.artifactory.example.com/internal/librenms:librenms-1.47-internal   "/entrypoint.sh /usr…"   2 hours ago         Up 2 hours          80/tcp, 514/tcp, 514/udp                               librenms-poller-1
f1d9815f8096        docker-interne.artifactory.example.com/internal/librenms:librenms-1.47-internal   "/entrypoint.sh /usr…"   2 hours ago         Up 2 hours          80/tcp, 0.0.0.0:514->514/tcp, 0.0.0.0:514->514/udp   librenms_syslog
85769044c815        docker-interne.artifactory.example.com/internal/librenms:librenms-1.47-internal   "/entrypoint.sh /usr…"   2 hours ago         Up 2 hours          80/tcp, 514/tcp, 514/udp                               librenms_cron
000e45ddd26b        docker-interne.artifactory.example.com/internal/librenms:librenms-1.47-internal   "/entrypoint.sh /usr…"   2 hours ago         Up 2 hours          514/tcp, 514/udp, 0.0.0.0:8080->80/tcp                 librenms
f706dacde751        docker-interne.artifactory.example.com/external/memcached:1.5.12-alpine           "docker-entrypoint.s…"   2 hours ago         Up 2 hours          11211/tcp                                              librenms_memcached
26b57f01dc44        docker-interne.artifactory.example.com/external/rrdcached:1.7.0                   "/entrypoint.sh /usr…"   2 hours ago         Up 2 hours          42217/tcp                                              librenms_rrdcached

librenms.env:

MEMORY_LIMIT=256M
UPLOAD_MAX_SIZE=16M
OPCACHE_MEM_SIZE=128

TZ=Europe/Paris
PUID=1000
GUID=1000

LIBRENMS_POLLER_THREADS=16
LIBRENMS_POLLER_INTERVAL=5

LIBRENMS_DISTRIBUTED_POLLER_ENABLE=true

LIBRENMS_CRON_DISCOVERY_ENABLE=false
LIBRENMS_CRON_DAILY_ENABLE=false
LIBRENMS_CRON_ALERTS_ENABLE=true
LIBRENMS_CRON_BILLING_ENABLE=false
LIBRENMS_CRON_BILLING_CALCULATE_ENABLE=false
LIBRENMS_CRON_CHECK_SERVICES_ENABLE=true
LIBRENMS_CRON_POLLER_ENABLE=true

LIBRENMS_SNMP_COMMUNITY=librenmsdocker
MEMCACHED_HOST=memcached
MEMCACHED_PORT=11211
RRDCACHED_HOST=rrdcached
RRDCACHED_PORT=42217

Validate:

[root@ntest-librenms librenms]# /usr/local/bin/docker-compose exec --user librenms librenms php validate.php
====================================
Component | Version
--------- | -------
LibreNMS  | 1.47
DB Schema | 275
PHP       | 7.2.13
MySQL     | 5.5.44-MariaDB-log
RRDTool   | 1.7.0
SNMP      | NET-SNMP 5.7.3
====================================

[OK]    Composer Version: 1.8.0
[OK]    Dependencies up-to-date.
[OK]    Database connection successful
[OK]    Database schema correct
[WARN]  IPv6 is disabled on your server, you will not be able to add IPv6 devices.
[WARN]  Your install is over 24 hours out of date, last update: Sun, 30 Dec 2018 14:29:16 +0000
	[FIX]:
	Make sure your daily.sh cron is running and run ./daily.sh by hand to see if there are any errors.
[WARN]  Your local git branch is not master, this will prevent automatic updates.
	[FIX]:
	You can switch back to master with git checkout master

---

We can see all is up and working, but when i check on web gui i just see one poller (cron container).
On stats, librenms container have some high net i/o but not the other poller.

How can i check if its broken somewhere ?

[jsenecal]

Do you see anything in the Devices Polled counter in the Poller (/pollers) status page ?

[sk4mi]

Yes I get devices polled for cron-librenms.rzo.example.com.
But there is only one poller. I don’t see the other one.

[jsenecal]

How many side-car cron workers do you have running ? Le mar. 22 janv. 2019, à 03 h 51, sk4mi <notifications@github.com> a écrit :

…

Yes I get devices polled for cron-librenms.rzo.example.com. But I there is only one poller. Sent with GitHawk <http://githawk.com> — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#14 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABfg5lc2f_zYG0opRtcE3iNMhTfsBrahks5vFtECgaJpZM4aBqFV> .

-- - Jonathan

[sk4mi]

I launched one cron sidecar.

If I run X pollers, I have to run same number of cron sidecar ?

_{Sent with GitHawk}

[sk4mi]

I managed to launch distributed pollers, but i needed to delete cron sidecar and add cron config on each poller.

There is an other way ?

_{Sent with GitHawk}

[jsenecal]

Yes, you need multiple sidecar cron containers to have multiple pollers. They are configured using environment variables as shown in the README but custom config files (for librenms specific configs) can also be used.

The part that handles poller-wrapper in crontab is here: entrypoint.sh#L276

[jsenecal]

@crazy-max This is a request for help and as such should be closed as it does not represent an actual issue in the code.

[crazy-max]

Yes thx @jsenecal