Skip to content

Pulse Docker Monitoring brittle/broken API tokens #740

@bossanova808

Description

@bossanova808

Describe the bug
I have found Docker monitoring very brittle with recent upgrades and quite difficult to restore when broken.

Somewhere along the way V3 -> V4 all my Docker monitoring broke - with the change to needing individual tokens I think.

Trying to restore Docker monitoring since then has been...quite confusing.

The instructions are not good/clear (e.g.
Go to 'Pulse -> Docker and allow re-adding' or similar (...but this does not actually exist in the UI).
or
Generate a token with Container Manage privileges (...but does not point out this must be manually done in the API tokens area rather than the obvious 'generate token' in the Docker area)

Once that is worked out and manually done, things were still not working - due to permissions errors with the re-installed agent:

> sudo journalctl -u pulse-docker-agent -f

Nov 22 09:08:11 pihole (er-agent)[1045620]: pulse-docker-agent.service: Failed to execute /usr/local/bin/pulse-docker-agent: Permission denied
Nov 22 09:08:11 pihole (er-agent)[1045620]: pulse-docker-agent.service: Failed at step EXEC spawning /usr/local/bin/pulse-docker-agent: Permission denied
Nov 22 09:08:11 pihole systemd[1]: pulse-docker-agent.service: Main process exited, code=exited, status=203/EXEC
Nov 22 09:08:11 pihole systemd[1]: pulse-docker-agent.service: Failed with result 'exit-code'.
^C

> cd /usr/local/bin/

> l pulse-docker-agent
-rwxr-x--- 1 root root 7459000 Nov 21 22:00 pulse-docker-agent

> sudo chmod a+x pulse-docker-agent
> sudo journalctl -u pulse-docker-agent -f
Nov 22 09:08:42 pihole pulse-docker-agent[1046343]: {"level":"debug","component":"pulse-docker-agent","host":"unix:///var/run/docker.sock","runtime":"docker","time":"2025-11-22T09:08:42+11:00","message":"Connected to container runtime"}
Nov 22 09:08:42 pihole pulse-docker-agent[1046343]: {"level":"info","component":"pulse-docker-agent","runtime":"docker","daemon_host":"unix:///var/run/docker.sock","version":"29.0.2","time":"2025-11-22T09:08:42+11:00","message":"Connected to container runtime"}
Nov 22 09:08:42 pihole pulse-docker-agent[1046343]: {"level":"info","pulse_url":"http://monitor:7655","interval":30000,"time":"2025-11-22T09:08:42+11:00","message":"Starting Pulse Docker agent"}
Nov 22 09:08:44 pihole pulse-docker-agent[1046343]: {"level":"debug","component":"pulse-docker-agent","container_id":"9e78024c8bb6","cpu_percent":2.0534250185514225,"time":"2025-11-22T09:08:44+11:00","message":"CPU calculated from PreCPUStats"}
Nov 22 09:08:44 pihole pulse-docker-agent[1046343]: {"level":"debug","component":"pulse-docker-agent","container":"pihole","writableLayerBytes":111104061,"rootFilesystemBytes":224963247,"mountCount":2,"time":"2025-11-22T09:08:44+11:00","message":"Collected container disk metrics"}
Nov 22 09:08:44 pihole pulse-docker-agent[1046343]: {"level":"debug","component":"pulse-docker-agent","container_id":"2b7cc7313a4a","cpu_percent":0.40305203984103205,"time":"2025-11-22T09:08:44+11:00","message":"CPU calculated from PreCPUStats"}
Nov 22 09:08:44 pihole pulse-docker-agent[1046343]: {"level":"debug","component":"pulse-docker-agent","container":"orb-homeserver25","writableLayerBytes":0,"rootFilesystemBytes":26283649,"mountCount":1,"time":"2025-11-22T09:08:44+11:00","message":"Collected container disk metrics"}
Nov 22 09:08:44 pihole pulse-docker-agent[1046343]: {"level":"debug","component":"pulse-docker-agent","container_id":"b02abb906635","cpu_percent":0.15363870865200102,"time":"2025-11-22T09:08:44+11:00","message":"CPU calculated from PreCPUStats"}
Nov 22 09:08:44 pihole pulse-docker-agent[1046343]: {"level":"debug","component":"pulse-docker-agent","container":"nebula-sync","writableLayerBytes":0,"rootFilesystemBytes":6660669,"mountCount":0,"time":"2025-11-22T09:08:44+11:00","message":"Collected container disk metrics"}
Nov 22 09:08:44 pihole pulse-docker-agent[1046343]: {"level":"debug","component":"pulse-docker-agent","containers":3,"targets":1,"time":"2025-11-22T09:08:44+11:00","message":"Report sent to Pulse targets"}

Then, to top it off, once I have finally managed to restore the monitoring for one of my docker hosts, I can no longer see any API tokens in the Pulse UI. and get an 'error loading API tokens' error popping up.

Image

Is there a more stable Docker tag to use than the recommended 'latest'? Over the last couple of months I have seen quite a bit of breakage/change which is not really ideal from a system monitoring tool.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions