Guides for how to use NetData in Caprover

[phuctm97]

Caprover has builtin NetData integration but the default setup seems to only show node metrics and not apps or Docker containers.

I'm new to NetData and I know that I can read NetData docs to DIY but is there any specific integration or guide on how to use NetData effectively with Caprover?

[githubsaturn]

The built in Netdata should show the containers. Look at the right panel in this screenshot:

PS: note that it takes a minute or two for these to show up, so if you're not seeing these, wait for 2 minutes and refresh the page.

[githubsaturn]

One step further is to bypass everything and just run the netdata
https://github.com/titpetric/netdata?tab=readme-ov-file#docker-cli

See if this works.

[phuctm97]

Here is the output of docker container inspect captain-netdata-container | grep -i mounts --after-context=40:

        "Mounts": [
            {
                "Type": "bind",
                "Source": "/var/run/docker.sock",
                "Destination": "/var/run/docker.sock",
                "Mode": "",
                "RW": true,
                "Propagation": "rprivate"
            },
            {
                "Type": "volume",
                "Name": "43e1df2d39fbaa1ece90a77fc72db6b4bb738b129c87164d382fd21d818a4d13",
                "Source": "/var/lib/docker/volumes/43e1df2d39fbaa1ece90a77fc72db6b4bb738b129c87164d382fd21d818a4d13/_data",
                "Destination": "/etc/netdata/override",
                "Driver": "local",
                "Mode": "",
                "RW": true,
                "Propagation": ""
            },
            {
                "Type": "bind",
                "Source": "/proc",
                "Destination": "/host/proc",
                "Mode": "ro",
                "RW": false,
                "Propagation": "rprivate"
            },
            {
                "Type": "bind",
                "Source": "/sys",
                "Destination": "/host/sys",
                "Mode": "ro",
                "RW": false,
                "Propagation": "rprivate"
            }
        ],
        "Config": {
            "Hostname": "f554824b601a",
            "Domainname": "",
            "User": "",
            "AttachStdin": false,

Which looks correct, right?

[phuctm97]

I'm looking into NetData docs right now. I'll try to run NetData container myself. How should I run NetData manually so that it will work with Caprover? I guess run with the container name captain-netdata-container, do I need to publish any port?

[phuctm97]

One thing that may be source of error that I did run a disk clean up (docker container prune --force, docker image prune --all, and docker volume prune). There are only volumes from my app. I'm not if that deleted Caprover default volume for NetData or not.

[githubsaturn]

Your configs look correct. Can you try this:

docker run -d --cap-add SYS_PTRACE \
           -v /proc:/host/proc \
           -v /sys:/host/sys \
           -v /var/run/docker.sock:/var/run/docker.sock \
           -p 19999:19999 \
           --restart unless-stopped \
           titpetric/netdata

And just try to see if you can open http://IP:19999 ?

If that works, try this:

docker run -d --cap-add SYS_PTRACE \
           -v /proc:/host/proc \
           -v /sys:/host/sys \
           -v /var/run/docker.sock:/var/run/docker.sock \
           -p 18888:19999 \ ### <<< changed port to avoid mistakenly using the other one
           --restart unless-stopped \
           caprover/netdata

Now open And just try to see if you can open http://IP:18888?

[phuctm97]

I just got back to this and tried the command:

docker run -d --cap-add SYS_PTRACE
-v /proc:/host/proc
-v /sys:/host/sys
-v /var/run/docker.sock:/var/run/docker.sock
-p 19999:19999
--restart unless-stopped
titpetric/netdata

The container keeps failing. The Docker logs show: exec format error.

I think it's the instance architecture's problem. When I run the image there is also a warning saying that the image is built for linux/arm64 but my instance is linux/arm64/v8.

Maybe Caprover NetData config doesn't work with linux/arm64/v8 by default? What can I do in this case?

[githubsaturn]

Yes, this is the same image and config that has not been touched in years and it has always worked.

If you tried the netdata image and even that didn't work, then it's not CapRover. Likely there's an issue with your docker installation. What exactly, i am very unsure. It could be that your docker is installed under a different user, not the root, or your installation uses a different socket. Or many other possibilities.

To confirm, you can try a different vps provider for testing. Test out Digital Ocean and see if you still have the same issue.

[phuctm97]

I followed Netdata docs to debug Docker Engine problem and found this error message in Caprover Netdata container

2025-04-17 04:12:46: go.d INFO: main[main] looking for 'docker_engine.conf' in [/etc/netdata/go.d /usr/lib/netdata/conf.d/go.d]
2025-04-17 04:12:46: go.d INFO: main[main] found '/usr/lib/netdata/conf.d/go.d/docker_engine.conf
2025-04-17 04:13:07: go.d ERROR: docker_engine[local] Get "http://127.0.0.1:9323/metrics": dial tcp 127.0.0.1:9323: connect: connection refused
2025-04-17 04:13:07: go.d ERROR: docker_engine[local] check failed

(I'm using the builtin Caprover Netdata setup)

[phuctm97]

It appears that Netdata requires host network mode for Docker metrics to work. But the Caprover Netdata config is using caprover_overlay_network, is that intended behavior?

[githubsaturn]

host mode has nothing to do with Netdata being able to pull docker container. Netdata uses the docker socket to connect to Docker API.

Did you try using a different provider?

To confirm, you can try a different vps provider for testing. Test out Digital Ocean and see if you still have the same issue.

[githubsaturn]

Did you ever get to solve the issue? Looks like someone else is having a similar problem
caprover/netdata-docker#8