[Bug]: Error trying to run in Docker for first time.

[YouveGotMeowxy]

Bug description

I'm trying to get ND up and running in the docker container, and getting errors out of the box.

2023-01-05 14:30:54 Netdata detected docker.sock
2023-01-05 14:30:54 Creating docker group 1001
2023-01-05 14:30:54 Assign netdata user to docker group 1001
2023-01-05 14:30:54 addgroup: permission denied (are you root?)
2023-01-05 14:30:54 Could not add group docker with ID 1001, its already there probably
2023-01-05 14:30:54 usermod: group '1001' does not exist
2023-01-05 14:30:54 Could not add netdata user to group docker with ID 1001
2023-01-05 14:30:54 2023-01-05 20:30:54: netdata INFO  : MAIN : CONFIG: cannot load cloud config '/var/lib/netdata/cloud.d/cloud.conf'. Running with internal defaults.
2023-01-05 14:30:54 2023-01-05 20:30:54: netdata INFO  : MAIN : Using host prefix directory '/host'
2023-01-05 14:30:54 2023-01-05 20:30:54: netdata ERROR : MAIN : Cannot open file '/var/log/netdata/health.log'. Leaving -1 to its default. (errno 13, Permission denied)
2023-01-05 14:30:54 2023-01-05 20:30:54: netdata INFO  : MAIN : TIMEZONE: using TZ variable 'America/Chicago'
2023-01-05 14:30:54 2023-01-05 20:30:54: netdata INFO  : MAIN : TIMEZONE: fixed as 'America/Chicago'
2023-01-05 14:30:54 2023-01-05 20:30:54: netdata INFO  : MAIN : SIGNAL: Enabling reaper
2023-01-05 14:30:54 2023-01-05 20:30:54: netdata INFO  : MAIN : process tracking enabled.
2023-01-05 14:30:54 2023-01-05 20:30:54: netdata INFO  : MAIN : resources control: allowed file descriptors: soft = 1048576, max = 1048576
2023-01-05 14:30:54 2023-01-05 20:30:54: netdata INFO  : MAIN : Out-Of-Memory (OOM) score is already set to the wanted value 0
2023-01-05 14:30:54 2023-01-05 20:30:54: netdata ERROR : MAIN : Cannot adjust netdata scheduling policy to batch (3), with priority 0. Falling back to nice. (errno 38, Function not implemented)
2023-01-05 14:30:54 2023-01-05 20:30:54: netdata ERROR : MAIN : Cannot get my current process scheduling policy. (errno 38, Function not implemented)
2023-01-05 14:30:54 2023-01-05 20:30:54: netdata ERROR : MAIN : Cannot switch to user's netdata group (gid: 201). (errno 1, Operation not permitted)
2023-01-05 14:30:54 2023-01-05 20:30:54: netdata ERROR : MAIN : Cannot become user 'netdata'. Continuing as we are.
2023-01-05 14:30:54 2023-01-05 20:30:54: netdata INFO  : MAIN : netdata started on pid 1.
2023-01-05 14:30:54 2023-01-05 20:30:54: netdata INFO  : MAIN : Initializing spawn client.
2023-01-05 14:30:54 2023-01-05 20:30:54: netdata INFO  : MAIN : Executing /usr/libexec/netdata/plugins.d/system-info.sh
2023-01-05 14:30:54 Netdata detected docker.sock
2023-01-05 14:30:54 Creating docker group 1001
2023-01-05 14:30:54 Assign netdata user to docker group 1001
2023-01-05 14:30:54 Spawn server is up.
2023-01-05 14:30:54 2023-01-05 20:30:54: netdata INFO  : MAIN : NETDATA_CONTAINER_OS_NAME=Alpine Linux
2023-01-05 14:30:54 2023-01-05 20:30:54: netdata INFO  : MAIN : NETDATA_CONTAINER_OS_ID=alpine
2023-01-05 14:30:54 2023-01-05 20:30:54: netdata INFO  : MAIN : NETDATA_CONTAINER_OS_ID_LIKE=unknown
2023-01-05 14:30:54 2023-01-05 20:30:54: netdata INFO  : MAIN : NETDATA_CONTAINER_OS_VERSION=unknown
2023-01-05 14:30:54 2023-01-05 20:30:54: netdata INFO  : MAIN : NETDATA_CONTAINER_OS_VERSION_ID=3.16.3
2023-01-05 14:30:54 2023-01-05 20:30:54: netdata INFO  : MAIN : NETDATA_CONTAINER_OS_DETECTION=/etc/os-release
2023-01-05 14:30:54 2023-01-05 20:30:54: netdata INFO  : MAIN : NETDATA_CONTAINER_IS_OFFICIAL_IMAGE=true
2023-01-05 14:30:54 2023-01-05 20:30:54: netdata INFO  : MAIN : NETDATA_HOST_OS_NAME=Ubuntu
2023-01-05 14:30:54 2023-01-05 20:30:54: netdata INFO  : MAIN : NETDATA_HOST_OS_ID=ubuntu
2023-01-05 14:30:54 2023-01-05 20:30:54: netdata INFO  : MAIN : NETDATA_HOST_OS_ID_LIKE=debian
2023-01-05 14:30:54 2023-01-05 20:30:54: netdata INFO  : MAIN : NETDATA_HOST_OS_VERSION=20.04.5 LTS (Focal Fossa)
2023-01-05 14:30:54 2023-01-05 20:30:54: netdata INFO  : MAIN : NETDATA_HOST_OS_VERSION_ID=20.04
2023-01-05 14:30:54 2023-01-05 20:30:54: netdata INFO  : MAIN : NETDATA_HOST_OS_DETECTION=/host/etc/os-release
2023-01-05 14:30:54 2023-01-05 20:30:54: netdata INFO  : MAIN : NETDATA_HOST_IS_K8S_NODE=false
2023-01-05 14:30:54 2023-01-05 20:30:54: netdata INFO  : MAIN : NETDATA_SYSTEM_KERNEL_NAME=Linux
2023-01-05 14:30:54 2023-01-05 20:30:54: netdata INFO  : MAIN : NETDATA_SYSTEM_KERNEL_VERSION=5.15.79.1-microsoft-standard-WSL2
2023-01-05 14:30:54 2023-01-05 20:30:54: netdata INFO  : MAIN : NETDATA_SYSTEM_ARCHITECTURE=x86_64
2023-01-05 14:30:54 2023-01-05 20:30:54: netdata INFO  : MAIN : NETDATA_SYSTEM_VIRTUALIZATION=microsoft
2023-01-05 14:30:54 2023-01-05 20:30:54: netdata INFO  : MAIN : NETDATA_SYSTEM_VIRT_DETECTION=lscpu
2023-01-05 14:30:54 2023-01-05 20:30:54: netdata INFO  : MAIN : NETDATA_SYSTEM_CONTAINER=docker
2023-01-05 14:30:54 2023-01-05 20:30:54: netdata INFO  : MAIN : NETDATA_SYSTEM_CONTAINER_DETECTION=dockerenv
2023-01-05 14:30:54 2023-01-05 20:30:54: netdata INFO  : MAIN : NETDATA_SYSTEM_CPU_LOGICAL_CPU_COUNT=12
2023-01-05 14:30:54 2023-01-05 20:30:54: netdata INFO  : MAIN : NETDATA_SYSTEM_CPU_VENDOR=AuthenticAMD
2023-01-05 14:30:54 2023-01-05 20:30:54: netdata INFO  : MAIN : NETDATA_SYSTEM_CPU_MODEL=AMD Ryzen 5 1600 Six-Core Processor
2023-01-05 14:30:54 2023-01-05 20:30:54: netdata INFO  : MAIN : NETDATA_SYSTEM_CPU_FREQ=3194000000
2023-01-05 14:30:54 2023-01-05 20:30:54: netdata INFO  : MAIN : NETDATA_SYSTEM_CPU_DETECTION=lscpu procfs
2023-01-05 14:30:54 2023-01-05 20:30:54: netdata INFO  : MAIN : NETDATA_SYSTEM_TOTAL_RAM=16774213632
2023-01-05 14:30:54 2023-01-05 20:30:54: netdata INFO  : MAIN : NETDATA_SYSTEM_RAM_DETECTION=procfs
2023-01-05 14:30:54 2023-01-05 20:30:54: netdata INFO  : MAIN : NETDATA_SYSTEM_TOTAL_DISK_SIZE=1649648214016
2023-01-05 14:30:54 2023-01-05 20:30:54: netdata INFO  : MAIN : NETDATA_SYSTEM_DISK_DETECTION=sysfs
2023-01-05 14:30:54 2023-01-05 20:30:54: netdata INFO  : MAIN : NETDATA_INSTANCE_CLOUD_TYPE=unknown
2023-01-05 14:30:54 2023-01-05 20:30:54: netdata INFO  : MAIN : NETDATA_INSTANCE_CLOUD_INSTANCE_TYPE=unknown
2023-01-05 14:30:54 2023-01-05 20:30:54: netdata INFO  : MAIN : NETDATA_INSTANCE_CLOUD_INSTANCE_REGION=unknown
2023-01-05 14:30:54 2023-01-05 20:30:54: netdata ERROR : MAIN : Failed to initialize database at /var/cache/netdata/netdata-meta.db, due to "unable to open database file"
2023-01-05 14:30:54 2023-01-05 20:30:54: netdata FATAL : netdata : Failed to initialize SQLite # : Invalid argument
2023-01-05 14:30:54 
2023-01-05 14:30:54 2023-01-05 20:30:54: netdata INFO  : MAIN : /usr/libexec/netdata/plugins.d/anonymous-statistics.sh 'FATAL' 'netdata:netdata' '0888@database/r:rrd_init       /22'
2023-01-05 14:30:55 2023-01-05 20:30:55: netdata ERROR : MAIN : Execution of anonymous statistics script returned http code 400.
2023-01-05 14:30:55 2023-01-05 20:30:55: netdata INFO  : MAIN : EXIT: netdata prepares to exit with code 1...
2023-01-05 14:30:55 2023-01-05 20:30:55: netdata INFO  : MAIN : /usr/libexec/netdata/plugins.d/anonymous-statistics.sh 'EXIT' 'ERROR' '-'
2023-01-05 14:30:55 2023-01-05 20:30:55: netdata ERROR : MAIN : Execution of anonymous statistics script returned http code 400.
2023-01-05 14:30:55 2023-01-05 20:30:55: netdata INFO  : MAIN : EXIT: cleaning up the database...
2023-01-05 14:30:55 2023-01-05 20:30:55: netdata INFO  : MAIN : Cleaning up database [0 hosts(s)]...
2023-01-05 14:30:55 2023-01-05 20:30:55: netdata INFO  : MAIN : EXIT: all done - netdata is now exiting - bye bye...
2023-01-05 14:30:55 EOF found in spawn pipe.

I also had an errror before getting this far, which I fixed on my own:

FATAL : netdata : Cannot create directory '/var/lib/netdata/registry'. # : Invalid argument

I fixed that by manually creating the registry folder in my mount (I should note that the container did auto create a bunch of other folders and files in the mounts automatically, so not sure why it didn't do the same for that one), but this error seems (I may be wrong) like it's having trouble just finding the netdata-meta.db, and since it's a db, simply created a blank file there w/that name seems like it wouldn't work because it would have to be initialized/formatted as a .db? It seems like the container should auto-recognize missing required files and populate them as needed, for a smooth startup.

Also, at this moment, based on the 2 errors I'm already getting on a first run, even just "getting past" this next 1 error, I'm half-expecting to get hit w/another one after this, lol.

Expected behavior

To just be able to fire up the container on a first run w/out fighting with folder/file issues first. :)

Steps to reproduce

Here's my Compose:

  netdata:
      hostname: 'netdata'
      image: 'netdata/netdata:latest'
      pid: 'host'
      environment:
        TZ: 'America/Chicago'
      volumes:
        - '/opt/docker/configs/netdata/config:/etc/netdata:rw'
        - '/opt/docker/configs/netdata/lib:/var/lib/netdata:rw'
        - '/opt/docker/configs/netdata/cache:/var/cache/netdata:rw'
        - '/etc/passwd:/host/etc/passwd:ro'
        - '/etc/group:/host/etc/group:ro'
        - '/proc:/host/proc:ro'
        - '/sys:/host/sys:ro'
        - '/etc/os-release:/host/etc/os-release:ro'
        - '/var/run/docker.sock:/var/run/docker.sock:ro'
      ports:
        - '19999:19999'
      networks:
        - 'monitors'
      security_opt:
        - 'apparmor:unconfined'
      cap_add:
        - 'SYS_PTRACE'
        - 'SYS_ADMIN'
      deploy:
        mode: 'global'
        resources:
          limits:
            memory: '500M'
        restart_policy:
          condition: 'any'

Installation method

docker

System info

Linux Cloud 5.15.79.1-microsoft-standard-WSL2 #1 SMP Wed Nov 23 01:01:46 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
/etc/lsb-release:DISTRIB_ID=Ubuntu
/etc/lsb-release:DISTRIB_RELEASE=20.04
/etc/lsb-release:DISTRIB_CODENAME=focal
/etc/lsb-release:DISTRIB_DESCRIPTION="Ubuntu 20.04.5 LTS"
/etc/os-release:NAME="Ubuntu"
/etc/os-release:VERSION="20.04.5 LTS (Focal Fossa)"
/etc/os-release:ID=ubuntu
/etc/os-release:ID_LIKE=debian
/etc/os-release:PRETTY_NAME="Ubuntu 20.04.5 LTS"
/etc/os-release:VERSION_ID="20.04"
/etc/os-release:VERSION_CODENAME=focal
/etc/os-release:UBUNTU_CODENAME=focal


NOTE:
I'm running Docker Desktop, WSL2.

Netdata build info

Can't get this info because ND won't stay running.

Additional info

No response

[ilyam8]

Hi, @YouveGotMeowxy.

To just be able to fire up the container on a first run w/out fighting with folder/file issues first.

Then you need either to use volumes (as suggested in our docs) or make sure Netdata has read/write permissions on the directories on the host machine mounted in the container (for instance, create netdata user with uid 201 and chown -R netdata:netdata /opt/docker/configs/*).

[YouveGotMeowxy]

Hi @ilyam8 ! :)

Is there any way to just add all that stuff right into the dockerfile and thus the image, so the end user doesn't have to do it themselves?

I (and many others) prefer to use mounts over volumes, for several reasons, if not just personal preference. For example, I have something like 100 containers, and this would be the only 1 I'd have to use a volume for rather than a mount, lol.

For example, in one of my dockerfiles I do a (it doesn't create a user, but I'd imagine it's also trivial to add that as well):

RUN chown -R apache:apache /htdocs
RUN find /htdocs -type d -print0 | xargs -0 chmod 0755
RUN find /htdocs -type f -print0 | xargs -0 chmod 0644

and the /htdocs folder in the container is just a volume mount:

    volumes:
      - '/opt/docker/configs/mycontainer/htdocs:/htdocs:rw'

[STaRDoGG]

@YouveGotMeowxy You can chown -R the netdata mount as your normal user, and then add that UID/GID to the Compose with user: '1000:1000'

Assuming your user's UID/GID is 1000.

[netdata-community-bot]

This issue has been mentioned on the Netdata Community Forums. There might be relevant details there:

https://community.netdata.cloud/t/issue-with-netdata-docker-agent-failed-to-create-sqlite/4118/2