Not collect external hdd

[Niccolum]

Host operating system: output of uname -a

Linux raspberrypi 6.1.21-v8+ #1642 SMP PREEMPT Mon Apr 3 17:24:16 BST 2023 aarch64 GNU/Linux

node_exporter version: output of node_exporter --version

node_exporter, version 1.5.0 (branch: HEAD, revision: 1b48970)
build user: root@6e7732a7b81b
build date: 20221129-18:59:41
go version: go1.19.3
platform: linux/arm64

node_exporter command line flags

node-exporter:
    container_name: monitoring-node-exporter
    image: prom/node-exporter:latest
    hostname: rpi-exporter
    restart: unless-stopped
    networks:
      - internal
    ports:
      - "9100:9100"
    command:
      - --path.procfs=/host/proc
      - --path.sysfs=/host/sys
      - --path.rootfs=/host
      - --collector.filesystem.mount-points-exclude
      - ^/(sys|proc|dev|host|etc|rootfs/var/lib/docker/containers|rootfs/var/lib/docker/overlay2|rootfs/run/docker/netns|rootfs/var/lib/docker/aufs)($$|/)
      - --path.udev.data=/host/run/udev/data
    volumes:
      - /proc:/host/proc:ro
      - /sys:/host/sys:ro
      - /:/rootfs:ro
      - /:/host:ro,rslave

node_exporter log output

monitoring-node-exporter  | ts=2023-05-05T16:27:32.167Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)"
monitoring-node-exporter  | ts=2023-05-05T16:27:32.168Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:41)"
monitoring-node-exporter  | ts=2023-05-05T16:27:32.170Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$
monitoring-node-exporter  | ts=2023-05-05T16:27:32.171Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(sys|proc|dev|host|etc|rootfs/var/lib/docker/containers|rootfs/var/lib/docker/overlay2|rootfs/run/docker/netns|rootfs/var/lib/docker/aufs)($|/)
monitoring-node-exporter  | ts=2023-05-05T16:27:32.174Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$
monitoring-node-exporter  | ts=2023-05-05T16:27:32.176Z caller=node_exporter.go:110 level=info msg="Enabled collectors"
monitoring-node-exporter  | ts=2023-05-05T16:27:32.176Z caller=node_exporter.go:117 level=info collector=arp
monitoring-node-exporter  | ts=2023-05-05T16:27:32.176Z caller=node_exporter.go:117 level=info collector=bcache
monitoring-node-exporter  | ts=2023-05-05T16:27:32.176Z caller=node_exporter.go:117 level=info collector=bonding
monitoring-node-exporter  | ts=2023-05-05T16:27:32.176Z caller=node_exporter.go:117 level=info collector=btrfs
monitoring-node-exporter  | ts=2023-05-05T16:27:32.176Z caller=node_exporter.go:117 level=info collector=conntrack
monitoring-node-exporter  | ts=2023-05-05T16:27:32.176Z caller=node_exporter.go:117 level=info collector=cpu
monitoring-node-exporter  | ts=2023-05-05T16:27:32.176Z caller=node_exporter.go:117 level=info collector=cpufreq
monitoring-node-exporter  | ts=2023-05-05T16:27:32.176Z caller=node_exporter.go:117 level=info collector=diskstats
monitoring-node-exporter  | ts=2023-05-05T16:27:32.176Z caller=node_exporter.go:117 level=info collector=dmi
monitoring-node-exporter  | ts=2023-05-05T16:27:32.176Z caller=node_exporter.go:117 level=info collector=edac
monitoring-node-exporter  | ts=2023-05-05T16:27:32.176Z caller=node_exporter.go:117 level=info collector=entropy
monitoring-node-exporter  | ts=2023-05-05T16:27:32.176Z caller=node_exporter.go:117 level=info collector=fibrechannel
monitoring-node-exporter  | ts=2023-05-05T16:27:32.176Z caller=node_exporter.go:117 level=info collector=filefd
monitoring-node-exporter  | ts=2023-05-05T16:27:32.176Z caller=node_exporter.go:117 level=info collector=filesystem
monitoring-node-exporter  | ts=2023-05-05T16:27:32.176Z caller=node_exporter.go:117 level=info collector=hwmon
monitoring-node-exporter  | ts=2023-05-05T16:27:32.176Z caller=node_exporter.go:117 level=info collector=infiniband
monitoring-node-exporter  | ts=2023-05-05T16:27:32.176Z caller=node_exporter.go:117 level=info collector=ipvs
monitoring-node-exporter  | ts=2023-05-05T16:27:32.176Z caller=node_exporter.go:117 level=info collector=loadavg
monitoring-node-exporter  | ts=2023-05-05T16:27:32.176Z caller=node_exporter.go:117 level=info collector=mdadm
monitoring-node-exporter  | ts=2023-05-05T16:27:32.176Z caller=node_exporter.go:117 level=info collector=meminfo
monitoring-node-exporter  | ts=2023-05-05T16:27:32.176Z caller=node_exporter.go:117 level=info collector=netclass
monitoring-node-exporter  | ts=2023-05-05T16:27:32.176Z caller=node_exporter.go:117 level=info collector=netdev
monitoring-node-exporter  | ts=2023-05-05T16:27:32.176Z caller=node_exporter.go:117 level=info collector=netstat
monitoring-node-exporter  | ts=2023-05-05T16:27:32.176Z caller=node_exporter.go:117 level=info collector=nfs
monitoring-node-exporter  | ts=2023-05-05T16:27:32.176Z caller=node_exporter.go:117 level=info collector=nfsd
monitoring-node-exporter  | ts=2023-05-05T16:27:32.176Z caller=node_exporter.go:117 level=info collector=nvme
monitoring-node-exporter  | ts=2023-05-05T16:27:32.176Z caller=node_exporter.go:117 level=info collector=os
monitoring-node-exporter  | ts=2023-05-05T16:27:32.176Z caller=node_exporter.go:117 level=info collector=powersupplyclass
monitoring-node-exporter  | ts=2023-05-05T16:27:32.176Z caller=node_exporter.go:117 level=info collector=pressure
monitoring-node-exporter  | ts=2023-05-05T16:27:32.176Z caller=node_exporter.go:117 level=info collector=rapl
monitoring-node-exporter  | ts=2023-05-05T16:27:32.176Z caller=node_exporter.go:117 level=info collector=schedstat
monitoring-node-exporter  | ts=2023-05-05T16:27:32.176Z caller=node_exporter.go:117 level=info collector=selinux
monitoring-node-exporter  | ts=2023-05-05T16:27:32.176Z caller=node_exporter.go:117 level=info collector=sockstat
monitoring-node-exporter  | ts=2023-05-05T16:27:32.176Z caller=node_exporter.go:117 level=info collector=softnet
monitoring-node-exporter  | ts=2023-05-05T16:27:32.177Z caller=node_exporter.go:117 level=info collector=stat
monitoring-node-exporter  | ts=2023-05-05T16:27:32.177Z caller=node_exporter.go:117 level=info collector=tapestats
monitoring-node-exporter  | ts=2023-05-05T16:27:32.177Z caller=node_exporter.go:117 level=info collector=textfile
monitoring-node-exporter  | ts=2023-05-05T16:27:32.177Z caller=node_exporter.go:117 level=info collector=thermal_zone
monitoring-node-exporter  | ts=2023-05-05T16:27:32.177Z caller=node_exporter.go:117 level=info collector=time
monitoring-node-exporter  | ts=2023-05-05T16:27:32.177Z caller=node_exporter.go:117 level=info collector=timex
monitoring-node-exporter  | ts=2023-05-05T16:27:32.177Z caller=node_exporter.go:117 level=info collector=udp_queues
monitoring-node-exporter  | ts=2023-05-05T16:27:32.177Z caller=node_exporter.go:117 level=info collector=uname
monitoring-node-exporter  | ts=2023-05-05T16:27:32.177Z caller=node_exporter.go:117 level=info collector=vmstat
monitoring-node-exporter  | ts=2023-05-05T16:27:32.177Z caller=node_exporter.go:117 level=info collector=xfs
monitoring-node-exporter  | ts=2023-05-05T16:27:32.177Z caller=node_exporter.go:117 level=info collector=zfs
monitoring-node-exporter  | ts=2023-05-05T16:27:32.187Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100
monitoring-node-exporter  | ts=2023-05-05T16:27:32.187Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100

Are you running node_exporter in Docker?

yes

What did you do that produced an error?

$ curl localhost:9100/metrics | grep filesystem_device_error | grep fuse

node_filesystem_device_error{device="/dev/sdb2",fstype="fuseblk",mountpoint="/media/nvidov/external_hdd"} 1
node_filesystem_device_error{device="gvfsd-fuse",fstype="fuse.gvfsd-fuse",mountpoint="/run/user/1000/gvfs"} 1

What did you expect to see?

$ curl localhost:9100/metrics | grep filesystem_device_error | grep fuse
node_filesystem_device_error{device="gvfsd-fuse",fstype="fuse.gvfsd-fuse",mountpoint="/run/user/1000/gvfs"} 1

I bought an external hdd drive "WD My Passport", connected via usb3.0 port to my raspberry pi 4. It automatically mounts in /media/nvidov with the type fuseblk (on macos it is defined as Macos Extended, like in the picture below). When you reformat it to another fstype, rpi starts to work very slowly and the disk speed drops significantly. The logs are empty. I don't know where to look.

Thanks for help

[dswarbrick]

I assume that you've noticed this in the node_exporter README:

It's not recommended to deploy it as a Docker container because it requires access to the host system.

Having said that, you can get more info about why node_exporter reports an error for that filesystem / mountpoint by setting your log level to debug. I expect that you will see an error message of "Error on statfs() system call", and that this is most likely due to the fact that you are not binding the /media/nvidov/external_hdd mountpoint in your Docker container.

[SuperQ]

It's not recommended to deploy it as a Docker container because it requires access to the host system.

We should really remove that Docker warning now that Docker support is better handled. We officially support Docker, but "careful use" so that the host system is taken into consideration is done.

[SuperQ]

That appears to be a Docker Compose configuration. Please see the Docker Compose example in the README for correct usage.

For questions/help/support please use our community channels. There are more people available to potentially respond to your request and the whole community can benefit from the answers provided.

[Niccolum]

My solution was:
user: "1000"
in docker compose.

@dswarbrick was right, in debug logs I saw

monitoring-node-exporter | ts=2023-05-08T08:44:53.860Z caller=filesystem_linux.go:96 level=debug collector=filesystem msg="Error on statfs() system call" rootfs=/host/media/nvidov/external_hdd err="permission denied"

because of $USER was nobody

Thanks all for help