[macOS] Disk shortage bug

[VicSegers]

This is a: Bug

Details

When updating to "Pi-hole FTL v5.12, Web v5.9 and Core v5.7" I received this message after running:

Related Issues

• I have searched this repository/Pi-hole forums for existing issues and pull requests that look similar

How to reproduce the issue

1. Environment data

• Operating System: macOS 12.1
• Hardware: MacBook Pro (2016)
• Kernel Architecture: Darwin 21.2.0
• Docker Install Info and version:
  • Software source: Homebrew
  • Supplimentary Software: /
• Hardware architecture: x86_64

2. docker-compose.yml contents, docker run shell command, or paste a screenshot of any UI based configuration of containers here

docker-compose.yml

version: "3"

# More info at https://github.com/pi-hole/docker-pi-hole/ and https://docs.pi-hole.net/
services:
  pihole:
    container_name: pihole
    image: pihole/pihole:latest
    ports:
      - "53:53/tcp"
      - "53:53/udp"
      - "67:67/udp"
      - "80:80/tcp"
      - "443:443/tcp"
    environment:
      TZ: 'Belgium/Brussels'
      WEBPASSWORD: 'foo'
    # Volumes store your data between container upgrades
    volumes:
      - './etc-pihole/:/etc/pihole/'
      - './etc-dnsmasq.d/:/etc/dnsmasq.d/'
    # Recommended but not required (DHCP needs NET_ADMIN)
    #   https://github.com/pi-hole/docker-pi-hole#note-on-capabilities
    cap_add:
      - NET_ADMIN
    restart: unless-stopped

3. any additional info to help reproduce

These common fixes didn't work for my issue

• I have tried removing/destroying my container, and re-creating a new container
• I have tried fresh volume data by backing up and moving/removing the old volume data
• I have tried running the stock docker run example(s) in the readme (removing any customizations I added)
• I have tried a newer or older version of Docker Pi-hole (depending what version the issue started in for me)
• I have tried running without my volume data mounts to eliminate volumes as the cause

If the above debugging / fixes revealed any new information note it here.
Add any other debugging steps you've taken or theories on root cause that may help.

[PromoFaux]

What does running df -h from inside the container output?

@DL6ER could there be something funky in FTLs detection here?

[DL6ER]

Also, what is the output of

stat -f /etc/pihole

inside the container?

FTL computes the size of the directory by computing the fragment size times the block size:
https://github.com/pi-hole/FTL/blob/83a950b59062b5cf5f90b58abbf6842b6624ca65/src/files.c#L168

[VicSegers]

What does running df -h from inside the container output?

@DL6ER could there be something funky in FTLs detection here?

df -h in the container:

Filesystem      Size  Used Avail Use% Mounted on
overlay          59G  3.8G   52G   7% /
tmpfs            64M     0   64M   0% /dev
tmpfs           994M     0  994M   0% /sys/fs/cgroup
shm              64M  1.2M   63M   2% /dev/shm
grpcfuse        466G  213G  254G  46% /etc/pihole
/dev/vda1        59G  3.8G   52G   7% /etc/hosts
tmpfs           994M     0  994M   0% /proc/acpi
tmpfs           994M     0  994M   0% /sys/firmware

and du -h on my host

8.0K	./etc-dnsmasq.d
4.0K	./etc-pihole/migration_backup
647M	./etc-pihole
647M	.

[VicSegers]

Also, what is the output of

stat -f /etc/pihole

inside the container?

FTL computes the size of the directory by computing the fragment size times the block size: https://github.com/pi-hole/FTL/blob/83a950b59062b5cf5f90b58abbf6842b6624ca65/src/files.c#L168

  File: "/etc/pihole/"
    ID: 0        Namelen: 255     Type: fuseblk
Block size: 1048576    Fundamental block size: 4096
Blocks: Total: 122061322  Free: 66413177   Available: 66413177
Inodes: Total: 2659341079 Free: 2656527080

[DL6ER]

grpcfuse seems to be some specialty of docker. I suspect it does not report everything properly. I'm already on my way to bed (approaching midnight over here) and will report back with idea soon.

[VicSegers]

Good night! This isn't urgent at all, everything works fine.

[PromoFaux]

It could be a Docker Desktop on Mac thing? Just Copy pasted your compose file and span it up on Docker Desktop for windows:

And no such messages:

My main install running on a Pi is also not seeing this message.

What version of docker is on the host?

[VicSegers]

Docker version 20.10.10, build b485636

[mrcemento]

Same issue here. Me too on docket on Mac OS. Seems definitely docker on Mac OS related.

[Tynox]

I got the some warning too after updated to the latest version on macOS 12.1. Docker engine: 20.10.11

[torrelasley]

Also getting this message on macOS docker image. This is the first google result for "Disk shortage" related to pihole.

[DL6ER]

It very much looks like docker for macOS bug. Sorry, I looked everywhere but I don't see an obvious solution for this. Even more because neither does anyone from the core developers team has a Mac available for testing nor does it seem easy to detect the misbehaving docker version (well, except for "this value is unrealistically large").

For now, the best I can offer you is setting

CHECK_DISK=0

in /etc/pihole/pihole-FTL.conf (see here). This will disable the test altogether.

I'll come back to you when I have something to test. Have a great time and merry Christmas everyone! 🎄

[PromoFaux]

To set that variable easily, you can set it as an environment variable e.g FTLCONF_CHECK_DISK=0 in your docker-compose / docker run scripts https://github.com/pi-hole/docker-pi-hole#advanced-variables

[VersionThis]

Just to add a data point that I am seeing this same behavior, running on a fresh install:

Macbook Retina
macOS 12.1 
Darwin Kernel Version 21.1.0
Docker version 20.10.6, build 370c289

root@a3ef6990b77b:~# date ; uname -a ; uptime ; pwd
Sat Dec 25 08:45:47 CST 2021
Linux a3ef6990b77b 5.10.25-linuxkit #1 SMP Tue Mar 23 09:27:39 UTC 2021 x86_64 GNU/Linux
 08:45:47 up 21:02,  0 users,  load average: 0.12, 0.24, 0.39
/root
root@a3ef6990b77b:~# df -h
Filesystem      Size  Used Avail Use% Mounted on
overlay          59G  3.9G   52G   7% /
tmpfs            64M     0   64M   0% /dev
tmpfs           994M     0  994M   0% /sys/fs/cgroup
shm              64M  712K   64M   2% /dev/shm
grpcfuse        466G  277G  189G  60% /etc/pihole
/dev/vda1        59G  3.9G   52G   7% /etc/hosts
tmpfs           994M     0  994M   0% /proc/acpi
tmpfs           994M     0  994M   0% /sys/firmware
root@a3ef6990b77b:~# stat -f /etc/pihole
  File: "/etc/pihole"
    ID: 0        Namelen: 255     Type: fuseblk
Block size: 1048576    Fundamental block size: 4096
Blocks: Total: 122061322  Free: 49524361   Available: 49524361
Inodes: Total: 1983962276 Free: 1980974440
root@a3ef6990b77b:~# 

Will disable the check as suggested to suppress this for now.

On a side note, I sooo wish Docker on Mac & Windows was more "production ready" than it is. No wonder the big Clouds have their own container engines now.

Thanks & Happy Holidays. 🎄

[pralor-bot]

This issue has been mentioned on Pi-hole Userspace. There might be relevant details there:

https://discourse.pi-hole.net/t/adminlte-version-is-v5-9-php-socket-0-connection-refused/51839/11

[rvanbaalen]

Not sure if it add something to the discussion but I'm seeing the exact same message. Latest pihole docker container (downloaded today), running Docker on Mac.

For now I've added

    environment:
      FTLCONF_CHECK_DISK: 0

to the docker-compose.yml file as suggested by @PromoFaux, that'll at least get rid of the message for now.

[ghost]

I too can confirm this has only happened to me on mac docker installs.

[github-actions]

This issue is stale because it has been open 30 days with no activity. Please comment or update this issue or it will be closed in 5 days.

[sellersj]

Still happening with latest pi-hole 5.9

[rdwebdesign]

What do you see running df -h inside the container?

[sellersj]

# df -h
Filesystem      Size  Used Avail Use% Mounted on
overlay          59G  2.2G   54G   4% /
tmpfs            64M     0   64M   0% /dev
shm              64M  716K   64M   2% /dev/shm
grpcfuse        466G  406G   60G  88% /etc/pihole
/dev/vda1        59G  2.2G   54G   4% /etc/hosts
tmpfs           2.0G     0  2.0G   0% /proc/acpi
tmpfs           2.0G     0  2.0G   0% /sys/firmware

I also found another command in pi-hole and here's the command and output

# df -B1 / 2> /dev/null | awk 'END{ print $3,$2,$5 }'
2362048512 62725623808 4%

I also tried to run

mapfile -t file_system < <(df -h)

but the space between the < my system didn't like and I'm not sure if this is a bug or just not possible to run like this on the command line.
https://github.com/pi-hole/pi-hole/blob/6ffa2ba1b2a68c7b0919689e137017ae344aed3c/advanced/Scripts/piholeDebug.sh#L598

The current warning I see is:

Disk shortage (/etc/pihole/pihole-FTL.db) ahead: 36893004% used
/etc/pihole: 18.4EB used, 500.0GB total

[rdwebdesign]

grpcfuse was already discussed on this page, starting here (#951 (comment)).

For now, as a workaround, you can disable the check using FTLCONF_CHECK_DISK=0 in your docker-compose file or run command.

[blumarten]

was there ever a solution for this? I have a primary pi-hole on a headless mac mini and a secondary on a pi zero. I get the message pretty much every day on the mac and a couple of times per week on the pi

[PromoFaux]

@blumarten Just a few comments up: #951 (comment)

[blumarten]

@blumarten Just a few comments up: #951 (comment)

thank you mate

[russalex83]

[ ]

vi /etc/pihole/pihole-FTL.conf

#add variable:

CHECK_DISK=0

https://docs.pi-hole.net/ftldns/configfile/#check_disk

[pralor-bot]

This issue has been mentioned on Pi-hole Userspace. There might be relevant details there:

https://discourse.pi-hole.net/t/false-disk-shortage/60134/2

[pralor-bot]

This issue has been mentioned on Pi-hole Userspace. There might be relevant details there:

https://discourse.pi-hole.net/t/new-install-disk-used-available-by-3million/62113/2

[bluehz]

Still seeing this problem with fresh install. Any resolution?

[DL6ER]

Sorry, we cannot detect the environment outside the container and macOS+docker is apparently still reporting incorrect information here. I'll implement a workaround that will silence the warning in case the results found are not useful, that is used_space >> total_space but be aware that this will only work when total_space is correct. We've seen Pi-holes being installed on large disk array machines with truly available space in the low EB range so we should not define an arbitrary/artificial limit here.