Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rasperry Docker PiHole / "Starting lighttpd" Hanging #593

Closed
7 tasks
yozgatliahmed opened this issue Mar 28, 2020 · 26 comments
Closed
7 tasks

Rasperry Docker PiHole / "Starting lighttpd" Hanging #593

yozgatliahmed opened this issue Mar 28, 2020 · 26 comments

Comments

@yozgatliahmed
Copy link

yozgatliahmed commented Mar 28, 2020

This is a...

  • Request for a new or modified feature
  • Issue trying to run the docker image
  • Issue trying to build / test / develop the docker image

Description

I want Install PiHole and RainLoop as Docker

Expected Behavior

I Install fresh Raspian, activatet SSH and Install Docker.
And install Pihole "docker pull pihole/pihole"

Actual Behavior

Its hanging "Starting lighttpd"

Steps to Reproduce and debugging done

e.g. your docker run command, pages to visit, CLI commands you ran

  1. Install Docker
  2. Install PiHole: docker pull pihole/pihole
  3. Its hanging "Starting lighttpd"

Debug steps I have tried

docker rm pihole (=Error: No such container: pihole)
docker rmi pihole/pihole (=Error response from daemon: conflict: unable to remove repository reference "pihole/pihole" (must force) - container f2ec19c0a213 is using its referenced image 96082728873a)

  • I have tried destroying my container instance, pulling the newest image version, and re-creating a new container
  • I have tried running the nearly stock docker run example in the readme (removing any customizations I added)
  • I have tried running without my volume data mounts to eliminate volumes as the cause
  • I have searched this repository for existing issues and pull requests that look similar

Your Environment

  • Docker Host Operating System and OS Version: Raspian on Raspberry Pi 3
  • Docker Version: 19.03.8
  • Hardware architecture: Raspberry Pi
@dschaper
Copy link
Member

What is the exact docker run command you used?

@yozgatliahmed
Copy link
Author

  1. docker pull diginc/pi-hole
  2. docker pull pihole/pihole
  3. sudo docker run pihole/pihole

@dschaper
Copy link
Member

That's not sufficient to start the container. Read through https://github.com/pi-hole/docker-pi-hole#quick-start for more information.

@takov751
Copy link

takov751 commented Apr 2, 2020

I have started the docker with the given docker-compose.yaml with just the necessary
i am attaching the output from sudo docker-compose up
https://paste.ubuntu.com/p/VqvK7vsZCW/

tested system ubuntu 20.04 server Linux ubuntu 5.4.0-1006-raspi2 #6-Ubuntu SMP Mon Mar 16 13:08:46 UTC 2020 aarch64 aarch64 aarch64 GNU/Linux

@eratio08
Copy link

eratio08 commented Apr 11, 2020

Getting the same error. Pi-hole seems to work fine but the dashboard is not reachable Compose setup is:

  pihole:
    image: pihole/pihole
    container_name: pihole
    restart: unless-stopped
    domainname: ${PI_HOLE_VIRTUAL_HOST}
    dns:
      - 127.0.0.1
      - 192.168.178.1
    ports:
      - "${IP}:53:53/tcp"
      - "${IP}:53:53/udp"
      - "67:67/udp"
      - "8080:80
    volumes:
      - ${VOLUMEN_BASE_DIR}/pihole/data:/etc/pihole/
      - ${VOLUMEN_BASE_DIR}/pihole/dnsmasq.d/:/etc/dnsmasq.d/
    environment:
      VIRTUAL_HOST: ${PI_HOLE_VIRTUAL_HOST}
      WEBPASSWORD: ${PI_HOLE_WEBPASSWORD}
      DNS1: 9.9.9.9
      DNS2: 149.112.112.112

Error message:

(server.c.970) couldn't get 'max filedescriptors' Operation not permitted 
Stopping lighttpd
ghttpd: no process found
  • uname -a: Linux rpi4 4.19.113-3-MANJARO-ARM #1 SMP PREEMPT Sun Mar 29 21:45:27 CDT 2020 aarch64 GNU/Linux
  • docker -v: Docker version 19.03.8-ce, build afacb8b7f0
  • docker-compose -v: docker-compose version 1.25.4, build unknown
  • Image: pihole/pihole latest 505ad79f9e8a

EDIT:
Setting privileged: truesolves the issue. Source: https://forum.manjaro.org/t/pi-hole-won-t-start-after-update/128907

@punoko
Copy link

punoko commented May 10, 2020

Same issue with a Raspberry Pi 4 on Ubuntu 20.04 64-bit using the default docker-compose example.

pihole          | https://s3.amazonaws.com/lists.disconnect.me/simple_ad.txt
pihole          | sudo: getrlimit: Operation not permitted
pihole          | sudo: setrlimit: Operation not permitted
pihole          | sudo: setrlimit: Operation not permitted
pihole          | sudo: getrlimit: Operation not permitted
pihole          | sudo: setrlimit: Operation not permitted
pihole          | sudo: setrlimit: Operation not permitted
  [✓] Pi-hole Enabled
pihole          |   Pi-hole version is v5.0 (Latest: v5.0)
pihole          |   AdminLTE version is v5.0 (Latest: v5.0)
pihole          |   FTL version is v5.0 (Latest: v5.0)
pihole          | [cont-init.d] 20-start.sh: exited 0.
pihole          | [cont-init.d] done.
pihole          | [services.d] starting services
pihole          | Starting lighttpd
pihole          | Starting crond
pihole          | Starting pihole-FTL (no-daemon) as root
pihole          | 
pihole          | Critical error - immediate abort
pihole          | 
pihole          | Critical error - immediate abort
pihole          | [services.d] done.
pihole          | 2020-05-10 22:05:36: (server.c.970) couldn't get 'max filedescriptors' Operation not permitted 
pihole          | Stopping lighttpd
pihole          | lighttpd: no process found
pihole          | Starting lighttpd
pihole          | 2020-05-10 22:05:37: (server.c.970) couldn't get 'max filedescriptors' Operation not permitted 
pihole          | Stopping lighttpd
pihole          | lighttpd: no process found
  • uname -a : Linux ubuntu 5.4.0-1008-raspi #8-Ubuntu SMP Wed Apr 8 11:13:06 UTC 2020 aarch64 aarch64 aarch64 GNU/Linux
  • docker -v : Docker version 19.03.8, build afacb8b7f0
  • docker-compose -v : docker-compose version 1.25.0, build unknown
  • Image: pihole/pihole latest 4a3ca1b729d7

Running a privileged container as suggested above works but is not an elegant solution.
Maybe enabling a specific capability could be enough? But I don't know which one.

@diginc
Copy link
Collaborator

diginc commented May 10, 2020

Are you using ubuntu's docker SNAP? Uninstall and try the official install if so: curl -fsSL https://get.docker.com | sudo sh

edit: also unless your docker debian is arm64, you'll quite possibly have to still use armhf version of the docker. Docker will try to download the wrong on by default unless you call out armhf tag, the symptom you have this problem is you'll have a negative block list count in the web UI once everything runs.

@punoko
Copy link

punoko commented May 10, 2020

I am using the docker.io package from the Ubuntu repo which doesn't seem to be a SNAP as far as I can tell:

$ snap list
No snaps are installed yet. Try 'snap install hello-world'.

It also seems to be arm64:

$ sudo dpkg --list | grep -e docker -e runc -e containerd
ii  containerd                     1.3.3-0ubuntu2                    arm64        daemon to control runC
ii  docker-compose                 1.25.0-1                          all          Punctual, lightweight development environments using Docker
ii  docker.io                      19.03.8-0ubuntu1                  arm64        Linux container runtime
ii  python3-docker                 4.1.0-1                           all          Python 3 wrapper to access docker.io's control socket
ii  python3-dockerpty              0.4.1-2                           all          Pseudo-tty handler for docker Python client (Python 3.x)
ii  runc                           1.0.0~rc10-0ubuntu1               arm64        Open Container Project - runtime

The reason I am using this is because the focal release (20.04) isn't available from the official Docker repo yet:

$ curl -fsSL https://get.docker.com | sudo sh
# Executing docker install script, commit: 1b02882d63b9cfc484ad6b0180171c679cfe0f3a
+ sh -c apt-get update -qq >/dev/null
+ sh -c DEBIAN_FRONTEND=noninteractive apt-get install -y -qq apt-transport-https ca-certificates curl >/dev/null
+ sh -c curl -fsSL "https://download.docker.com/linux/ubuntu/gpg" | apt-key add -qq - >/dev/null
Warning: apt-key output should not be parsed (stdout is not a terminal)
+ sh -c echo "deb [arch=arm64] https://download.docker.com/linux/ubuntu focal stable" > /etc/apt/sources.list.d/docker.list
+ sh -c apt-get update -qq >/dev/null
E: The repository 'https://download.docker.com/linux/ubuntu focal Release' does not have a Release file.

Installing the eoan release (19.10) from the official Docker repo...

$ cat /etc/apt/sources.list.d/docker.list 
deb [arch=arm64] https://download.docker.com/linux/ubuntu eoan stable
$ sudo apt-get update
$ sudo apt-get install docker-ce docker-ce-cli containerd.io

... yields the exact same errors as previously reported in unprivileged containers.
Again, everything works perfectly fine without errors while running in a privileged container with the privileged: truekey in docker-compose.

@diginc
Copy link
Collaborator

diginc commented May 11, 2020

I don't trust docker.io and there is no official support for docker on 20.04 - the work around seems to be using bionic's apt source repository on focal: https://askubuntu.com/a/1230190

@punoko
Copy link

punoko commented May 11, 2020

Just tried with bionic, same behavior/errors.

@tanasegabriel
Copy link

tanasegabriel commented May 11, 2020

Something changed in Linux land, rather than Docker. The version of both docker.io and docker-compose bundled in Ubuntu 20.04 are relatively new (off-topic, but I'm hoping this will be the way to manage docker moving forward).

There's reports of the same behaviour happening on Manjaro and Gentoo

Both of those distributions are known for good 64-bit support on RPi. This unrelated forum question hints at a bug on seccomp on arm64

L.E. seccomp/libseccomp#235 seems to be solving this. I'm hoping the updated version will be available soon.

@NoSoloHacking
Copy link

Hello,
The issue is with the privileged mode check this article to fix it: https://www.nosolohacking.info/docker-pihole-error-server-c-970-couldnt-get-max-filedescriptors-operation-not-permitted/

@ghost
Copy link

ghost commented Jun 21, 2020

@NoSoloHacking Thanks for the tip. It does enable lighttpd to start but shouldn't there be a a way to use a smaller amount of capabilities than using all of --priviledged? I tried adding just --cap-add SYS_RESOURCE but that wasn't enough.

@pralor-bot
Copy link

This issue has been mentioned on Pi-hole Userspace. There might be relevant details there:

https://discourse.pi-hole.net/t/pi-hole-runs-on-podman/34719/1

@NoSoloHacking
Copy link

@NoSoloHacking Thanks for the tip. It does enable lighttpd to start but shouldn't there be a a way to use a smaller amount of capabilities than using all of --priviledged? I tried adding just --cap-add SYS_RESOURCE but that wasn't enough.

You are right however, I haven´t found any other way to achieve it.

@legendofmiracles
Copy link

Probably very unnecessary to mention it, but i am experiencing the same issues on archlinux arm. on a not even one day old install.

@legendofmiracles
Copy link

Update: The error also went away when running with privileged: true
Sorry for adding two unnecessary comments

@tanasegabriel
Copy link

tanasegabriel commented Sep 20, 2020

I'd avoid running privileged: true unless absolutely necessary. There's no need for pihole to run under those capabilities.
Anyway, the latest version of seccomp available through the Focal repositories is 2.4.3. I don't experience the issues here while on this version. If you're on Ubuntu, just update your packages and all should work as expected.

I believe that once all of the mainstream distros updated their upstream dependencies, this issue can be closed.

@ghost
Copy link

ghost commented Sep 21, 2020

I built my own libseccomp-2.4.4 package on Arch Linux Arm for aarch64 (really a libvirt KVM/Qemu aarch64 HVM) and it worked. I was able to start Pi-Hole rootless in podman using podman-compose. Thanks @tanasegabriel

$ yay -Q libseccomp
libseccomp 2.4.4-1

Have to allow non-root to open ports 53, 67, 80 and 443:

$ sudo sh -c "echo 52 > /proc/sys/net/ipv4/ip_unprivileged_port_start"

version: "3"

# More info at https://github.com/pi-hole/docker-pi-hole/ and https://docs.pi-hole.net/
services:
  pihole:
    container_name: pihole
    image: pihole/pihole:latest
    ports:
      - "53:53/tcp"
      - "53:53/udp"
      - "67:67/udp"
      - "80:80/tcp"
      - "443:443/tcp"
    environment:
      TZ: 'US/Eastern'
      # WEBPASSWORD: 'set a secure password here or it will be random'
    # Volumes store your data between container upgrades
    volumes:
       - 'etc-pihole:/etc/pihole/'
       - 'etc-dnsmasq:/etc/dnsmasq.d/'
    dns:
      - 127.0.0.1
      - 1.1.1.1
    # Recommended but not required (DHCP needs NET_ADMIN)
    #   https://github.com/pi-hole/docker-pi-hole#note-on-capabilities
    cap_add:
      - NET_ADMIN
      - NET_BIND_SERVICE
      - SYS_RESOURCE
    restart: always

$ podman-compose -f docker-compose.yml up

@ghost
Copy link

ghost commented Sep 22, 2020

@yozgatliahmed I think this was solved.

@pralor-bot
Copy link

This issue has been mentioned on Pi-hole Userspace. There might be relevant details there:

https://discourse.pi-hole.net/t/pi-hole-runs-on-podman/34719/6

@gstrauss
Copy link

As noted by @tanasegabriel in #593 (comment) this issue was addressed in seccomp/libseccomp#235

@dschaper
Copy link
Member

Closing this as resolved. Please let us know if you feel this should be reopened.

gstrauss added a commit to gstrauss/lighttpd1.4 that referenced this issue Oct 18, 2020
modify use of getrlimit() to not be fatal if an error occurs

Some container configurations might cause getrlimit() and setrlimit() to
fail.  lighttpd will issue error trace if this occurs, but will now
proceed.

x-ref:
  "aarch64: Use architecture specific header to general syscalls"
  seccomp/libseccomp#235
  "Rasperry Docker PiHole / "Starting lighttpd" Hanging"
  pi-hole/docker-pi-hole#593
  "Pi-hole runs on podman"
  https://discourse.pi-hole.net/t/pi-hole-runs-on-podman/34719/5
lighttpd-git pushed a commit to lighttpd/lighttpd1.4 that referenced this issue Oct 20, 2020
modify use of getrlimit() to not be fatal if an error occurs

Some container configurations might cause getrlimit() and setrlimit() to
fail.  lighttpd will issue error trace if this occurs, but will now
proceed.

x-ref:
  "aarch64: Use architecture specific header to general syscalls"
  seccomp/libseccomp#235
  "Rasperry Docker PiHole / "Starting lighttpd" Hanging"
  pi-hole/docker-pi-hole#593
  "Pi-hole runs on podman"
  https://discourse.pi-hole.net/t/pi-hole-runs-on-podman/34719/5
@ghost
Copy link

ghost commented Nov 18, 2020

For Arch Linux Arm, the patch for "aarch64: Use architecture specific header to general syscalls" was cherry-picked into libseccomp 2.4.2-1.1.

@ghost
Copy link

ghost commented Dec 11, 2020

Once the container opens port 53, etc. the privileged port range can adjusted higher again as that will not revoke the open ports.

When saving and restoring /proc/sys/net/ipv4/ip_unprivileged_port_start you have to deal with potential race conditions from other multitasking software changing /proc/sys/net/ipv4/ip_unprivileged_port_start too.

@pralor-bot
Copy link

This issue has been mentioned on Pi-hole Userspace. There might be relevant details there:

https://discourse.pi-hole.net/t/pi-hole-runs-on-podman/34719/9

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests