Cannot ping since Docker 23

[kroese]

Description

I just upgraded from Docker 20.x to 23.x, and now all my containers have lost their ability to ping to external IPv4 addresses.

They can ping to the host machine, any machine in the LAN, any external IPv6 address (for example ipv6.google.com), that all works fine. It's just outside IPv4 addresses that receive no response, even though internet is working fine for all other uses.

The containers are in bridge networking and the userland proxy is disabled.

Is there any explanation why this is happening?

Reproduce

docker run --rm -it alpine ping -c 1 8.8.8.8

Expected behavior

Receiving a reply

docker version

Client: Docker Engine - Community
 Version:           23.0.1
 API version:       1.42
 Go version:        go1.19.5
 Git commit:        a5ee5b1
 Built:             Thu Feb  9 19:48:02 2023
 OS/Arch:           linux/amd64
 Context:           default

Server: Docker Engine - Community
 Engine:
  Version:          23.0.1
  API version:      1.42 (minimum version 1.12)
  Go version:       go1.19.5
  Git commit:       bc3805a
  Built:            Thu Feb  9 19:48:02 2023
  OS/Arch:          linux/amd64
  Experimental:     true
 containerd:
  Version:          1.6.18
  GitCommit:        2456e983eb9e37e47538f59ea18f2043c9a73640
 runc:
  Version:          1.1.4
  GitCommit:        v1.1.4-0-g5fd4c4d
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

docker info

Client:
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.10.2
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.16.0
    Path:     /usr/libexec/docker/cli-plugins/docker-compose
  scan: Docker Scan (Docker Inc.)
    Version:  v0.23.0
    Path:     /usr/libexec/docker/cli-plugins/docker-scan

Server:
 Containers: 7
  Running: 7
  Paused: 0
  Stopped: 0
 Images: 9
 Server Version: 23.0.1
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 2456e983eb9e37e47538f59ea18f2043c9a73640
 runc version: v1.1.4-0-g5fd4c4d
 init version: de40ad0
 Security Options:
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 5.19.0-31-generic
 Operating System: Ubuntu 22.10
 OSType: linux
 Architecture: x86_64
 CPUs: 4
 Total Memory: 3.656GiB
 Docker Root Dir: /mnt/storage/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Experimental: true
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

Additional Info

Internet is working fine from inside the container, its just the ICMP ping protocol that stopped functioning.

In the documentation about routing ping packets ( https://docs.docker.com/engine/security/rootless/#routing-ping-packets ) it says that you need to modify '/etc/sysctl.conf' to allow ping, but it made no difference for me. I am not even running in rootless mode, but it was worth a try since it describes my issue perfectly.

[neersighted]

a826ca3 seems plausibly related as a change that targets (essentially) sysctls that govern unprivileged ping, and that was only included in the 23.0 branch.

[hafx]

Hello,

Did you find a solution about this issue ?
I also have docker 23 in rootless mode.

All is working (external IPs are reachable inside the container).
Only the ping (ICMP) is not working.

Any idea ?

Thanks

[kroese]

@hafx In rootless mode you need to edit /etc/sysctl.conf like described here https://docs.docker.com/engine/security/rootless/#routing-ping-packets

But I am not running in rootless mode, so your issue is different.