Container internet speed drops over time

[lrnd1]

Description

Docker containers gradually drop in internet performance.

Restarting container does not fix the problem, only restarting docker engine.

Internet speed was always ~500 mbps or higher, as I did several speed tests on my router as well.

Reproduce

Restart docker engine -> everything goes back to normal
Wait -> performance gradually drops

Expected behavior

No response

docker version

Client:
 Version:           27.3.1
 API version:       1.47
 Go version:        go1.22.7
 Git commit:        ce12230
 Built:             Fri Sep 20 11:38:18 2024
 OS/Arch:           darwin/arm64
 Context:           desktop-linux

Server: Docker Desktop 4.36.0 (175267)
 Engine:
  Version:          27.3.1
  API version:      1.47 (minimum version 1.24)
  Go version:       go1.22.7
  Git commit:       41ca978
  Built:            Fri Sep 20 11:41:19 2024
  OS/Arch:          linux/arm64
  Experimental:     false
 containerd:
  Version:          1.7.21
  GitCommit:        472731909fa34bd7bc9c087e4c27943f9835f111
 runc:
  Version:          1.1.13
  GitCommit:        v1.1.13-0-g58aa920
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

docker info

Client:
 Version:    27.3.1
 Context:    desktop-linux
 Debug Mode: false
 Plugins:
  ai: Ask Gordon - Docker Agent (Docker Inc.)
    Version:  v0.1.0
    Path:     /Users/lori/.docker/cli-plugins/docker-ai
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.18.0-desktop.2
    Path:     /Users/lori/.docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.30.3-desktop.1
    Path:     /Users/lori/.docker/cli-plugins/docker-compose
  debug: Get a shell into any image or container (Docker Inc.)
    Version:  0.0.37
    Path:     /Users/lori/.docker/cli-plugins/docker-debug
  desktop: Docker Desktop commands (Alpha) (Docker Inc.)
    Version:  v0.0.15
    Path:     /Users/lori/.docker/cli-plugins/docker-desktop
  dev: Docker Dev Environments (Docker Inc.)
    Version:  v0.1.2
    Path:     /Users/lori/.docker/cli-plugins/docker-dev
  extension: Manages Docker extensions (Docker Inc.)
    Version:  v0.2.27
    Path:     /Users/lori/.docker/cli-plugins/docker-extension
  feedback: Provide feedback, right in your terminal! (Docker Inc.)
    Version:  v1.0.5
    Path:     /Users/lori/.docker/cli-plugins/docker-feedback
  init: Creates Docker-related starter files for your project (Docker Inc.)
    Version:  v1.4.0
    Path:     /Users/lori/.docker/cli-plugins/docker-init
  sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc.)
    Version:  0.6.0
    Path:     /Users/lori/.docker/cli-plugins/docker-sbom
  scout: Docker Scout (Docker Inc.)
    Version:  v1.15.0
    Path:     /Users/lori/.docker/cli-plugins/docker-scout

Server:
 Containers: 29
  Running: 29
  Paused: 0
  Stopped: 0
 Images: 28
 Server Version: 27.3.1
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 Swarm: inactive
 Runtimes: runc io.containerd.runc.v2
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 472731909fa34bd7bc9c087e4c27943f9835f111
 runc version: v1.1.13-0-g58aa920
 init version: de40ad0
 Security Options:
  seccomp
   Profile: unconfined
  cgroupns
 Kernel Version: 6.10.14-linuxkit
 Operating System: Docker Desktop
 OSType: linux
 Architecture: aarch64
 CPUs: 11
 Total Memory: 21.47GiB
 Name: docker-desktop
 ID: f3f47fe9-a8f8-4eb0-be04-efb401a088d1
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 No Proxy: hubproxy.docker.internal
 Labels:
  com.docker.desktop.address=unix:///Users/lori/Library/Containers/com.docker.docker/Data/docker-cli.sock
 Experimental: false
 Insecure Registries:
  hubproxy.docker.internal:5555
  127.0.0.0/8
 Live Restore Enabled: false

WARNING: daemon is not using the default seccomp profile

Diagnostics ID

49601AEC-AC4E-4766-8F39-80ECDAD2F368/20241129180217

Additional Info

Runnning docker on Mac mini M4 Pro.
I don't know if this issue is also related, however accessing some of my containers web interfaces over reverse proxy becomes slow as well.

More specifically, after the container has been running for a while, trying to access it, the web interface keeps loading for a long time until it can load for the first time. After the webpage loads once, it stays very responsive.

Ex. I open jellyfin.domain.xxx -> black screen -> wait 10-20 seconds -> jellyfin logo appears -> refresh page -> jellyfin logo appears instantly -> wait 20-30 seconds menu buttons start appearing -> refresh page -> back to menu buttons -> wait 20-30 seconds -> all my movies get loaded instantly -> site stays responsive even after further refreshes, until I stop accessing the site for a longer period.

I use azukaar / Cosmos-Server as reverse proxy, restarting it, also does not fix the network speed issue for containers. I will give an update after testing if it solves the slow web interface loading issue. It's harder to test as it needs time to slow down.
The two issues I'm having might be unrelated.

[lrnd1]

After lots of troubleshooting, I figured out a solution for the long loading time and the constantly dropping internet performance.
It was a combination of two issues.

1. I had DoS protection enabled on my Asus router, which got overwhelmed by my qbittorrent container and slowed down the speed.
2. I used some containers (including my reverse proxy server) in host mode. Without having "Use kernel networking for UDP" enabled, it causes some other issues (see Port conflicts with docker host networking #7498). This caused periodical drops in reaching my containers from outside my local network.
   Also after changing the two settings, I had to recreate my reverse proxy server. Restarting docker engine alone, did not solve the issue for some reason.