All services randomly go down and show Bad Gateway - only systemctl restart docker fixes it

[xloyb]

Hi everyone,

I'm experiencing a recurring and frustrating issue where all my applications suddenly become inaccessible and show Bad Gateway, without any apparent reason — no server reboot, no deployments, nothing changed.

What happens:

• All app URLs suddenly return 502 Bad Gateway
• Easypanel dashboard is still accessible
• All services appear healthy in the dashboard (1/1 replicas, green status)
• No changes were made before it happened
• It happens randomly, sometimes after weeks of stable uptime

What I found via SSH:
Traefik loses connectivity to other containers over Docker's overlay network. Even though all containers are running, Traefik can't reach them internally:

docker exec $(docker ps -q --filter name=easypanel-traefik) wget -qO- http://easypanel:3000
# Returns: can't connect to remote host (10.x.x.x): Host is unreachable

The overlay network becomes corrupted at the kernel level, containers are listed as connected but packets can't travel between them.

The only fix that works:

sudo systemctl restart docker

This rebuilds Docker's overlay network routing tables and everything comes back within 60 seconds.

My questions:

1. Has anyone else experienced this random overlay network corruption?
2. What triggers it? No reboot or change happens on my end before it breaks
3. Is there a way to auto-detect and recover from this automatically without manual SSH intervention?
4. Could a watchdog script or cron job prevent this?

Environment:

• VPS, 16GB RAM, 60GB disk
• Rocky Linux (RHEL-based)
• Easypanel latest
• Docker Swarm mode
• Multiple projects (~15 services across 3 projects)

This is happening repeatedly and is causing unexpected downtime. Any help or insights from the team or community would be greatly appreciated!

[deiucanta]

Hi @xloyb,

During the next outage, please run these before restarting Docker and share the output:

traefik_id=$(docker ps -q --filter label=com.docker.swarm.service.name=easypanel-traefik | head -n 1)

docker exec "$traefik_id" wget -S -T 5 -O /dev/null http://easypanel:3000
docker exec "$traefik_id" wget -S -T 5 -O /dev/null http://tasks.easypanel:3000

sudo systemctl is-active firewalld
docker service logs easypanel-traefik --since 1h --tail 100 --timestamps
sudo journalctl --since "1 hour ago" -u docker -u firewalld -u NetworkManager --no-pager
sudo journalctl -k --since "1 hour ago" --no-pager

The first two checks help distinguish a Docker service-address problem from broader container connectivity issues.

Please also mention:

• The outage time and timezone.
• Whether the dashboard remains accessible through SERVER_IP:3000 or its domain.
• Whether automatic updates or firewall management tools are enabled.