debug-container

A purpose-built Kubernetes debug container image. Attach it to any running pod as an ephemeral container — get a fully-equipped debugging shell without modifying the target container.

Why?

Production containers are stripped to the bare minimum — no shell, no curl, no tcpdump. When something goes wrong, kubectl exec can't help if the tools aren't there. This image gives you a complete debugging toolkit that attaches alongside the running container, keeping the application untouched.

What's Included

Category	Tools
Shell	zsh, Oh My Zsh (agnoster theme), syntax highlighting, autosuggestions, fzf (Ctrl+R)
Editors	neovim
Networking	dig, nslookup, curl, wget, tcpdump, nmap, netcat, traceroute, iproute2
Process/System	htop, strace, ltrace, lsof, procps, sysstat
Kubernetes	kubectl (auto-configured with target's service account)
Utilities	jq, tmux, tree, file, less, unzip, git

Quick Start

Interactive Session

kubectl debug -it <pod-name> \
  --image=docker.io/k7bdevs/debug-container:latest \
  --target=<container-name> \
  --share-processes \
  --profile=general \
  -- zsh

Note: When you exit, the ephemeral container dies and cannot be restarted.

Persistent Session (Recommended)

Start the container in the background, then open shells with kubectl exec — exit freely without killing the debug container.

# Start (no -it flag)
kubectl debug <pod-name> \
  --image=docker.io/k7bdevs/debug-container:latest \
  --target=<container-name> \
  --share-processes \
  --profile=general \
  -- sleep infinity

# Get the debugger container name
DEBUGGER=$(kubectl get pod <pod-name> -o jsonpath='{.spec.ephemeralContainers[-1].name}')

# Open a shell (repeatable — exit as many times as you want)
kubectl exec -it <pod-name> -c $DEBUGGER -- zsh

Shell Features

On startup, the debug container automatically:

• Lands you in the target's filesystem — cd /proc/1/root
• Imports the target's environment variables — KUBERNETES_SERVICE_HOST, app-specific vars, etc.
• Configures kubectl — discovers the service account token from the target container

Built-in Commands

Command	Description
t <cmd>	Run a command inside the target container's rootfs via chroot (e.g., t mongosh, t bash)
cdtarget	cd into the target container's filesystem (/proc/1/root)
k	Alias for kubectl
Ctrl+R	Fuzzy search command history (fzf)

The t Command

The target container has binaries (like mongosh) that aren't in the debug container. Running them directly fails because they resolve dependencies against the debug container's /. The t command uses chroot to switch the root filesystem:

t mongosh          # MongoDB shell with syntax highlighting
t cat /etc/hosts   # Read the target's files
t bash             # Drop into the target's shell

Building

# Using build.sh
export IMAGE="docker.io/youruser/debug-container"
export TAG="latest"
./build.sh

# Or directly with docker buildx
docker buildx build --platform linux/amd64,linux/arm64 \
  -t docker.io/youruser/debug-container:latest --push .

How It Works

Shared PID Namespace (--share-processes)

All containers in the pod share the same PID namespace. The debug container can see the target's processes and access their /proc/<pid>/root filesystem.

/proc/1/root

Linux exposes every process's root filesystem at /proc/<pid>/root. With shared PIDs, PID 1 is the target container's main process — so /proc/1/root is the gateway to its entire filesystem.

chroot for Target Binaries

cd /proc/1/root lets you browse files, but running binaries fails because absolute paths (/usr/lib/...) still resolve against the debug container's /. chroot /proc/1/root changes what / means, so all paths resolve correctly against the target's filesystem.

--profile=general

Grants SYS_PTRACE capability, which is required to access /proc/<pid>/root across different container mount namespaces.

Non-root Pod Support

Ubuntu 24.04 ships a ubuntu user (uid=1000, gid=1000). When pods enforce runAsUser: 1000, the debug container inherits this user. The image sets uid 1000's primary group to root (gid=0) so that /proc/<pid>/root symlinks (owned by <uid>:root) remain accessible.

File Structure

├── Dockerfile       # Image definition
├── build.sh         # Multi-arch build script
├── entrypoint.sh    # HOME bootstrap for non-root users
├── zshenv.global    # /etc/zsh/zshenv — writable HOME setup (runs first)
└── zshrc.global     # /etc/zsh/zshrc — shell config, plugins, debug helpers

License

MIT