Architecture

This page documents how dockma's parts fit together, for people maintaining it. Each diagram is followed by a table that links every step to the code that does it. The links name a symbol, never a line; scripts/check-doc-refs.py fails CI when a linked file or symbol disappears. It can't tell when a diagram stops matching what the code does, so update this page together with any change to a flow it shows.

Related pages:

• CLAUDE.md: the host–guest protocol table, the file-by-file source map, and the project's rules and gotchas.
• shell-protocol.md: the dockma shell frames.
• debugging.md: stepping through these flows with a debugger, layer by layer.
• README.md: what users see.

1. System overview

flowchart LR
  subgraph mac["macOS"]
    dockercli["docker CLI"]
    cli["dockma CLI"]
    launchd["launchd job io.dockma.daemon"]
    subgraph statedir["~/.dockma"]
      dsock["docker.sock"]
      csock["control.sock"]
      ssock["shell.sock"]
      files["config.json, data.img, guest/, logs/"]
    end
    published["published ports on 127.0.0.1"]
    daemon["dockma daemon<br/>owns the VZVirtualMachine"]
  end
  subgraph guest["Linux guest"]
    agent["dockma-agent"]
    dockerd["dockerd, containerd, runc"]
    hook["/proc/fs/dockma-notify"]
    containers["containers"]
  end
  dockercli --> dsock --> daemon
  cli --> csock --> daemon
  cli -->|dockma shell| ssock --> daemon
  launchd -->|runs dockma daemon| daemon
  published --> daemon
  vsock["vsock<br/>1024 Docker API bytes<br/>1025 control JSON lines, the agent dials<br/>1026 TCP and UDP tunnels<br/>1027 shell frames"]
  daemon <--> vsock <--> agent
  agent --> dockerd --> containers
  agent -->|file changes from the Mac| hook
  daemon -.->|virtiofs home and rosetta, virtio-blk data.img, console ports| guest

Loading

Part	Code	What it is
dockma CLI	Dockma	Every subcommand; it talks to the daemon only through control.sock and launchd.
launchd job	LaunchAgent.plist	Runs dockma daemon; KeepAlive.SuccessfulExit=false restarts it only after a non-zero exit.
daemon	Daemon	One process per state directory; owns the VM and every forward.
vsock ports	VMConfigurationBuilder.Port	1024–1027, duplicated as constants in main. Messages: CLAUDE.md, "Host–guest protocol".
VM devices	VMConfigurationBuilder.build	CPUs, memory, disks, virtiofs shares, vsock, console ports, network.
state directory	Paths	Every file under ~/.dockma (or DOCKMA_HOME).
agent	main	One static Go binary, respawned by busybox init.
kernel hook	dn_apply	Built into virtiofs; turns M|C|D <path> into guest inotify events.

2. Host components

flowchart TB
  subgraph cliproc["dockma CLI process"]
    start["Start, Stop, Restart, Status"]
    shellcmd["ShellCommand"]
    install["Install"]
    cclient["ControlClient"]
    session["ShellSession"]
  end
  subgraph daemonproc["dockma daemon process (main RunLoop)"]
    daemon["Daemon"]
    cserver["ControlServer"]
    host["VMHost"]
    agentclient["AgentClient"]
    forwarder["PortForwarder"]
    listener["SocketListener"]
    relay["DatagramRelay"]
    proxy["VsockSocketProxy"]
    watcher["FileWatcher"]
  end
  launchagent["LaunchAgent"]
  release["GuestRelease"]
  start --> launchagent
  start --> cclient --> cserver --> daemon
  install --> release
  shellcmd --> session -->|shell.sock| proxy
  daemon --> host
  daemon --> agentclient
  daemon --> forwarder --> listener
  forwarder --> relay
  daemon --> proxy
  daemon --> watcher
  agentclient -->|ports| forwarder
  agentclient -->|watch| watcher
  watcher -->|files| agentclient

Loading

Component	Code	Role
start	Start.run	Installs a release guest if needed, loads the job, waits for Docker.
stop	Stop.run	launchctl bootout, then waits for the daemon to exit.
daemon entry	DaemonCommand.run	Creates the Daemon and runs the main RunLoop, not dispatchMain() (see CLAUDE.md).
control client	ControlClient.waitUntilReady	One JSON request, one reply; wait is answered once Docker is ready.
control server	ControlServer.listen	Accepts on control.sock and hands requests to Daemon.handle.
VM wrapper	VMHost.start	Owns VZVirtualMachine; VMHost.listen and VMHost.connect for vsock.
agent link	AgentClient.accept	Takes the agent's connection on 1025 and dispatches its messages in AgentClient.handle.
port forwarding	PortForwarder.update	Turns the agent's port set into listeners and relays.
Docker and shell sockets	VsockSocketProxy.listen	Unix socket → fresh vsock connection → SocketCopier.pipe.
UDP	DatagramRelay	One vsock flow per UDP client, framed by DatagramFrame.
file events	FileWatcher.watch	FSEvents on the bind mounts the agent names.
launchd	LaunchAgent.install	Writes the plist and bootstraps the job.
guest install	GuestRelease.install	Downloads, verifies and swaps in a release guest.
shell client	ShellSession.run	Raw TTY, signals and stdin flow control over shell.sock.

3. Guest components

flowchart TB
  stage0["stage-0 /init<br/>initramfs into tmpfs, switch_root"]
  init["busybox init (inittab)"]
  rcs["rcS: mounts, DHCP, data disk,<br/>home and rosetta shares, CA certs"]
  disk["dockma-disk"]
  dockerdinit["init.d/dockerd:<br/>dockma-agent daemon-config, dockerd"]
  shutdown["init.d/shutdown"]
  subgraph agentproc["dockma-agent goroutines"]
    serveDocker["serve 1024: proxy"]
    serveTunnel["serve 1026: handleTunnel"]
    serveShell["serve 1027: handleShell"]
    control["dialControl, handleControl"]
    watcher["portWatcher: events, relist"]
    dns["serveDNS on 10053"]
    fileq["fileForwarder queue and worker"]
    echo["echoFilter: fanotify"]
    trim["trimDataDisk"]
  end
  hook["dockma_notify.c in virtiofs"]
  stage0 --> init
  init -->|sysinit| rcs --> disk
  init -->|respawn| agentproc
  init -->|respawn| dockerdinit
  init -->|shutdown| shutdown
  watcher -->|ports, watch, docker ready| control
  control -->|files| fileq --> hook
  echo -.->|guest writes| fileq

Loading

Component	Code	Role
stage 0	init	Moves the initramfs into a tmpfs, because runc can't pivot_root out of rootfs.
init table	inittab	rcS once; the agent and dockerd respawned; shutdown before poweroff.
boot	cmdline_path	Decodes dockma.home= and dockma.state= from the cmdline built by KernelCmdline.commandLine.
data disk	dockma-disk	Formats only a blank disk; checks and grows an existing one.
network	default.script	DHCP lease, resolv.conf, the nft redirect of port 53 to the agent.
dockerd	dockerd	Generates and validates the config with runDaemonConfig, then execs dockerd.
Docker proxy	proxy	Served by serve, which admits only the host (isHostPeer).
tunnels	handleTunnel	Reads tcp|udp <host:port>; UDP goes to relayUDP.
shell	handleShell	Sessions on a pty or pipes.
control	handleControl	Sends hello, docker, ports, watch; applies settime, files, shutdown. Redialed by dialControl.
container watcher	portWatcher.relist	Docker events → PublishedPorts and WatchedPaths.
DNS	serveDNS	host.docker.internal, relay to the lease's servers.
file events	fileForwarder.enqueue	Bounded queue; the worker runs fileForwarder.apply.
echo filter	echoFilter.recordGuestWrites	Records the guest's own writes, so the Mac's echoes of them are dropped.
kernel hook	dn_write	/proc/fs/dockma-notify; dn_apply invalidates and raises events.
shutdown	shutdown	Stops dockerd, then unmounts the data disk.

4. Flows

4a. dockma start

sequenceDiagram
  participant CLI as dockma start
  participant LD as launchd
  participant D as daemon
  participant VM as VZ VM
  participant A as agent
  participant DK as dockerd
  CLI->>CLI: install release guest if needed, validate guest protocol
  CLI->>LD: write plist, bootstrap job
  LD->>D: run dockma daemon
  D->>D: take daemon.lock, listen on control.sock
  CLI->>D: connect to control.sock (kqueue wakes on the socket or the daemon's exit)
  CLI->>D: wait
  D->>D: load config, prepare data disk, build VM config
  D->>VM: listen on vsock 1025, then start
  D->>D: VM running: serve docker.sock and shell.sock
  VM->>A: boot: stage 0, rcS, init respawns agent and dockerd
  A->>D: dial 1025, hello
  D->>A: settime
  A->>DK: wait for docker.sock (inotify), open the /events stream
  A->>D: docker ready
  D-->>CLI: status (answers the wait)
  CLI->>D: ping through docker.sock

Loading

Step	Code
guest checks	installReleaseGuest, GuestImage.validate
job	Start.loadJob → LaunchAgent.install
waiting for the socket	Start.connectToDaemon, KernelEvents
daemon start	Daemon.start → Daemon.boot → Daemon.vmStarted
agent connects	dialControl → AgentClient.accept; onConnect in Daemon.boot sends settime
Docker readiness	fileWaiter.wait, then portWatcher.watch calls portWatcher.setDockerReady once the stream opens → Daemon.answerReadyWaiters
final check	DockerPing.isResponding

A daemon that fails during startup exits 0, so launchd doesn't loop. start sees the control connection close or the process exit, and prints the daemon log.

4b. A published port

sequenceDiagram
  participant U as docker run -p 8080:80
  participant DK as dockerd
  participant A as agent
  participant D as daemon
  participant C as client on the Mac
  U->>DK: create and start container
  DK-->>A: container event on /events
  A->>DK: GET /containers/json
  A->>D: ports [8080/tcp to 127.0.0.1:8080]
  D->>D: PortForwarder.update binds 127.0.0.1:8080
  C->>D: connect 127.0.0.1:8080
  D->>A: vsock 1026, line "tcp 127.0.0.1:8080"
  A->>DK: dial 127.0.0.1:8080 (docker-proxy)
  Note over C,DK: bytes copied both ways, with half-close

Loading

Step	Code
events stream	portWatcher.watch → portWatcher.relist → PublishedPorts
sending changes only	portWatcher.update, handleControl
host side	AgentClient.handle → PortForwarder.update → PortForwarder.plan
per connection	PortForwarder.tunnel → SocketCopier.pipe
guest side	handleTunnel → parseTunnelHeader → pipe
UDP instead	DatagramRelay ↔ relayUDP, one connection per client, idle after 90 s
port held on the Mac	PortForwarder.retryFailed, every 10 s from Daemon.vmStarted

4c. File changes on the Mac

sequenceDiagram
  participant A as agent
  participant D as daemon
  participant FS as FSEvents
  participant H as kernel hook
  participant W as watcher in a container
  A->>D: watch [bind sources under home]
  D->>D: keep real paths in home, at most 256
  D->>FS: start stream on those paths
  FS-->>D: changed paths and flags
  D->>D: lstat each: gone is D, new is C, else M
  D->>A: files batches, each line under 32 KiB
  A->>A: bounded queue, worker
  A->>H: C path
  A->>A: echo of a guest write? (inode, mtime, size)
  A->>H: M path, unless an echo
  H-->>W: inotify CREATE, MODIFY, CLOSE_WRITE

Loading

Step	Code
which paths	WatchedPaths, sent by handleControl
trust boundary	FileWatcher.allowedPaths
mapping and batching	FileWatcher.event, FileWatcher.batches
guest queue	fileForwarder.enqueue → fileForwarder.apply
echo filter	echoFilter.recordGuestWrites, fileForwarder.isEcho
hook	dn_apply

C goes to the hook before the echo check, because the check's lstat would cache the name, and the hook would then treat the file as one the guest already knew.

4d. dockma stop

sequenceDiagram
  participant CLI as dockma stop
  participant LD as launchd
  participant D as daemon
  participant A as agent
  participant G as guest init
  CLI->>LD: launchctl bootout
  LD->>D: SIGTERM
  D->>D: requestStop: close forwards and listeners
  D->>A: shutdown
  A->>G: poweroff
  G->>G: init.d/shutdown: stop dockerd, unmount data disk
  G-->>D: guest stopped
  D->>D: exit 0
  CLI->>CLI: kqueue sees the daemon exit
  Note over D: no agent: hard stop after 2 s<br/>shutdown sent: hard stop 25 s after the request

Loading

The deadlines must stay in this order:

Deadline	Value	Code
dockerd's own shutdown	15 s by default	shutdown-timeout, each container's stop timeout plus 5 s
guest script's wait	20 s	shutdown
hard stop	25 s	Daemon.shutdownGrace, Daemon.scheduleForceStop
launchd's SIGKILL	30 s	ExitTimeOut in LaunchAgent.plist
dockma stop gives up	40 s	LaunchAgent.bootout

The daemon's exit code tells launchd what to do (Daemon.exitCode). It is 0 after a requested stop, and 1 after an unexpected poweroff or a VM error, which launchd restarts. A guest reboot (panic=5) doesn't stop the VM; the agent reconnects.

4e. dockma shell

ShellSession.run connects to shell.sock. The daemon only copies bytes to vsock 1027 (VsockSocketProxy.listen), and handleShell runs the session. The frames and session rules are in shell-protocol.md.

5. Build and release

flowchart LR
  subgraph guestbuild["vm/Dockerfile (make guest)"]
    ksrc["kernel-src"] --> kernel["kernel"]
    ksrc --> cpio["cpiotool"]
    agentstage["agent: go vet, go test, build"]
    dockerstage["docker: static bundle"]
    rootfs["rootfs"]
    kernel --> out["dockma-kernel, dockma-initrd.img"]
    cpio --> rootfs
    agentstage --> rootfs
    dockerstage --> rootfs
    rootfs --> out
  end
  tag["git tag X.Y.Z"] --> release["release.yml"]
  release --> guestyml["guest.yml"] --> guestbuild
  release --> cliyml["cli.yml: make test, signed binary, docc check"]
  release --> publish["publish: SHA256SUMS, attestations, draft release"]
  publish --> installer["install.sh"]
  publish --> dinstall["dockma install"]
  installer --> dinstall
  dinstall --> guestdir["~/.dockma/guest:<br/>kernel, initrd, guest-protocol, origin"]

Loading

Step	Code
guest stages	kernel, agent, rootfs, with versions pinned as ARGs
kernel config gates	dockma.config, check-docker-config.sh
CI	cli.yml, guest.yml, lint.yml
release	publish, notes from release-notes.py
installer	latest_version, verify_attestation
guest install	GuestRelease.install, GuestImage.origin
protocol check	GuestImage.protocolVersion must equal protocol

6. Where to change what

Changes that must touch several places at once:

• A new control message (vsock 1025): AgentMessage and Go's message in main, AgentClient.handle and handleControl, and tests on both sides. Both sides ignore unknown types, so a new optional message needs no protocol bump; an incompatible one bumps GuestImage.protocolVersion and protocol together.
• A vsock port: VMConfigurationBuilder.Port and the constants in main.
• Shell frames: ShellFrame, readFrame, the golden bytes in both test suites, and shell-protocol.md.
• A kernel option: dockma.config. The Dockerfile's olddefconfig gate fails if it's dropped. Prove the feature works with a probe in e2e.sh.
• Agent behavior: bump version in main: the minor version for a feature, the patch for a fix.
• A CLI flag or command: regenerate the docc reference (CLAUDE.md, "What this is") and update the README.
• A shutdown timing: keep the order in 4d.
• Anything under vm/: takes effect only after make guest install-guest and dockma restart.