Architecture

Architecture

This page explains how the code is organised and how data actually flows through it. It is written from the implementation at main, not from an idealised diagram, and where the shape is awkward it says so.

Package layering

Five internal packages on three layers, and two binaries built on top of them. Dependencies point down only.

graph TD
  MAIN["main (repo root)<br/>→ the netdoc binary"]
  CMD["cmd/netdoc-sim<br/>→ the netdoc-sim binary"]

  UI["internal/ui<br/>Bubble Tea, rendering, tool jobs"]
  SIM["internal/simulation<br/>virtual networks, hunts, challenges"]
  DIAG["internal/diagnostic<br/>target parsing, probes, verdicts"]
  REP["internal/report<br/>JSON report shape"]
  SAFE["internal/textsafe<br/>sanitises untrusted text"]

  MAIN --> UI
  MAIN --> DIAG
  MAIN --> REP
  MAIN --> SAFE
  CMD --> SIM
  CMD --> SAFE
  UI --> DIAG
  UI --> SAFE
  SIM --> DIAG
  SIM --> REP
  SIM --> SAFE
  DIAG --> SAFE

Loading

Note what is not there: no edge between ui and simulation in either direction, and nothing pointing upward. internal/diagnostic does not import internal/report either; the flattening from probe results into the JSON shape happens in main, and the simulator does its own.

Layer	Packages	Rule
0	internal/textsafe, internal/report	Dependency-free leaves.
1	internal/diagnostic	Network semantics. Knows nothing about terminals.
2	internal/ui, internal/simulation	Peers. Neither may depend on the other.

Two consequences worth stating plainly:

• internal/diagnostic must never import internal/ui. Network semantics go in diagnostic; interaction and rendering go in ui. This is what lets the simulator run the diagnosis engine without a terminal.
• internal/simulation never links into netdoc. It is imported by cmd/netdoc-sim alone. The simulator is a second binary, not a second product.

This is not a convention, it is enforced. architecture_test.go in the repository root parses the imports of every production file under internal/ and fails the build if any edge points sideways or up, or if a package appears that has no declared layer. A transitive violation is caught too, because every direct edge is checked.

Flow 1: probes → evidence → diagnosis

This is the core of netdoc, and it is genuinely this simple.

graph LR
  T["Target string<br/>host:port, URL, IP"] --> P["ParseTarget"]
  P --> B["BuildProbes<br/>assemble the DAG"]
  B --> S["Probe selection<br/>--check / --skip"]
  S --> R["RunAll<br/>execute the DAG"]
  R --> RES["map[ProbeID]ProbeResult<br/>the evidence"]
  RES --> D["Diagnose<br/>summary + verdict"]
  D --> OUT["TUI rows<br/>or JSON report"]

Loading

1. Parse. ParseTarget has two independent axes: the port (explicit :port beats a scheme default beats 443) and the protocol rows (an explicit http/https/ssh/smtp scheme wins; otherwise it is inferred from the port). Hosts are validated against a strict allowlist. This is the first trust boundary and it is fuzz-tested (FuzzParseTarget).

2. Build. BuildProbesFromSources assembles a DAG of Probe values. A Probe is deliberately small:

type Probe struct {
    ID   ProbeID
    Name string
    Deps []ProbeID
    Run  func(ctx context.Context, deps map[ProbeID]ProbeResult) ProbeResult
}

Which nodes exist depends on the target's protocol, and on whether a second-opinion resolver is configured, and --public-dns "" omits that node entirely rather than emitting a skipped row.

3. Select. --check / --skip filter the DAG. --check keeps the requested IDs plus their prerequisite closure; --skip removes IDs and anything left without a prerequisite. Validation happens before any probe runs, and rejects unknown IDs with exit 2.

4. Run. RunAll is a fixpoint scheduler. Every probe whose dependencies all have results starts immediately, in its own goroutine, under its own timeout child context. A probe receives an immutable snapshot of just its dependencies' outputs, never the live results map. A failed or skipped dependency records SkipPrereq synchronously, so a chain of doomed probes resolves without ever spawning a goroutine.

The TUI scheduler in internal/ui implements the same semantics for the interactive path. RunAll is the headless one used by --json and by the simulator.

5. The evidence. The result of the whole DAG is map[ProbeID]ProbeResult. ProbeResult carries the typed facts (status, an optional stable Cause, addresses, the selected IP, source address and interface, per-address attempts, per-family connectivity, captive-portal evidence, duration) plus derived human text (Detail, Fix). The human text is never parsed back. Everything downstream reads the typed fields.

Every probe's human-readable fields pass through internal/textsafe on the way out, so a hostile server's banner cannot inject terminal escapes into your terminal.

6. Diagnose. Diagnose(target, order, results) returns (summary, verdict). It is a pure function over the results map, with no I/O, no clock, no network. It has two modes: a truth table over egress/DNS/proxy state for generic runs, and a walk up the protocol stack (DNS → TCP → TLS → HTTP → banner) reporting the first rung that broke for targeted runs. Case order is significant and commented as such, because several cases overlap.

7. Render. internal/ui draws rows; main.go's buildReport flattens the same results into internal/report's stable JSON shape. Both read the same typed results, and there is no second diagnosis path.

Flow 2: the simulator pipeline

netdoc-sim inverts the problem. Instead of observing an unknown network, it builds a network whose fault is known by construction, then grades netdoc's diagnosis against that.

graph TD
  SC["Scenario<br/>YAML: topology, faults, tests, expect"] --> VAL["Validate<br/>before any namespace exists"]
  VAL --> PREP["Backend.Prepare<br/>user + net + mount namespaces"]
  PREP --> FAULT["ApplyFaults"]
  FAULT --> T0(("T0"))
  T0 --> SCHED["Scheduler<br/>timed faults"]
  T0 --> TESTS["Run netdoc, unmodified,<br/>inside the client node"]
  TESTS --> EV["Collect evidence<br/>from kernel + services"]
  SCHED --> EV
  EV --> CMP["Compare diagnosis<br/>against evidence"]
  CMP --> REPT["Report<br/>text or JSON"]
  EV --> CLEAN["Cleanup<br/>namespaces released"]

Loading

The structure that makes this trustworthy:

The launcher holds no privilege. It re-executes a director inside a new user, network, and mount namespace, with the caller's uid mapped to root only there. The director can create bridges, veth pairs, routes, nftables rules, qdiscs, and low-port listeners inside its own namespaces; the kernel gives it no authority over the host network. No root, no sudo, no setuid helper.

netdoc runs unmodified. The simulator nsenters into the selected client node and runs the real binary. It does not reimplement probes or verdict logic, and the process is not handed any token telling it a simulation is happening.

Evidence is read from the kernel, not from the YAML. Routes, neighbour state, nftables counters, and route tables are read back from inside the node after the probes ran. Reachability is measured by the node holder making its own dials from inside its own namespace.

The evidence path never reads the diagnosis. This is the invariant the whole design rests on. Observations independent of netdoc establish truth; the diagnosis is then graded against that truth. Nothing derived from netdoc's report is stored as simulator evidence, which is why no evidence field carries a diagnosis verdict, and why simulator evidence is able to contradict netdoc. That is the point of having it.

Teardown is structural, not best-effort. Node processes carry PDEATHSIG=SIGKILL; when the owning process exits, the kernel reclaims their namespaces and network objects. Nothing is registered under /run/netns or /etc/netns. The cleanup defer is registered so that it runs even on panic or cancellation, under a detached context with its own deadline.

Flow 3: hunt, challenge, and the oracle

Hunts and Challenge Mode are the same machinery with different contestants.

graph LR
  BASE["Known-good<br/>control scenario"] --> MUT["Mutation operator<br/>seeded, deterministic"]
  MUT --> RUN["Ordinary simulation run"]
  RUN --> OBS["observed<br/>reads simulator evidence only"]
  RUN --> RECOG["recognized<br/>reads one diagnosis only"]
  OBS --> JUDGE{"Compare"}
  RECOG --> JUDGE
  HUMAN["Human answer<br/>challenge only"] --> JUDGE
  JUDGE --> RESULT["Finding, or match-up result"]

Loading

The oracle keeps two halves rigidly apart, and this separation is the thing to understand:

• observed reads simulator evidence and derived simulator truth only, never the diagnosis. A mutation that was generated or applied establishes nothing on its own; only the certificate a client actually refused, the CONNECT a proxy actually declined, the kernel counter that actually matched a packet, or the client's own dial of a controlled endpoint does.
• recognized reads one diagnosis only, never simulator evidence.

Recognition is expressed over netdoc's stable cause vocabulary and its structured per-family verdicts, not over probe IDs. That is deliberate: a probe that is renamed, split, or merged without changing what the user is told leaves the oracle correct.

Challenge Mode adds a person as a second contestant on the recognized side, and adds an eligibility contract deciding which conditions may be set at all. Crucially, "Network Doctor already recognizes it" is not part of that contract: a condition netdoc has no vocabulary for is still eligible, and scores as a loss for netdoc rather than being excluded.

The full contract, the four eligibility tests, the evidence predicates, and the scoring rules are documented authoritatively in docs/simulation.md, next to the code that implements them. Hunts and Triage and Challenge Mode explain them conceptually.

Cross-cutting design rules

These hold everywhere and are worth knowing before you write a patch.

Unprivileged, always. No probe needs root. No code path escalates privilege or rewrites configuration. The path-MTU check and the LAN map both use ordinary sockets.

Bounded, always. Every probe has a timeout. Every tool job is cancellable and its output is capped. Every simulation has setup, cleanup, and evidence deadlines.

Argument slices, never shell strings. External commands are built as []string and executed directly. Scenario values never become shell strings either; the backend constructs argument slices from validated logical names and addresses. Scenario authors cannot supply kernel interface names, commands, executable paths, arbitrary proxy URLs, certificate paths, qdisc handles, or raw firewall expressions.

Sanitise everything external. internal/textsafe cleans remote and subprocess text before it reaches a terminal or a report. It is fuzz-tested (FuzzSanitize), and changing it requires running that fuzz target.

OS behaviour lives in build-tagged or platform-suffixed files. All probes, the diagnosis engine, and the TUI are pure Go and identical across Linux, macOS, and Windows. Platform-specific garnish (default gateway, Wi-Fi SSID, socket send queue, OEM code page decoding) is isolated, and degrades to empty rather than failing the probe when unavailable. Release builds are CGO_ENABLED=0, with no cgo.

Where next

• Development and Contributing: building, testing, and the validation gate.
• Simulator Overview: what a scenario is and how a run works.
• How Network Doctor Works: the same probe graph from a user's point of view.