Skip to content

Netdryrun v0.3.3

Choose a tag to compare

@github-actions github-actions released this 30 Oct 03:54
· 156 commits to main since this release

Advisory: 0.3.3 and earlier can pass on a lab whose topology exec: command failed (containerlab logs it and still exits 0). Put routing config in setup: (or, on FRR, in frr.conf), not in topology exec:. Fixed in 0.4. Details: advisories/0.3.3-exec-not-checked.

The agent runtime. 0.3.3 lets an AI agent get a verdict without local Docker, answers MCP calls within seconds, and makes record write baselines that survive normal growth. The verdict strings, exit codes and the netdryrun.mcp-result/v1 schema are unchanged.

  • Fix for a false pass in 0.3.2: an FRR lab that skipped a config line. FRR loads its config line by line at boot and skips a line it rejects (a typo such as default-orignate, an incomplete command). 0.3.2 ran the checks on the lab without it, so a run could pass on a change that never loaded. 0.3.3 reads every FRR node's boot log after the checks: any rejected line makes the run an error (exit 2), naming the node, line and reason. A line a daemon answered with "Failure to communicate" but applied anyway doesn't count: that daemon's own running config has it in the same section, with its whole block as written. Every other rejection counts, and so does anything that can't be read. Arista cEOS, SR Linux and SONiC startup config isn't checked yet (a documented limit, fix scheduled for 0.4); config applied with setup: commands is. Advisory: advisories/0.3.2-rejected-config.
  • Correction: Verdict Corpus v2 is 16 of 17 deterministic cases, not 17/18. The release gate found that v2 case 18 (18-aggregate-discard-route-removed) gives different verdicts on the same build: FRR's BGP import check sometimes withholds the aggregate and sometimes announces it (three reruns: unsafe, unsafe, safe). We had credited its earlier flip to 0.3.1; that was wrong. The case is now listed as nondeterministic with every verdict seen and left out of the score; the case and its expected verdict are unchanged. Corpus results.
  • A verdict through a pull request, for agents without Docker. netdryrun run --verdict FILE (and check) writes the agent result, the same object the MCP tools return, with the files tested. The GitHub Action writes it (verdict-report) and uploads it. Its PR comment now starts with a fixed, machine-readable header: status, verdict, counts and the head commit. There is one comment per lab, and only a comment by comment-author (default github-actions[bot]) is updated. The header is held to the run's checked exit code: a missing or contradicting result reads error. Guide: AI agents without Docker.
  • Asynchronous MCP: start_run returns a run_id at once, and get_result polls until the result, which is exactly dry_run's. A dry run takes 60–120 s, longer than some clients wait for one tool call. dry_run is unchanged. Each run writes its own evidence file, and the server destroys the labs it started when the client disconnects.
  • A run whose files changed while it ran is an error. The result hashes the files again at the end; if any changed, it reads status: error and lists them in tested.changed_during_run.
  • record writes prefix-count ranges by default (today's count as min, 20% headroom as max), instead of exact counts that broke on normal growth. --exact-prefixes keeps the old behaviour.
  • record now adds the standard guards; --no-guards to opt out. On every BGP node it writes no default route via BGP and no link subnet of the topology via BGP. Each guard is run on the recording lab first; one the lab already fails is left out, with a comment in the file saying why, and one that can't be evaluated is an INCOMPLETE BASELINE line. A prefix range has room for one leaked default route, and the guard is what catches it. The MCP record_baseline tool does the same (guards: false to opt out).
  • A partial baseline is loud: more routes than record writes, a failed route read, or a link with no address where Netdryrun reads it (SONiC, SR Linux) is an # INCOMPLETE BASELINE line at the top of the file.
  • Driver fixes from the cross-platform and delta reviews: a link-local neighbor that several sessions share (fe80::1 on two interfaces) is never evaluated, since either session's state or count would answer for the other: check a global address (FRR, SR Linux, EOS); on FRR a route check never passes on a graceful-restart stale path; a link-local next_hop in a bgp_route check is refused (two sessions can share it, so the check couldn't say which path it tests); a path with no MED attribute no longer passes med: 0 (its MED is unknown, and with missing-as-worst it is the worst, not 0); every SR Linux BGP route type (EVPN, L3VPN, labeled) counts as BGP; EOS Idle(Admin) reads as idle.
  • Known limits, stated, fix scheduled for 0.4: on SR Linux, EOS and SONiC a graceful-restart stale path still counts as present (leave GR off in labs, or use a timeout above the stale timer); SR Linux prefix counts include routes dropped by loop detection (pair the count with route checks); record snapshots the default VRF only (add checks for other VRFs by hand).
  • Exact MCP setup for Cursor, VS Code, Windsurf and Codex CLI on the AI agents page.
  • How this release was checked: releases/0.3.3/GATE.md records static checks (now including a check for retired public strings), unit tests on Python 3.10–3.14, mutation testing of the verdict logic, and the live lab matrix with chaos tests. Each Verdict Corpus has its own line listing the harmful changes it approves (the known, published misses), so any new one blocks the release. review.md holds the reviews: a cold code review, a multi-vendor AI audit (Grok, ChatGPT and a local open model), an adversarial review of every surviving mutant in a verdict-deciding function, a cross-platform review of the drivers, and a fresh review of every change made after them. Every finding has its decision.

Full list: CHANGELOG.