Netdryrun v0.4.0

Verify a change against its intent. 0.4.0 adds netdryrun verify: declare what a change is meant to do, and every other difference in the network's behavior fails. It boots the lab before and after the change, snapshots both, and matches the difference against the intent. No checks to write. FRR only in 0.4; other nodes are refused or named as unobserved.

• netdryrun verify (CLI), the MCP tool verify_change, and the GitHub Action's mode: verify: one engine. The verdicts are INTENT MATCHED, NO BEHAVIOR CHANGE, UNDECLARED CHANGES: DO NOT APPLY and INCONCLUSIVE. verify never prints SAFE TO PROCEED, and every report echoes the intent: explained is not the same as safe. An approval takes four boots (each side twice), and a side that doesn't reproduce is INCONCLUSIVE. The standing guards in netdryrun.yml come from the base side, never from the change. Reference: verify.
• The snapshot covers BGP sessions, VRFs, routes of every protocol (selected, FIB-installed, next hops), every BGP path with its attributes, the Adj-RIB-Out of every session and each node's running config. A config change with no observable effect in the lab fails unless declared, and every approval lists the changed config lines the lab didn't exercise.
• The Action judges what lands: the PR merged into the base branch. A PR that edits the intent file, the workflow judging it, or the checks together with the configs fails before any lab boots, unless a code-owner review of that file is required and the workflow opts in.
• Fixes for false passes in 0.3.3 (live mixed-vendor interop labs: cJunosEvolved, cEOS, SR Linux, FRR):
  • Juniper: a path with no MED read as MED 0, so med: 0 passed on a route sent without a MED. And bgp_prefixes read Junos's accepted count, which also leaves out a received route hidden for an unusable next hop. It now subtracts only the routes rejected by import policy, as the other drivers count. A hidden route with a reason the driver doesn't know leaves the count a range, and a bound is judged only when the whole range agrees.
  • SR Linux: a route rejected by import policy counted as received, so active: false passed on it and present: false failed. It is now left out, as on every other driver.
  • Arista EOS: cEOS prints MED 0 for a path with no MED, so a med: 0 check on EOS is refused before the lab boots (exit 2). AS sets are read in the same form as every other driver.
  • A topology exec: command that fails is an error on every platform (containerlab only logs it). Startup config a NOS rejected is never a pass on Arista cEOS, SR Linux and SONiC too (FRR since 0.3.3). Advisory: advisories/0.3.3-exec-not-checked.
  • A repeated key in device JSON is never "the last one wins".
• Advertisement intervals stretch --settle and --hold, so a leak sent after the default windows can't let a negative check pass on a count still at 0. record covers every active VRF on FRR and SONiC.
• A machine-readable capability statement: capabilities.json, generated from the source (on netdryrun.com once the site deploys 0.4).
• The repository is now github.com/netdryrun/Netdryrun. Old links and uses: netdryrun/netdryrun@vX keep working through GitHub's redirect.
• Moved to 0.4.1: verify on an FRR lab with equal multipath paths to an advertised prefix gives no verdict (exit 2, never an approval); reading the graceful-restart stale flag on SR Linux, EOS and SONiC; counting only usable routes in SR Linux prefix counts; EOS out-delay.
• How this release was checked: releases/0.4.0/GATE.md records static checks, unit tests on Python 3.10–3.14, mutation testing of the verdict logic (the verify modules included), and the live lab matrix with both Verdict Corpora and chaos tests. review.md holds the reviews: a cold code review, an expert gauntlet, a multi-vendor audit, an adversarial review of every mutation acceptance, and two runs of a mixed-vendor interop test. Every finding has its decision.

Full list: CHANGELOG.

Netdryrun v0.3.3

Advisory: 0.3.3 and earlier can pass on a lab whose topology exec: command failed (containerlab logs it and still exits 0). Put routing config in setup: (or, on FRR, in frr.conf), not in topology exec:. Fixed in 0.4. Details: advisories/0.3.3-exec-not-checked.

The agent runtime. 0.3.3 lets an AI agent get a verdict without local Docker, answers MCP calls within seconds, and makes record write baselines that survive normal growth. The verdict strings, exit codes and the netdryrun.mcp-result/v1 schema are unchanged.

• Fix for a false pass in 0.3.2: an FRR lab that skipped a config line. FRR loads its config line by line at boot and skips a line it rejects (a typo such as default-orignate, an incomplete command). 0.3.2 ran the checks on the lab without it, so a run could pass on a change that never loaded. 0.3.3 reads every FRR node's boot log after the checks: any rejected line makes the run an error (exit 2), naming the node, line and reason. A line a daemon answered with "Failure to communicate" but applied anyway doesn't count: that daemon's own running config has it in the same section, with its whole block as written. Every other rejection counts, and so does anything that can't be read. Arista cEOS, SR Linux and SONiC startup config isn't checked yet (a documented limit, fix scheduled for 0.4); config applied with setup: commands is. Advisory: advisories/0.3.2-rejected-config.
• Correction: Verdict Corpus v2 is 16 of 17 deterministic cases, not 17/18. The release gate found that v2 case 18 (18-aggregate-discard-route-removed) gives different verdicts on the same build: FRR's BGP import check sometimes withholds the aggregate and sometimes announces it (three reruns: unsafe, unsafe, safe). We had credited its earlier flip to 0.3.1; that was wrong. The case is now listed as nondeterministic with every verdict seen and left out of the score; the case and its expected verdict are unchanged. Corpus results.
• A verdict through a pull request, for agents without Docker. netdryrun run --verdict FILE (and check) writes the agent result, the same object the MCP tools return, with the files tested. The GitHub Action writes it (verdict-report) and uploads it. Its PR comment now starts with a fixed, machine-readable header: status, verdict, counts and the head commit. There is one comment per lab, and only a comment by comment-author (default github-actions[bot]) is updated. The header is held to the run's checked exit code: a missing or contradicting result reads error. Guide: AI agents without Docker.
• Asynchronous MCP: start_run returns a run_id at once, and get_result polls until the result, which is exactly dry_run's. A dry run takes 60–120 s, longer than some clients wait for one tool call. dry_run is unchanged. Each run writes its own evidence file, and the server destroys the labs it started when the client disconnects.
• A run whose files changed while it ran is an error. The result hashes the files again at the end; if any changed, it reads status: error and lists them in tested.changed_during_run.
• record writes prefix-count ranges by default (today's count as min, 20% headroom as max), instead of exact counts that broke on normal growth. --exact-prefixes keeps the old behaviour.
• record now adds the standard guards; --no-guards to opt out. On every BGP node it writes no default route via BGP and no link subnet of the topology via BGP. Each guard is run on the recording lab first; one the lab already fails is left out, with a comment in the file saying why, and one that can't be evaluated is an INCOMPLETE BASELINE line. A prefix range has room for one leaked default route, and the guard is what catches it. The MCP record_baseline tool does the same (guards: false to opt out).
• A partial baseline is loud: more routes than record writes, a failed route read, or a link with no address where Netdryrun reads it (SONiC, SR Linux) is an # INCOMPLETE BASELINE line at the top of the file.
• Driver fixes from the cross-platform and delta reviews: a link-local neighbor that several sessions share (fe80::1 on two interfaces) is never evaluated, since either session's state or count would answer for the other: check a global address (FRR, SR Linux, EOS); on FRR a route check never passes on a graceful-restart stale path; a link-local next_hop in a bgp_route check is refused (two sessions can share it, so the check couldn't say which path it tests); a path with no MED attribute no longer passes med: 0 (its MED is unknown, and with missing-as-worst it is the worst, not 0); every SR Linux BGP route type (EVPN, L3VPN, labeled) counts as BGP; EOS Idle(Admin) reads as idle.
• Known limits, stated, fix scheduled for 0.4: on SR Linux, EOS and SONiC a graceful-restart stale path still counts as present (leave GR off in labs, or use a timeout above the stale timer); SR Linux prefix counts include routes dropped by loop detection (pair the count with route checks); record snapshots the default VRF only (add checks for other VRFs by hand).
• Exact MCP setup for Cursor, VS Code, Windsurf and Codex CLI on the AI agents page.
• How this release was checked: releases/0.3.3/GATE.md records static checks (now including a check for retired public strings), unit tests on Python 3.10–3.14, mutation testing of the verdict logic, and the live lab matrix with chaos tests. Each Verdict Corpus has its own line listing the harmful changes it approves (the known, published misses), so any new one blocks the release. review.md holds the reviews: a cold code review, a multi-vendor AI audit (Grok, ChatGPT and a local open model), an adversarial review of every surviving mutant in a verdict-deciding function, a cross-platform review of the drivers, and a fresh review of every change made after them. Every finding has its decision.

Full list: CHANGELOG.

Netdryrun v0.3.2

Corrected in 0.3.3: Verdict Corpus v2 is 16 of 17 deterministic cases, not 17/18 (case 18 gives different verdicts on the same build and is now left out of the score). 0.3.2 could also pass an FRR change whose config line was rejected at boot: see the advisory. Upgrade to 0.3.3.

The agent contract. 0.3.2 is about what an AI agent needs to choose Netdryrun, read its result and act on it safely. The CLI, its exit codes, the JSON report and the GitHub Action keep their behaviour, apart from the fixes below.

Breaking change for agents: the pass verdict is renamed. The MCP result's pass verdict SAFE TO PROCEED is now CHECKS PASSED (only what the checks cover): a pass covers only what the checks look at. An agent, prompt or AGENTS.md rule that matched the old text no longer sees it, so it treats a pass as not-a-pass (it fails closed). Decide on the status field, not the verdict text. The three verdict strings are now frozen for v1.

• A typed result. dry_run and run_checks results follow a published JSON Schema, mcp-result-v1.json. A failed result lists each failed check with a one-line reason, an evidence excerpt and one read-only next_diagnostic command for its node.
• Results name what they tested: tested gives the netdryrun.yml that ran and a sha256 of it, the topology and every file the topology uses, read before the run. A verdict applies to those exact files.
• Tool descriptions carry the rules, because small models read only the tool text: write the change first, a failed run means fix and rerun, inconclusive means stop, never weaken checks, use record_baseline when there are none. An agent skill (skills/netdryrun/SKILL.md) and an AGENTS.md snippet carry the same rules.
• A router whose BGP config never loaded no longer passes its leak guards. A node the file says runs BGP that still reports no BGP neighbors at the convergence gate's cap now fails its post-convergence checks as "not evaluated". Found by this release's multi-vendor AI audit. Still a limit: a peer with a long advertisement interval can deliver a route late; raise --settle and --hold above it.
• A lab that never reached a testable state is inconclusive (exit 3), with failure scenarios in the file too, and when the only readable answers were pings with no replies (bgpd off on every router).
• MCP paths: a relative path is read from NETDRYRUN_MCP_ROOT when that is one folder, and refused with an example when it names two different files; the report is written to a new file and renamed into place.
• An IPv6 scope in a check address must be an interface name: text after % reached device CLI commands.
• Network Change Verdict v1, an open spec for the report format: spec/verdict-v1.
• Verdict Corpus rerun on the built 0.3.2 package in the release gate: every verdict matches the published results (v1 24/30, v2 17/18).
• How this release was checked: releases/0.3.2/GATE.md records static checks, unit tests on Python 3.10–3.14, mutation testing of the verdict logic, the live lab matrix with chaos tests, and review.md: a cold code review and a multi-vendor AI audit (Grok, ChatGPT and a local open model), with every finding and its decision.

Full list: CHANGELOG.

Netdryrun v0.3.1

Upgrade if you use 0.3.0. A pre-launch audit found ways 0.3.0 could report passed without testing what the file said. 0.3.1 closes them, along with the security fixes listed in the changelog. 0.3.0 is yanked on PyPI: exact pins still install it.

Run netdryrun validate after upgrading. Some files 0.3.0 accepted are now refused (exit 2). Each of those was a check that didn't test what it said: a misspelled top-level key, a quoted "false", an unknown afi or protocol, a duplicate key, min: 0, vrf: all.

• Advisory, with files you can rerun in seconds against both versions: advisories/0.3.0-false-pass
• Failure scenarios are judged only once the fault is visible in BGP. Negative checks wait for every BGP session on their node.
• Drivers never read an error as "nothing": an unknown VRF, a failed command or unrecognized output is an error, not "no routes".
• New: the inject_routes fault, the srl-hidden-leak example, a JSON Schema for reports, MCP path handling on WSL, and pipx install netdryrun in the docs (Ubuntu 24.04 refuses a plain pip install).
• Verdict Corpus rerun on the released 0.3.1: v1 24/30 (unchanged), v2 17/18 (case 18, a removed aggregate discard route, was a false pass on earlier builds and is now rejected). These are fixed test sets that guard against regressions, not an accuracy rate for your network: results and method.
• How this release was checked: releases/0.3.1/GATE.md (cleared, including the proof from PyPI after publishing). The record covers the static checks, unit tests on Python 3.10–3.14, verdict invariants, mutation testing of the verdict logic (every surviving mutant killed or shown harmless), a live lab matrix with chaos tests, and an independent review. The wheel and sdist carry signed build provenance (netdryrun-v0.3.1.sigstore.json).

Full list: CHANGELOG.

Netdryrun v0.3.0

Full Changelog: v0.2.0...v0.3.0

Netdryrun v0.2.0

Full Changelog: v0.1.0...v0.2.0

NetDryRun v0.1.0

Full Changelog: https://github.com/netdryrun/netdryrun/commits/v0.1.0