v0.5.2.0 — mesh routing hardening, telemetry, Linux swarm ops

0.5.2.0: mesh routing hardening, telemetry, and Linux swarm ops

Minor release since v0.5.1.0 (2026-08-11 DMG). Rolls up routing pool fixes, opt-in gateway-led mesh capacity coordination, windowed telemetry, discovery probe hygiene, and Linux swarm helper scripts. Includes everything in 0.5.1.0.

New

• Gateway-led mesh capacity coordinator (opt-in, PR #71) — mesh_coordinator gateway mode with heartbeat routing_capacity gossip, mesh_capacity selection helpers, and local agent.max_concurrency self-admission. Dashboard and macOS Settings expose the controls. Default off preserves legacy decentralized selection.
• Windowed telemetry traffic rows (PR #66) — GET /netllm/v1/telemetry and dashboard Home / Serving views show windowed source/scenario throughput; scroll position preserved across dashboard tab refreshes.
• Linux swarm ops helpers (PR #53) — packaging/linux/install-swarm-firewall.sh and install-ollama-concurrency.sh for LAN mesh firewall rules and Ollama concurrency tuning on Debian/RPM installs.

Fixed

• Pool-scoped overflow (PR #71) — Separate model pools on one host no longer union into cross-capability overflow (chat requests could rewrite to embedding models). Overflow is scoped to pools listing the requested name, filtered by capability, with one served model bound per (pool, host).
• Sticky host binding (PR #71) — Pool effectiveness checks on dashboard and macOS align with alias-aware candidacy.
• Symmetric pool spillover on gateways (PR #70) — local_spillover tiered overflow among pool hosts works symmetrically when the gateway's local tier is saturated.
• Mesh candidacy for pool overflow (PR #68) — Unified mesh candidacy so pool overflow reaches LAN peers instead of collapsing to local-only substitution.
• Discovery best-probe selection (PR #52) — Prefer the probe URL with the richest model catalog; skip zero-model URL merge when provider_urls lists multiple ports.
• macOS pool model matcher (PR #71) — @MainActor Settings helper marked nonisolated for menubar-lifecycle CI.

Upgrade

macOS (notarized DMG)

Download llm-swarm-router.dmg from GitHub Releases or upgrade in place:

packaging/scripts/macos-app-install.sh --dmg ~/Downloads/llm-swarm-router.dmg

Menubar Updates → Check for Updates… picks up the release when the DMG asset is published.

From a git checkout (macOS 26+):

git checkout v0.5.2.0
uv sync
apps/netllm-mac/Scripts/build.sh release
packaging/scripts/macos-app-install.sh --source apps/netllm-mac/build/Stage/llm-swarm-router.app

CLI / Linux / Windows

git checkout v0.5.2.0
uv sync
./netllm restart   # or systemctl --user restart netllm / menubar Restart Agent

Hard-refresh the dashboard (Cmd+Shift+R) if cached dashboard.js is stale.

Verify:

curl -sf http://127.0.0.1:11400/health
./netllm doctor
./netllm test
open http://127.0.0.1:11400/ui/

Test plan (release gate)

• ./scripts/verify-before-pr.sh --full
• ./scripts/ci.sh e2e
• ./scripts/live-routing-smoke.sh (maintainer LAN mesh, optional)

Previous release

• v0.5.1.0 notes — dashboard section-scoped save

v0.5.1.0 — dashboard section-scoped save

0.5.1.0: dashboard section-scoped save

Minor release fixing a dashboard save path that re-ran cloud credential verification on unrelated Network/Swarm edits. Includes everything in 0.5.0.1.

Fixed

• Section-scoped dashboard save (PR #65) — Save POSTs only config sections whose wire patch changed from the last-saved snapshot. Server-side apply_config_guards(..., patch=...) skips cloud provider verification when the patch omits cloud, so saving a cluster token or swarm toggle no longer surfaces Moonshot/OpenRouter “verify your API key” warnings for providers you did not edit.
• No-op save fallback — When a dirty save has no wire diff (e.g. toggling a field to its current value), the dashboard no longer falls back to posting the full config (which re-triggered cloud guards).

Docs

• Config guard scoping documented in config-guards-audit.md and package DOX rails.

Upgrade

From v0.5.0.1:

git checkout v0.5.1.0   # after tag ships
uv sync
./netllm restart      # or menubar Restart Agent / systemctl --user restart netllm

Hard-refresh the dashboard (Cmd+Shift+R) if the browser still serves cached dashboard.js.

Verify:

curl -sf http://127.0.0.1:11400/health
./netllm doctor
open http://127.0.0.1:11400/ui/   # Network save without cloud warnings

Test plan (release gate)

• ./scripts/verify-before-pr.sh
• ./scripts/ci.sh e2e — includes test_saving_never_blanks_a_stored_secret swarm-only POST assertion

Previous release

• v0.5.0.1 notes — Phase B closeout (netllm connect, F-38/F-57 fixes)

v0.5.0.1 — Phase B closeout

0.5.0.1: Phase B closeout (post-0.5.0.0)

Patch release closing Phase B items left open at the v0.5.0.0 tag: editor connect CLI, error-envelope completion, contract hygiene, dashboard scenario counters, and Settings live agent status. Includes everything in 0.5.0.0.

Golden-vector contract suite grows from 366 → 373 (+7 Phase B additions: F-38 JSON parse, F-57 guard, B5 pool isolation, plus F-56 lint unit tests).

New

• netllm connect <id> — prints per-harness wiring (env exports, Codex TOML block) for Cursor, Claude Code, Codex, Gemini CLI, Honcho, and others. Flags: --json, --print-env, optional --toggle to register the source. See editor-integration.md.
• Serving tab scenario counters — dashboard Serving tab shows per-source and per-scenario request counts from GET /netllm/v1/telemetry (menubar Serving Stats shows source counts; scenario keys are dashboard-only for now).
• Settings live agent status — macOS Settings statusLabel reads live server.state (same source as Start/Stop), fixing split display after adopting a healthy listener on launch.

Fixed

• Malformed JSON → shaped 400 (F-38) — invalid JSON bodies on /v1/* proxy routes return a proper OpenAI or Anthropic error envelope at 400, not a raw FastAPI 500. Upstream 401/429 from backends still map to 502 (routing failure, not forwarded auth/rate-limit) — see 03-request-lifecycle.md.
• Capability guard quotes caller model (F-57) — embedding/chat capability 400s quote requested_model, not the post-rewrite internal name.
• Contract hygiene (F-56) — divergence lint tracks stable vector IDs so renames cannot bypass the golden-vector gate.
• Pool isolation contract (B5) — golden vector naming-model-pools-isolation-multi-host pins request-aware pool candidacy across multi-host meshes.

Docs

• F-28 resolved (B7) — packaging limits (arm64-only menubar, ad-hoc DMG, subnet scan IDS) documented in platform-matrix.md, macos-install.md, and macos-troubleshooting.md.
• Manual smoke checklists: menubar-adopt-smoke.md, codex-responses-smoke.md.

Caveats (unchanged from 0.5.0.0)

• Codex /v1/responses streaming — offline contract tests pass; live Codex CLI verification is maintainer-only (F-39 partial). See codex-responses-smoke.md.
• macOS arm64 only for packaged menubar; use CLI on Intel Macs or Linux/Windows packages.
• Subnet scan can trigger IDS on corporate LANs — disable subnet_scan or use static swarm.peers on monitored networks.

Upgrade

From v0.5.0.0:

git checkout v0.5.0.1   # after tag ships
uv sync
./netllm restart      # or menubar Restart Agent / systemctl --user restart netllm

Verify:

curl -sf http://127.0.0.1:11400/health
./netllm doctor
./netllm test
./netllm connect cursor --print-env   # optional
open http://127.0.0.1:11400/ui/       # Serving tab scenario counters

Test plan (release gate)

• uv run pytest -q — full suite
• uv run pytest tests/contract -q — 373 golden vectors
• ./scripts/ci.sh lint — includes engine erosion gate
• ./scripts/verify-before-pr.sh --full — macOS menubar lifecycle including L5b

Previous release

• v0.5.0.0 notes — routing engine consolidation + request-aware pools

v0.5.0.0

0.5.0.0: routing engine consolidation + request-aware pools

Minor release: one failover engine for all API surfaces, unified model matching, audit remediation (F-30…F-53), and request-aware model pools for heterogeneous LAN fleets. Includes everything in 0.4.5.1 (menubar status header fix).

Upgrade note: This release changes routing outcomes in documented cases (embeddings guard, Messages upstream errors, batch_shard failover, pool overflow). Read the behaviour sections below before upgrading a live mesh.

Structural (invisible in normal use)

• One routing engine — five duplicated proxy loops replaced by service/engine.py + four surface adapters; AttemptRecorder is the sole accounting path (F-24).
• One model matcher — ModelResolver answers candidacy and invocation with the same algorithm (F-25).
• Monolith split — service.py → 16 modules; CLI main.py → 80 lines + command subpackage (F-26).
• Contract suite — 356 golden-vector tests gate routing behaviour; CI erosion check on engine.py.
• Audit remediation (PR #41) — Anthropic streaming fix (F-30), streaming pre-flight errors (F-32), streaming telemetry parity (F-33), OpenAI/Anthropic error envelopes on /v1/*, SDK payload hardening, read gates on /metrics and /netllm/v1/client-env.

Behaviour you may notice

Embeddings guard (D4)

/v1/embeddings now returns 400 immediately for models classified as chat (name-heuristic). Unknown model names default to chat. Use [routing.model_aliases] with an embedding token in the alias if your encoder name is non-standard.

Messages upstream errors (D11)

/v1/messages forwards upstream 400/404 instead of flattening everything to 502 when backends are OpenAI-translated.

Streamed Messages fixes (D9)

• message_start carries the requested model name (not the backend-resolved name).
• Mid-stream errors terminate with message_stop.

Responses telemetry (D16)

Streamed /v1/responses (Codex CLI) now records success accounting — request counts, tokens, and per-backend routed_requests match chat.

batch_shard failover (D17)

When a shard-assigned backend fails, failover restarts at the candidate list head (more attempts, better recovery for shards on non-first backends).

Shard context on all surfaces (D5)

Shard headers apply to embeddings and Messages, not only chat.

Retry budget (D7)

Attempt cap includes injected cloud rows and Messages fallback tiers (bounded, not unbounded).

Request-aware model pools (D19)

Heterogeneous pools now match the requested model literally first; pool substitution to another allowlist model runs only when no pool member serves that name. Agent-hop requests skip pool substitution (exact_model_only). Prevents a local catch-all host from stealing traffic when another peer serves the requested model.

Provider payload adaptation

SDK-unknown params (top_k, etc.) are adapted at the OpenAI SDK layer for chat and embeddings; agent-hop terminating peers strip unknown fields consistently.

Telemetry / dashboard

• docs/telemetry-api.md is normative; dashboard reads server total_tokens directly.
• tests/contract/test_telemetry_contract.py gates documented keys.

Known issues (unchanged or filed)

• F-38 partial — error envelopes fixed for HTTPException paths; malformed JSON still returns raw 500; upstream 401/429 on OpenAI routes may still map to 502.
• F-57 — capability-guard 400 may echo post-rewrite model names from model_rewrites.
• F-54 — dashboard.js remains a single file (no module split in this release).

Full operator detail: architecture/refactor/RELEASE-NOTES.md.

macOS (0.4.5.1 fix included)

Menubar status header reads live server.state — no more Agent stopped with Stop Agent visible after adopt.

Upgrade

macOS (notarized DMG when available):

git clone https://github.com/matthewdcage/llm-swarm-router.git
cd llm-swarm-router && git checkout v0.5.0.0
uv sync && uv pip install venvstacks
apps/netllm-mac/Scripts/build.sh release
packaging/scripts/macos-app-install.sh --source apps/netllm-mac/build/Stage/llm-swarm-router.app

Linux / Windows / CLI-only:

git checkout v0.5.0.0
uv sync
./netllm restart   # or systemctl --user restart netllm

Verify:

./netllm doctor
./netllm test
curl -sf http://127.0.0.1:11400/health
scripts/live-routing-smoke.sh   # maintainer LAN mesh (optional)

Mixed model pools: confirm [routing.model_pools.*] hosts lists every participating agent; run ./netllm status on each node after upgrade.

Test plan (release gate)

• uv run pytest -q — full suite
• uv run pytest tests/contract -q — 356 golden vectors
• ./scripts/ci.sh lint — includes engine erosion gate
• ./scripts/verify-before-pr.sh — macOS menubar when applicable

v0.4.5.1 — macOS menubar status fix

0.4.5.1: macOS menubar agent status fix

Patch release fixing a menubar UI bug where the status header could show Agent stopped while Stop Agent was visible after the app adopted a healthy agent already listening on :11400.

Fixed

• Menubar status header — MenubarAppModel.statusTitle now reads live server.state (same source as Start/Stop), and the state observer updates synchronously on the main queue. Resolves the split display after adoptHealthyListener on launch or reinstall.

Upgrade

macOS (notarized DMG from GitHub Releases):

open ~/Downloads/llm-swarm-router.dmg
# drag llm-swarm-router to Applications, or:
packaging/scripts/macos-app-install.sh --dmg ~/Downloads/llm-swarm-router.dmg

Source build (macOS 26+ when DMG unavailable):

git clone https://github.com/matthewdcage/llm-swarm-router.git
cd llm-swarm-router && git checkout v0.4.5.1
uv sync && uv pip install venvstacks
apps/netllm-mac/Scripts/build.sh release
packaging/scripts/macos-app-install.sh --source apps/netllm-mac/build/Stage/llm-swarm-router.app

Verify:

curl -sf http://127.0.0.1:11400/health
open http://127.0.0.1:11400/ui/

Menubar should show Agent running · :11400 (green) with Stop Agent when the agent is healthy.

v0.4.1.0 — embeddings endpoint, capability-aware routing, smarter retries

0.4.1.0: embeddings endpoint, capability-aware routing, smarter retries

Feature release driven by a live two-machine swarm log audit: the router now serves POST /v1/embeddings, refuses chat requests against encoder/TTS models, stops wasting retries on backends that already failed, fixes an in-flight counter leak, and cleans up multi-homed hosts in peer scans.

New

• POST /v1/embeddings — OpenAI-compatible embeddings on the same base URL (http://<host>:11400/v1). Routes to whichever backend serves the embedding model (Ollama, oMLX, LM Studio, vLLM, or a LAN peer agent via agent-hop) with the same failover and spillover as chat. Anthropic-format backends are excluded (the Anthropic Messages API has no embeddings standard); Anthropic-wired clients use the OpenAI surface for embeddings.
• Model capability classification — GET /v1/models entries carry a capability field (chat, embedding, audio, rerank, other). Chat completions and Messages requests against embedding/TTS/encoder models return a clear 400 with a /v1/embeddings hint instead of burning the retry budget on upstream tokenizer.chat_template errors.
• Case-insensitive model resolution — clients sending differently-cased names (e.g. gemma-…-4bit vs the served …-4Bit) now route, with the served ID's exact casing forwarded upstream and the requested name restored in the response. Alias keys in [routing.model_aliases] also match case-insensitively.
• LMSTUDIO_API_KEY — LM Studio instances with API authentication enabled are now supported via env var or [[routing.backends]] api_key / api_key_env; the dashboard doctor flags auth-gated backends (401/403) with the fix.

Fixed

• Retry budget reaches healthy peers — failover retries now skip backends that already failed the current request instead of re-hitting the same broken local backend, so spillover reaches an untried LAN peer before returning 502.
• In-flight counter leak — merge_backends updates local backend rows in place rather than replacing them, so requests completing across a scan refresh decrement the live row. Previously in_flight could ratchet upward forever under load, distorting local_spillover decisions.
• Duplicate peers in subnet scan — multi-homed hosts (Wi-Fi + Ethernet, DHCP drift) answering on several IPs with one agent_id now collapse to a single row preferring their reported listen URL, with other IPs listed under also_reachable_at. The scan also flags this machine's own row (self), which the dashboard labels "this machine".

Upgrade

macOS (menubar, recommended on macOS 26+):

git clone https://github.com/matthewdcage/llm-swarm-router.git
cd llm-swarm-router && git checkout v0.4.1.0
uv sync && uv pip install venvstacks
apps/netllm-mac/Scripts/build.sh release
packaging/scripts/macos-app-install.sh --source apps/netllm-mac/build/Stage/llm-swarm-router.app

CLI only: uv sync && ./netllm init && ./netllm serve — dashboard at http://127.0.0.1:11400/ui/

Verify embeddings:

curl -s http://127.0.0.1:11400/v1/embeddings \
  -H "Authorization: Bearer netllm-local" \
  -H "Content-Type: application/json" \
  -d '{"model":"<embedding-model-from-/v1/models>","input":"hello swarm"}'

Full guides: macos-install.md · editor-integration.md

v0.4.0.1: Open trusted-LAN swarm

0.4.0.1: Open trusted-LAN swarm (DMG promise fix)

Patch release: multi-Mac mesh works without cluster tokens or CLI recovery steps on a trusted home LAN.

Default: open LAN swarm

• Menubar / welcome LAN mode applies local_spillover + subnet_scan with no cluster_token minted
• netllm init --swarm upgrades existing configs in place (no --force); open mesh by default
• netllm serve persists mesh defaults on LAN bind via ensure_lan_mesh_defaults
• Peers pair via mDNS / subnet scan; heartbeats stay open when token is empty

Optional secured pairing

• Settings → Require cluster token — generates token on save, Copy join command for other Macs
• CLI: netllm init --swarm --secure, netllm swarm-token --create, netllm swarm-token --rotate
• netllm join URL --token T unchanged for secured swarms

Doctor and messaging

• Missing cluster token on LAN is not a doctor failure (informational note instead)
• netllm swarm-token on open LAN exits 0 with guidance (no silent --rotate requirement)

Verify (two Macs, trusted LAN)

1. Install app on both machines, enable Listen on LAN, restart agent on each
2. Point clients at http://<either-LAN-IP>:11400/v1
3. ./netllm peers / Settings Peers tab should show both agents

Secured LAN: enable Require cluster token on the leader, copy join command, run once on the peer.

Previous release

• v0.4.0 notes

v0.4.0: The swarm promise — two machines, three commands

0.4.0: The swarm promise — two machines, three commands

Install-and-run now means it: netllm init --swarm on one machine, paste the printed netllm join command on every other, and the mesh forms with authentication and automatic same-model load spreading. No hand-edited config, no silent non-meshing.

Guided swarm setup

• netllm init --swarm / --single — one question on a TTY (scripts stay non-interactive); swarm mode binds 0.0.0.0:11400, auto-generates swarm.cluster_token, selects local_spillover, and prints the exact join command
• netllm join URL --token T — validates reachability and the token (401-aware), detects open-swarm/token mismatches, rejects self-joins, writes LAN bind + token + static peer
• netllm swarm-token [--rotate] — show or rotate the pairing token

Load spreading that follows load

• New local_spillover strategy (swarm default): serve locally below routing.spillover_max_local_in_flight concurrent requests (default 2), spill to the least-loaded LAN peer above it, never hop when the peer is just as busy — peer load travels in heartbeats plus an own-hops ledger
• Loop-guarded agent hops: forwards carry x-netllm-local-only, so a peer running any distributing strategy can never bounce a request back into the mesh; peers advertise only models they serve directly (no transitive catalog echo)

Mixed-provider fleets

• [routing.model_aliases] maps one canonical name to per-provider IDs (oMLX vs Ollama vs LM Studio naming); the proxy rewrites the upstream payload per backend and restores the canonical name in responses and streams
• Clear 404s: unknown models fail fast with the live catalog instead of spraying every backend

Discovery that explains itself

• Loopback-bound agents show in netllm peers as found but unreachable with the exact rebind fix
• LAN-bound agents auto-run a one-shot subnet scan when mDNS finds no peers within 10 s
• netllm doctor prints per-platform firewall commands (UDP 5353 + TCP 11400 for firewalld, ufw, netsh, macOS)

Performance

• Provider scans (with their 1-token diagnose probes) are TTL-cached instead of running on every proxied request, with a cache-stampede guard
• Health probes run off the event loop only when a probe could actually fire — fixes cross-agent status-page probe stalls

Verify (two machines)

# Machine A
./netllm init --swarm && ./netllm serve
# Machine B — paste the printed command
./netllm join http://<machine-A-ip>:11400 --token <printed> && ./netllm serve
# Either machine
./netllm peers && ./netllm models --lan
curl -s http://<each-machine>:11400/metrics | rg netllm_requests_total

Both request counters climb under parallel same-model load.

Compatibility

• Existing configs are untouched: local_first, loopback bind, and all v0.3 strategies behave exactly as before (locked by contract tests); v0.3.x peers can heartbeat into v0.4.0 swarms
• Standing acceptance harness: tests/test_e2e_two_agents.py (two real agents over HTTP in CI)

Previous release

• v0.3.0.4 notes: agent-hop swarm routing

v0.3.0.4: Agent-hop swarm routing

0.3.0.4: Agent-hop swarm routing

Fixes multi-machine mesh routing so a gateway (Honcho, editors, or any OpenAI client) reaches peers via peer agent URLs (http://<LAN-IP>:11400/v1) instead of unreachable loopback oMLX rows from another host.

Fixes

• Agent-hop backends: peer_agent_backends() emits one routable backend per LAN peer; gateways no longer merge peer 127.0.0.1:8080 URLs into the pool
• Round-robin distribution: round_robin alternates local inference and peer agent hops when the same model exists on multiple machines
• Self-peer guard: config save, peers --save, and dashboard peers-scan skip this host's own listen_url in swarm.peers
• mDNS LAN URL: wildcard bind (0.0.0.0:11400) advertises LAN IP in status/heartbeat (existing behavior; regression test added)

Verify (two-machine LAN)

# Gateway — Honcho points here; routing.default_strategy = "round_robin"
./netllm serve --host 0.0.0.0
./netllm status    # peer backend: http://<peer-LAN>:11400/v1

./netllm test --model <shared-model>

curl -s http://127.0.0.1:11400/metrics | rg netllm_requests_total
curl -s http://<peer-LAN-IP>:11400/metrics | rg netllm_requests_total

Both metrics should increase when traffic round-robins across machines.

Docs

• honcho-integration.md: agent-hop diagram and metrics check
• netllm-swarm skill: gateway + round_robin guidance

Previous release

• v0.3.0.3 notes: macOS swarm Settings QoL + web dashboard admin

v0.3.0.3: macOS swarm Settings QoL + dashboard admin

0.3.0.3: macOS swarm Settings QoL + web dashboard admin

Patch release improving LAN swarm discoverability in the menubar Settings app and fixing the web dashboard when opened via a LAN IP on the same Mac.

Fixes

• Settings live status: polls /health and /netllm/v1/status every 2s while Settings is open; Restart Agent waits for /health before refreshing (no more stale backends — until app quit)
• LAN peer discovery QoL: welcome Listen on LAN enables swarm.subnet_scan; Settings auto-runs subnet peer scan when the agent is healthy
• Web dashboard admin: http://<LAN-IP>:11400/ui/ on the same host works like 127.0.0.1 (local_admin_client_hosts); doctor/config failures no longer mark the whole dashboard unreachable
• macOS install docs: source build + install script primary until notarized GitHub DMGs ship (macOS 26+ Gatekeeper)

Notes

• macOS 26+: prefer build from source + install script (below), not the GitHub DMG, until notarized releases are published
• Remote machines on the LAN: dashboard status/models are read-only unless swarm.cluster_token is set

Install / upgrade (recommended on macOS 26+)

git clone https://github.com/matthewdcage/llm-swarm-router.git
cd llm-swarm-router && git checkout v0.3.0.3
uv sync && uv pip install venvstacks
apps/netllm-mac/Scripts/build.sh release
packaging/scripts/macos-app-install.sh --source apps/netllm-mac/build/Stage/llm-swarm-router.app

CLI only (no menubar): uv sync && ./netllm init && ./netllm serve — http://127.0.0.1:11400/ui/

Full guide: macos-install.md · Swarm: macos-troubleshooting.md#swarm--lan-peers

Previous release

• v0.3.0.2 notes: menubar agent start on LAN listen