v0.76.0

[0.76.0] - 2026-09-10

Added

Serving and inference

• Durable KV prefix cache: agent prefixes survive eviction, restart, and cold nodes by @michaelneale in #1228
• skippy-quantize: compose-mtp — splice an MTP draft into a sharded target GGUF by @i386 in #1439
• Load SafeTensors checkpoints directly by @i386 in #1619
• Complete graph-derived v2 model splitting by @i386 in #1672
• Iteration-level scheduler for concurrent staged serving (#1416) by @i386 in #1420
• Route repeated prompts to workers with verified radix cache reuse by @i386 in #1449
• Reuse warm prefixes across agent sessions by @i386 in #1601
• Stream tool calls as they are generated instead of after the turn ends by @michaelneale in #1371
• Pass reasoning effort through Skippy templates by @i386 in #1211
• Expose model-bound tokenizer capability by @i386 in #1227
• Support staged Qwen3.8 Flash-Next inference by @michaelneale in #1509
• Route Qwen3.8 identities to the qwen35 recurrent family by @michaelneale in #1283
• Make plugin-only Serve mode nodes discoverable/reachable by other mesh peers by @MahdiHedhli in #1192
• Add mode-aware management health endpoint by @i386 in #1351
• OpenAI exchange lifecycle hooks for out-of-process plugins (#1331) by @StevenMih in #1437
• Integrate exact lifecycle and trusted sessions by @ndizazzo in #1258
• Explain native runtime catalog selection and rejection by @Virgile-pct in #1675

Model catalog

• Add Muse Glimmer 30B to the model catalog by @michaelneale in #1240
• Recommend Qwen3.8 27B Q4_K_M by @michaelneale in #1343

Logging and observability

• Add durable lifecycle logging foundation by @ndizazzo in #1174
• Expose audited local log APIs and runtime telemetry by @ndizazzo in #1175
• Add typed logs ledger and maintenance UI by @ndizazzo in #1176
• Make operational logs durable, attributable, and queryable by @ndizazzo in #1440
• Add lifecycle timelines and recovery to the logs console by @ndizazzo in #1441

Configuration (settings that were previously inert are now honored)

• One canonical configuration reference with schema parity enforcement by @ndizazzo in #1474
• Derive static validation from an exhaustive schema-status manifest by @ndizazzo in #1475
• Wire kv_offload, kv_unified, and swa_full model_fit controls by @ndizazzo in #1476
• Wire hardware placement and model loading settings (PR 3/8, #1462) by @ndizazzo in #1477
• Make Skippy throughput and lifecycle config effective by @ndizazzo in #1480
• Wire speculative decoding configuration into Skippy by @ndizazzo in #1483
• Honor sampling, chat, and reasoning defaults in Skippy by @ndizazzo in #1485
• Honor multimodal model settings in Skippy by @ndizazzo in #1490
• Complete host wiring for aliases, plugins, and prompt-shape metrics by @ndizazzo in #1488

Documentation surfaces

• Publish Rust crate API docs with the website by @i386 in #1230
• Add generated CLI explorer to docs by @ndizazzo in #1546

Changed

Request handling and context

• Consolidate model=auto and model=mesh into one automatic-routing directive by @michaelneale in #1309
• Treat max_tokens as a ceiling, not a context reservation by @michaelneale in #1354
• Stop aborting healthy long prefills at five minutes by @michaelneale in #1355
• Return a visible error when streaming prompts exceed context by @michaelneale in #1352
• Compact oversized chat context for provider windows by @i386 in #1280
• Default auto context to the 128k agent-serving ceiling by @i386 in #1529

Caching, scheduling, and routing

• Replace flat prefix caches with unified radix by @i386 in #1429
• Make resident KV admission capacity aware by @i386 in #1452
• Add scheduler lab and cache-aware radix scheduling by @i386 in #1447
• Improve Skippy concurrent serving and cache-aware routing by @i386 in #1478
• Route warm prefixes by measured service cost by @i386 in #1603
• Spread concurrent new sessions with local route reservations by @michaelneale in #1631
• Send remote mesh inference directly into the serving pipeline by @michaelneale in #1463
• Improve MoA routing under fleet churn and load by @michaelneale in #1519
• Reuse recurrent KV across growing chat turns by @i386 in #1253
• Make Skippy negotiate real activation boundaries by @i386 in #1585
• Derive KV policy from loaded model by @i386 in #1588
• Remove runtime-slice architecture allowlist by @i386 in #1602

Performance

• Coalesce native KV page transfers by @michaelneale in #1368
• Keep exact-state recording off inference path by @michaelneale in #1389
• Move stage-zero exact-state copies off the request path by @i386 in #1605
• Cut per-token split decode overhead (~10%) by @danielwinterw in #1575

Console and startup

• Refine logging console UX and live delivery by @ndizazzo in #1339
• Keep embedded startup on one validated config image by @michaelneale in #1520

Dependencies

• Bump OpenTelemetry stack to 0.32 and rpassword to 7.5 by @danielwinterw in #1563
• Refresh runtimes and dependencies by @i386 in #1622

Removed

• Stop persisting KV cache state to disk by @michaelneale in #1399
• Remove stale disk cache defaults by @michaelneale in #1398

Fixed

Inference, KV cache, and MoA

• Fix blank responses from thinking models by @michaelneale in #1365
• Prevent reasoning-only model poisoning by @michaelneale in #1391
• Rescue terminal Buzz replies in MoA by @michaelneale in #1417
• Preserve caller tool schemas by @michaelneale in #1344
• Restore single-model degrade for model=mesh on host ingress by @michaelneale in #1291
• Keep degraded MoA requests on the resolved pipeline model by @michaelneale in #1292
• Return IDs for OpenAI tool calls on local-model-only serving by @michaelneale in #1318
• Serialize native proposal lifecycle by @i386 in #1242
• Skippy Nemotron MTP loading by @ndizazzo in #1245
• Complete remaining Skippy MTP runtime callers by @i386 in #1247
• Make native MTP actually work on single-node serving by @michaelneale in #1366
• Clear stale sidecar rows before single-row MTP proposals by @DaBlitzStein in #1544
• Restore recurrent shared prefixes by @michaelneale in #1342
• Restore resident KV suffix locality by @i386 in #1634
• Restore small-context admission and state handoff by @i386 in #1459
• Enforce resident prefix minimum by @i386 in #1557
• Make KV replay and restore fail closed by @i386 in #1569
• Harden KV cache ownership and runtime bounds by @i386 in #1570
• Coordinate lanes with bounded KV admission by @i386 in #1639
• Guard default KV quant against unloadable models by @i386 in #1532
• Fix unified-KV parallel-lane accounting in context planner by @i386 in #1543
• Certify Muse-Glimmer family for the resident-KV prefix cache by @michaelneale in #1294
• Keep multimodal evaluation on its session lane by @i386 in #1586
• Prompt-scaled cache-work deadline and surfaced stream errors by @i386 in #1590
• Stop a stalled SSE consumer pinning a generation worker by @ndizazzo in #1367
• Address PR 1478 review blockers by @i386 in #1561
• Restore builds and SafeTensors inference by @michaelneale in #1636

Split and distributed serving

• Fix split serving: stages above layer 0 fail to load by @michaelneale in #1290
• Let Iroh own transport path selection by @michaelneale in #1345
• Fix chat-chain pipelined splits: verify-history accounting, early-stop replies, session leaks by @danielwinterw in #1408
• Use raw f32 activation transport by @i386 in #1482
• Address Qwen4 split framing review findings by @i386 in #1516
• Fix speculative decoding over multi-node splits by @danielwinterw in #1577
• Fix split speculative decoding: verify-window double-record and backend-sampling rejection by @danielwinterw in #1581
• Allow split-serving nodes to use matching local GGUF files by @IvGolovach in #1568
• Keep staged llama.cpp mappings alive until every retained tensor is loaded by @IvGolovach in #1542
• Restore recurrent split prefix reuse by @i386 in #1662
• Reclaim split-prefill tokens after failed sessions by @michaelneale in #1523
• Preserve split capability when model size is unknown by @mikemikimike in #1558
• Drain hosted split stages on shutdown by @ndizazzo in #1433
• Close the iroh endpoint on shutdown instead of dropping it by @ndizazzo in #1431
• Keep Mesh peers stable on LAN after relay bootstrap by @michaelneale in #1660
• IPC idle timeout for external-relay plugins by @MahdiHedhli in #1191
• Cancel upstream before response headers by @ndizazzo in #1448
• Contain a panicking connection worker instead of stopping the accept loop by @Virgile-pct in #1536
• Make connection worker reads interruptible by shutdown by @Virgile-pct in #1565

Models, packaging, and downloads

• Require package-v2 serving admission by @i386 in #1693
• Emit and admit payload-free v2 metadata carriers by @i386 in #1706
• Inspect and serve compact v2 packages by @i386 in #1711
• Fix complete package-v2 model downloads by @i386 in #1722
• Resolve Hugging Face GGUF snapshot symlinks by @i386 in #1725
• Ignore unrelated projector sidecars by @i386 in #1628
• Name a local GGUF with --model instead of trying to resolve it by @michaelneale in #1269
• Report accurate parameter sizes for local GGUF models by @i386 in #1742
• Fix/windows skippy package large gguf by @wangwenjunfromlanzhou in #1307
• Avoid AArch64 Xet TLS illegal instructions by @ndizazzo in #1469
• Fix install one-liner 404 on NVIDIA hosts without a CUDA toolkit by @michaelneale in #1267
• Make the optional GitHub star step reliable in setup by @michaelneale in #1278

Platform and hardware

• Normalize native runtime library paths before LoadLibraryExW by @Virgile-pct in #1537
• Apply the CPU-only runtime budget on Windows by @Virgile-pct in #1567
• Sync mtmd struct layouts with the pinned llama.cpp headers by @DaBlitzStein in #1549
• Match mtmd_helper_bitmap_init_from_buf's current signature by @DaBlitzStein in #1550
• Resolve startup GPU selectors and honor CLI --device by @ndizazzo in #1467
• Credit RAM offload to discrete-GPU nodes on all survey platforms by @Virgile-pct in #1583
• Report the real macOS GPU name (not the CPU brand string) by @StevenMih in #1625
• Correct day/month transposition in macOS ps lstart parsing by @ndizazzo in #1744
• Merge bundled and release runtime catalogs during selection by @Virgile-pct in #1623
• Repair iOS keyring build and CLI timeout by @michaelneale in #1641

Console, CLI, and logs

• Retry transient model route misses in console chat by @i386 in #1277
• Harden lifecycle capture and console recovery by @ndizazzo in #1268
• Render LaTeX in active chat responses by @ndizazzo in #1466
• Use advertised capacity for node and mesh VRAM totals by @ndizazzo in #1746
• Make the dashboard immune to stray console output by @ndizazzo in #1382
• Keep --log-format json parseable and block new raw console prints in CI by @ndizazzo in #1376
• Surface serve and client commands in --help output by @wedigcode in #1607
• Close the four rc6 release-validation defects by @ndizazzo in #1405

Security

• Stop leaking plugin endpoint credentials over the management API by @michaelneale in #1364
• Redact plugin provider health details by @ndizazzo in #1432
• Move model lifecycle controls off peer-reachable ingress by @ndizazzo in #1279
• Cap remotely advertised model lists to protect the console from a hostile peer by @michaelneale in #1357
• Reject non-protobuf content types on OTLP/HTTP ingestion by @michaelneale in #1356
• Reject payload-bearing prediction-return opens early by @michaelneale in #1527
• Harden Skippy sideband framing against I/O amplification by @michaelneale in #1521
• Keep operator-enforced guardrails enabled by @michaelneale in #1530
• Remove dead ungated ErrorBoundary that leaks stack traces (#39) by @michaelneale in #1361
• Inject a fresh per-request client nonce at the local ingress by @StevenMih in #1397
• Supply a client-side per-request nonce in the generated provider config by @StevenMih in #1362
• Bound downstream endpoint resolution by @ndizazzo in #1438
• Clarify end-to-end encryption for mesh traffic by @michaelneale in #1470

Internal

CI, build, test, and repository work with no user-facing behavior change (120 changes)

CI and release engineering

• Fix crates.io publishing for new workspace crates, and make dev builds identify themselves by @michaelneale in #1223
• Repair the Fly console image build by @michaelneale in #1241
• Reshape PR and main CI around composable slices, prepping for Depot workers by @ndizazzo in #1244
• Cross-platform product bundle tree hashes by @michaelneale in #1271
• Distinguish skipped bootstrap routes by @ndizazzo in #1272
• Fail fast on topic lane projections by @ndizazzo in #1274
• Flatten paginated lane checks by @ndizazzo in #1276
• Make CUDA release checks hermetic by @ndizazzo in #1270
• Split planning and platform workflow graphs by @ndizazzo in #1282
• CI smoke matrix routing by @ndizazzo in #1284
• Keep main planning exhaustive by @ndizazzo in #1295
• Prepare protected Depot PR runners by @ndizazzo in #1302
• Add bounded Depot PR canary gate by @ndizazzo in #1306
• Isolate native caches on Depot PR runners by @ndizazzo in #1310
• Keep Depot cache namespace inert by @ndizazzo in #1317
• Pin Depot audit to merged policy by @ndizazzo in #1319
• Identify failing Depot canary endpoint by @ndizazzo in #1321
• Classify rejected Depot cache endpoint by @ndizazzo in #1323
• Add protected Depot authority sentinel by @ndizazzo in #1324
• Prove Depot sentinel cache writes by @ndizazzo in #1326
• Allow bounded Depot caching for approved PRs by @ndizazzo in #1333
• Guard approved Depot PR routing by @ndizazzo in #1334
• Repin Depot PR isolation audit by @ndizazzo in #1336
• Fingerprint Depot macOS toolchains by @ndizazzo in #1337
• Enable Depot for eligible same-repository PRs by @ndizazzo in #1545
• Sync source versions during GitHub releases by @ndizazzo in #1330
• CI recognizes Apple provider paths by @i386 in #1363
• Regenerate console-print ratchet drifted 2 lines by #1376 by @ndizazzo in #1379
• Select static-abi in pr-draft when its matrix consumers are planned by @ndizazzo in #1383
• Cancel sibling PR lanes after failure by @ndizazzo in #1388
• Bound compiler and local build caches by @ndizazzo in #1390
• Revert "ci: bound compiler and local build caches (#1390)" by @ndizazzo in #1394
• Prebuilt runner images for smoke/SDK/web jobs by @ndizazzo in #1380
• Bound compiler and local build caches by @ndizazzo in #1395
• Point pnpm at the runner image's baked store instead of the Actions cache by @ndizazzo in #1400
• Fix release builds failing in the containerized smoke job by @michaelneale in #1401
• Install just before the CI-contracts python tests run by @ndizazzo in #1407
• Skip build slices for draft pull requests by @ndizazzo in #1415
• Retry transient HF artifact uploads by @michaelneale in #1424
• Unblock release dispatches after the config schema module split by @michaelneale in #1426
• Compare release bases, not raw tag SHAs, in provenance check by @ndizazzo in #1430
• Validate CUDA runtime toolkit labels by @ndizazzo in #1465
• Keep change planning below expression limit by @ndizazzo in #1468
• Use dotted workspace version form in skippy-scheduler by @michaelneale in #1518
• Regenerate console-print ratchet after #1516 by @i386 in #1525
• Cache SDK smoke and reuse UI artifacts by @ndizazzo in #1556
• Centralize test model artifacts by @i386 in #1663
• Keep Swift SDK CI within its time budget by @i386 in #1664
• Remove obsolete workflow shims and batch planners by @ndizazzo in #1666
• Share release UI assets and use CPU images for composition by @ndizazzo in #1684
• Prepare release UI from container-owned checkouts by @ndizazzo in #1687
• Adopt qualified lean runner images for UI workloads by @ndizazzo in #1690
• Link CUDA package tools on driverless builders by @ndizazzo in #1694
• Fall back to any brew LLVM with Clang LibTooling in llama canary guard by @i386 in #1697
• Nightly agentic-replay coding benchmark on micstudio by @i386 in #1705
• Remove the nightly-competitive-benchmark workflow by @i386 in #1715
• Publish llama canary repair PRs only at terminal status by @i386 in #1723
• Use served package ref for runtime smoke gates by @ndizazzo in #1741

Build, dependencies, and the llama.cpp pin

• Update pinned llama.cpp revision by @i386 in #1216
• Regenerate patch queue for latest upstream by @i386 in #1232
• Pin iroh to 1.0.3 explicitly by @michaelneale in #1257
• Advance llama.cpp and simplify the Skippy patch queue by @i386 in #1402
• Address #1402 review follow-up (dead comparison + Inkling test coverage) by @i386 in #1412
• Link libvendor-hash.a and test Rust on llama.cpp pin bumps by @michaelneale in #1419
• Tiered family battery + agent-repaired canary on the family-certify runner by @i386 in #1436
• Run the llama.cpp canary daily and accept split GGUF metadata by @i386 in #1457
• Fix llama canary family certification failures by @i386 in #1471
• Wire the agent repair loop into the upstream canary by @i386 in #1481
• Heartbeat, arm64 build guard, and push-permission diagnostics for the canary repair loop by @i386 in #1489
• Preflight repair-token push permission before any repair work by @i386 in #1493
• Probe actual token write capability in the repair preflight by @i386 in #1496
• Prune stale /tmp worktrees and fix the preflight probe delete by @i386 in #1499
• Auto-approve agent tool permissions in the repair sandbox by @i386 in #1502
• Rebase patch queue onto upstream b19cbe925b by @i386 in #1504
• Rebase patch queue onto upstream b19cbe925b by @i386 in #1505
• Ensure the repair PR exists before applying the agent's body by @i386 in #1506
• Review certified canary repairs with a fresh agent turn by @i386 in #1507
• Certify the multimodal path with a real mmproj smoke lane by @i386 in #1508
• Certify Qwen3.8 Flash-Next in the llama family battery by @i386 in #1513
• Disable ggml compiler-cache autodetection when we decline sccache by @danielwinterw in #1531
• Rebase patch queue onto upstream cc83d7b482 by @i386 in #1533
• Rebase patch queue onto upstream 662a0b0121 by @i386 in #1564
• Restore deterministic llama patch replay by @i386 in #1587
• Keep llama canary repairs on the certified pin by @i386 in #1627
• Expand family battery with 15 additional architectures by @i386 in #1683
• Rebase patch queue onto upstream b31b71f3a0 by @i386 in #1721

Tests and benchmarks

• Run SWE-Gym through Harbor by @i386 in #1239
• Wire Playwright into CI and fix the 9 pre-existing /logs failures by @ndizazzo in #1377
• Tolerate hosted-runner frame cadence by @ndizazzo in #1386
• Pin scheduler workload fixtures by @i386 in #1450
• Keep PR tests portable across socket timer granularities by @michaelneale in #1555
• Close the direct prediction return ring in the distributed driver by @danielwinterw in #1560
• Add layered KV coverage and performance history by @i386 in #1572
• Prevent false Iroh diagnoses on macOS LAN tests by @michaelneale in #1573
• Allow trusted manual benchmark runs by @i386 in #1574
• Add Agentic Replay for commit performance comparisons by @i386 in #1578
• Route nightlies to the dedicated white runner by @i386 in #1579
• Tolerate rejected direct-return writes by @i386 in #1600
• Certify KV reuse with captured agentic replay by @i386 in #1606
• Run SafeTensors runtime smoke on affected PRs by @i386 in #1626
• Compare agentic replay across inference engines by @i386 in #1655
• Measure CPU runtime seed reuse and explicitly exclude the ineffective seed by @ndizazzo in #1704

Refactors, docs, and repo hygiene

• Split Skippy by functional boundary by @i386 in #1194
• Update mesh-llm-ui console dependencies by @ndizazzo in #1286
• Record unsafe Depot PR authority by @ndizazzo in #1328
• Define Depot PR cache isolation contract by @ndizazzo in #1329
• Record Depot PR rollout evidence by @ndizazzo in #1341
• Complete evidence-backed codebase audit waves by @ndizazzo in #1396
• Split root Justfile into semantic imports by @ndizazzo in #1458
• Assign imported Just files to Rust ownership by @ndizazzo in #1464
• Windows section - native CUDA works on CUDA 13 drivers; document current workarounds by @Virgile-pct in #1535
• Rework 0009 tagged tool-arg ordering to permute(required) by @i386 in #1540
• Apply rustfmt to dynamic_library.rs by @Virgile-pct in #1541
• Silence cfg-gated macOS clippy warnings by @i386 in #1591
• Quarantine name-driven topology adapter by @i386 in #1604
• Clean warning and deprecation noise by @i386 in #1618
• Register skippy-model ownership by @i386 in #1620
• Make pre-commit validation change-scoped by @i386 in #1580
• How to run crate test gates on a bare Windows checkout by @Virgile-pct in #1657
• Silence the Windows-only dead-code lints in mesh-llm-system by @Virgile-pct in #1659

New Contributors

• @MahdiHedhli made their first contribution in #1191
• @wangwenjunfromlanzhou made their first contribution in #1307
• @Virgile-pct made their first contribution in #1535
• @DaBlitzStein made their first contribution in #1549
• @mikemikimike made their first contribution in #1558
• @StevenMih made their first contribution in #1437
• @wedigcode made their first contribution in #1607

Full Changelog: v0.75.1...v0.76.0