Skip to content

Releases

v0.1.0

Choose a tag to compare

@sigilmakes sigilmakes released this 04 Aug 13:51

conduit v0.1.0 — inference control plane for local llama.cpp backends (D1–D22).

A Rust daemon + thin CLI that fronts multiple llama.cpp backends (Nixpkgs + ROCmFPX), arbitrates a single-tenant GPU lease (one model across all backends), and exposes an OpenAI-compatible inference endpoint + a control API. Ships a discovery-only pi extension.

  • conduit-core: config schema (path-agnostic dir + per-backend override, registry, auth), the one-model lease state machine (incl. a Benching state for conduit bench), per-model meta + thinking injection logic.
  • conduit daemon + CLI: /api control, /v1 OpenAI proxy (lazy load on first prompt, per-model thinking-budget injection, SSE stream-through), auth (local trusted / remote API key), cancellation-safe lease driver, conduit bench (D22).
  • extension/: discovery-only pi extension (replaces pi-llama-cpp + laguna-reasoning).
  • flake.nix: packages conduit (crane), llama-rocmfpx (ROCmFPX fork, gfx1151 + HIP_FORCE_MMQ + FA_ALL_QUANTS), conduit-extension.

Tests: 77 (52 core, 20 lib incl. bench, 5 end-to-end integration). cargo fmt/clippy -D warnings/tsc --noEmit/nix flake check --no-build all clean. CI green on Mnemosyne (Forgejo) + GitHub.

See docs/ARCHITECTURE.md for the full design log.