Repository navigation
Releases
Releases · sigilmakes/conduit
v0.1.0
conduit v0.1.0 — inference control plane for local llama.cpp backends (D1–D22).
A Rust daemon + thin CLI that fronts multiple llama.cpp backends (Nixpkgs + ROCmFPX), arbitrates a single-tenant GPU lease (one model across all backends), and exposes an OpenAI-compatible inference endpoint + a control API. Ships a discovery-only pi extension.
conduit-core: config schema (path-agnostic dir + per-backend override, registry, auth), the one-model lease state machine (incl. a Benching state forconduit bench), per-model meta + thinking injection logic.conduitdaemon + CLI:/apicontrol,/v1OpenAI proxy (lazy load on first prompt, per-model thinking-budget injection, SSE stream-through), auth (local trusted / remote API key), cancellation-safe lease driver,conduit bench(D22).extension/: discovery-only pi extension (replaces pi-llama-cpp + laguna-reasoning).flake.nix: packagesconduit(crane),llama-rocmfpx(ROCmFPX fork, gfx1151 + HIP_FORCE_MMQ + FA_ALL_QUANTS),conduit-extension.
Tests: 77 (52 core, 20 lib incl. bench, 5 end-to-end integration). cargo fmt/clippy -D warnings/tsc --noEmit/nix flake check --no-build all clean. CI green on Mnemosyne (Forgejo) + GitHub.
See docs/ARCHITECTURE.md for the full design log.