Ghostlink v1.17.0

[1.17.0] - 2026-08-08 (Real distributed-inference testing, three bug fixes, RPC allowlist, install script, JS SDK)

Added

• Real E2E CI gate for distributed inference (.github/workflows/distributed-e2e.yml, Dockerfile.rpc-fabric, docker-compose.rpc-fabric.yml, scripts/rpc_fabric_assert.py): a two-container Docker fabric proving Ghostlink's ggml-rpc-backed distributed inference actually executes across containers (real_inference: true, live RPC connection log evidence), not just that peer discovery found a node count.
• Real multi-node benchmark harness (docker-compose.rpc-fabric-benchmark.yml, scripts/rpc_fabric_benchmark.py), plus extensive real findings from testing on genuinely separate physical hardware documented in docs/BENCHMARKS.md: real single-node-vs-distributed throughput comparisons, and — the actual proof this project's roadmap has been chasing — a real 30B-class model that cannot load on one machine alone (ErrorOutOfDeviceMemory) loading and serving correctly once split across two real machines.
• RPC contributor IP allowlist (rpc_allowed_peers setting, crates/ghost-link/src/rpc_cluster.rs): ggml-rpc-server has no authentication of its own (an upstream llama.cpp limitation); Ghostlink now optionally fronts it with a Ghostlink-controlled TCP proxy that only forwards connections from allowlisted IPs/CIDR ranges. Empty allowlist (the default) is byte-for-byte the old direct-bind behavior — zero overhead, zero change, for anyone not using the feature.
• Version-mismatch detection for RPC peers (rpc_build_id field on NodeResources, carried through all three discovery wire paths — the shared binary encoder, DiscoveryFrame's UDP encoder, and mDNS TXT records): a coordinator now refuses to route distributed inference through a peer running a different llama.cpp build, closing a real bug found this session where mismatched builds silently corrupted output on larger models while the API reported healthy throughout. Only excludes on a confirmed mismatch — a peer that predates this field is still used, so this rolls out without breaking anyone mid-upgrade.
• One-line install script (scripts/install.sh, scripts/install.ps1): curl -fsSL .../install.sh | sh downloads, SHA256-verifies, and installs the real published ghost-link release binary — no sudo, no package manager, no Rust toolchain required.
• JS/TS client SDK (sdks/js/, package ghostlink-client): mirrors sdks/python's shape (chat.completions.create, real SSE streaming via /api/inference/chat, typed error hierarchy), built on native fetch/ReadableStream, ships ESM + CJS + .d.ts.
• Full per-crate READMEs for all five workspace crates (ghost-link, ghostlink-core, mcp-calculator, mcp-rag, mcp-vision) — each Cargo.toml's readme field now points at its own crate's README instead of the repo-wide root README.

Fixed

• Silent output corruption from version-mismatched ggml-rpc peers — see "Added" above; this is the fix, rpc_build_id detection is the mechanism.
• Unsupervised RPC contributor child process: rpc_cluster::ensure_contributing() already had working respawn logic but was only ever called once at server startup — if the spawned ggml-rpc-server child later crashed (e.g. the quantized-KV-cache/RPC-CPU-backend crash found this session), the node kept advertising RPC capability via discovery while actually unreachable. Now called every 30s on a background thread for the process lifetime whenever contribute_compute is on.
• 90-second model-ready timeout too short for real distributed loads: native_engine.rs used a flat 90s health-check budget for both single-node and distributed loads. Real distributed loads measured this session took anywhere from 168s to over 900s depending on model size, all previously aborted as false failures. Now scales to 600s specifically when a load attempt's args include --rpc, stays at 90s for single-node; GHOSTLINK_MODEL_READY_TIMEOUT_SECS env override for further tuning.

Changed

• ghost-link and ghostlink-core bumped 1.16.1 → 1.17.0 (new backward-compatible settings/protocol fields, no breaking changes — a minor bump per semver). ghostlink_gui_modern's package.json bumped to match, keeping the whole repo on one coordinated version number.

Documentation

• docs/ROADMAP.md and docs/BENCHMARKS.md updated extensively with the real findings above — hardware tables, methodology, honest caveats about what wasn't yet proven (e.g. "usable speed" for the 30B distributed result is not yet there, even though the capacity proof is real).

Validation

• cargo fmt --all --check, cargo clippy --workspace --all-targets -- -D warnings, cargo test --workspace — all clean (164 ghost-link + 183 ghostlink-core tests, 0 failures).
• Real Docker E2E fabric rebuilt and rerun after the allowlist change, confirming zero regression to the existing passing gate.
• JS SDK: tsc --noEmit clean, real tsup build (ESM + CJS + .d.ts), 17/17 vitest tests passing.
• Install scripts: both actually run end-to-end against the real live v1.16.1 release (not just syntax-checked) — real binary downloaded, checksum verified against the published SHA256SUMS, installed binary executed successfully.

Ghostlink v1.16.1

[1.16.1] - 2026-08-05 (CI fix: release-artifacts.yml release build)

Fixed

• release-artifacts.yml's "Run release validation gates" step runs cd control-plane && go test ./... but never installed a Go toolchain first (unlike ci.yml's Go job, which does). This only surfaced when the v1.16.0 tag push exercised the workflow for real for the first time — it only triggers on push: tags: v*, so no PR check had ever run it. Both matrix legs failed: macos-latest with go: command not found (no Go on that runner image at all), windows-latest with a transient TLS handshake timeout fetching a Go module (plausible without a real setup-go step warming the module cache). Fixed by adding actions/setup-go@v7 with go-version: stable, matching the already-working pattern in ci.yml.
• No functional code changes — this release exists solely to get v1.16.0's actual content (see below) published with working release binaries. v1.16.0 itself published successfully to crates.io; it just never got a GitHub Release with binaries attached.

This patch is cut from the commit immediately after the CI fix landed on main, before later unrelated work (an LLM-shaped benchmarking suite) merged — it carries v1.16.0's code unchanged plus only this workflow fix, not that follow-on feature work.

---

Ghostlink v1.16.0

[1.16.0] - 2026-08-05 (Reliability fixes: GPU probe timeout, TCP circuit breaker, model-list caching)

Fixed

• GPU hardware detection (system_profile.rs) had a probe-timeout regression: each probe_*_with_timeout wrapper unconditionally slept the full timeout duration before checking whether the probe had already finished, so every startup paid the full 5-10s per probe instead of returning as soon as the fast path completed. Replaced with a real bounded wait (detached thread + mpsc::recv_timeout) that returns immediately on completion and only blocks up to the timeout on a genuinely slow/hung probe. Full profile detection now completes in ~1.5s on a typical dev machine instead of a guaranteed multi-second floor.
• GUI production build (npm run build) was broken: vite-plugin-monaco-editor-esm's built-in worker entries hardcode monaco-editor/esm/vs/... paths that, against monaco-editor's current package.json "exports" map, resolve to a doubled esm/vs/esm/vs/... path that doesn't exist ("Could not resolve"). Reconfigured ghostlink_gui_modern/vite.config.ts to supply the same 5 workers (editor core, CSS, HTML, JSON, TypeScript) via customWorkers with the prefix stripped, which the exports map re-adds correctly. Affects both npm run build and npm run dev; this was blocking the release-artifacts.yml CI gate outright.

Added

• circuit_breaker module in crates/ghostlink-core/src/circuit_breaker.rs: a 3-state (Closed/Open/Half-Open) circuit breaker with jittered exponential backoff. Wired into the TCP transport bridge's reconnect loop (crates/ghostlink-core/src/runtime.rs spawn_tcp_bridge) via a new per-node breaker registry on ClusterState (circuit_breaker_for) — failure history now persists across pipeline executions targeting the same remote node, so a chronically-unreachable node fails fast on later calls instead of repeating the full connect/backoff sequence every time. Opt-in per call site (Option<CircuitBreaker>); the loopback benchmarking path passes None and is unaffected.
• api_response_cache module in crates/ghostlink-core/src/api_response_cache.rs: a TTL + ETag response cache. Wired into GET /api/models, which previously ran a real fs::read_dir/fs::metadata disk scan on every request — now cached for 5s and explicitly invalidated the moment a download completes or a model is deleted.
• LayerKvCache::write_kv_batch in crates/ghostlink-core/src/kv_cache.rs: writes multiple tokens' KV entries under a single write-lock acquisition, validating every entry upfront so a bad entry fails the whole batch atomically. Available as a primitive; like the rest of kv_cache.rs, it has no current caller in runtime.rs (Ghostlink delegates model execution to an external inference engine).

Removed

• Three modules from an in-progress performance pass didn't hold up under review and were cut before landing: a churn-coalescing module that duplicated ClusterState's existing lock-free snapshot cache, an MCP "server pool" that pooled against a per-call subprocess-spawn cost the real MCP client (mcp/registry.rs, persistent connections) doesn't have, and a protocol buffer pool targeting DiscoveryFrame::encode(), which already encodes into a stack buffer on a low-frequency discovery path.

Validation

• cargo fmt --all --check — OK
• cargo clippy --workspace --all-targets -- -D warnings — OK
• cargo test --workspace — OK
• cd control-plane && go test ./... — OK
• cd ghostlink_gui_modern && npm run test — OK (14 files, 142 tests)
• cd ghostlink_gui_modern && npm run build — OK

---

Ghostlink v1.15.1

[1.15.1] - 2026-07-28 (Release workflow fix)

• release-artifacts.yml's "Build release bundle" step was missing an
  explicit shell: bash. On ubuntu-latest/macos-latest that default
  shell already is bash, so it went unnoticed there — but windows-latest
  defaults to PowerShell, which fails immediately on the step's bash [[ ]] syntax. v1.15.0's release shipped with Linux and macOS binaries only;
  this release adds the missing Windows binary under a clean version tag
  rather than rewriting v1.15.0's already-published release.

---

Ghostlink v1.15.0

[1.15.0] - 2026-07-28 (Real distributed inference via llama.cpp RPC backend)

Closes the gap between what Ghostlink's clustering claimed to do and what
/v1/chat/completions actually executed: peer discovery and a distributed
planning/benchmark engine existed, but no request path ever ran a model
split across more than one machine. Verified before writing any integration
code that the existing ghost-link flow/stage-worker pipeline moves
synthetic benchmark payloads, not real model layers — so this uses
llama.cpp's own RPC backend (ggml-rpc) instead, which does real
cross-process tensor execution.

✨ Features

• Real distributed inference (ghost-link::rpc_cluster): a node opts in
  to contributing compute (contribute_compute + rpc_port in settings)
  and runs ggml-rpc-server, exposing its GPU/CPU over TCP. A node serving
  a request (distributed_inference: true) discovers healthy
  RPC-contributing peers from live cluster state, computes a
  VRAM-proportional --tensor-split, and launches its local llama-server
  with --rpc/-ts — zero manual flags from the operator. Off by default;
  single-node deployments see no behavior change. Verified live: a model
  forced entirely onto a second process's device via -ts 0,1 produced
  real generated text, and two full ghost-link serve processes with real
  UDP discovery between them auto-negotiated the RPC args end to end.
• NodeResources.rpc_port: UDP discovery frames and mDNS TXT records
  now carry each node's RPC-contribution port, so peers can be selected for
  distributed inference without any manual configuration.

🐛 Fixed

• Every ghost-link serve instance previously hardcoded its cluster node
  id to the literal string "studio-api", regardless of machine. Two
  real Ghostlink installs on two real machines would collide in
  ClusterState's id-keyed map — meaning no distributed feature (old or
  new, UDP or mDNS) ever worked across genuinely separate hardware,
  independent of this release. Now derived from the hostname
  (GHOSTLINK_NODE_ID env var to override).
• DiscoveryFrame::encode() — the function UDP discovery actually
  calls — is a separate, hand-duplicated serializer from
  NodeResources::encode_payload_into (kept for a zero-copy calling
  convention), discovered mid-implementation to silently drop the new
  rpc_port field entirely. mDNS discovery (which reuses the shared
  encoder) carried it correctly the whole time; UDP discovery didn't, and
  because UDP is tried first and wins ties in /api/workers/discover's
  merge, its None silently shadowed mDNS's correct value.

📚 Documentation

• docs/ROADMAP.md documents the full investigation, what was originally
  planned versus what actually shipped and why, and the verification
  performed at each step.

---

Ghostlink v1.13.0

[1.13.0] - 2026-07-26 (Tool-call context overflow fix)

Found in the wild: a fetch tool call that pulled an entire webpage (site
nav, a trivia quiz, promoted-songs list, footer — none of it relevant)
got folded straight into the prompt with no size limit, pushing a single
chat turn over the model's context window and failing outright with
llama_server request failed with status 400 Bad Request: exceed_context_size_error.

🐛 Fixed

• Tool observations are now capped at 4000 characters before being
  folded back into the prompt (mcp::toolcall::format_observation), with
  a [truncated, N more characters omitted] marker so the model (and
  anyone reading the transcript) knows content is missing rather than
  silently seeing a shortened result as complete. This bounds the damage
  any single tool call can do to the context budget, independent of how
  --ctx-size is configured.

✨ Changed

• Default context size (-c) doubled across every VRAM tier in
  native_engine::get_ctx_size — 8192→16384→32768 for 8/12/16GB+ (was
  4096→8192→16384), floor raised 2048→4096 for <8GB, and the
  no-VRAM-info fallback raised 4096→8192. The previous defaults were
  tight enough that ordinary tool-calling chat (system prompt + a few
  turns + one tool observation) could approach the ceiling even without
  the truncation bug above. GHOSTLINK_CTX_SIZE still overrides directly
  if you want a different value.
• RuntimeSettings::DEFAULT_CTX_SIZE (the GUI's own conversation-budget
  default, separate from the value above) raised 4096→8192 to match, so
  the two don't drift out of sync.

---

Ghostlink v1.12.0

[1.12.0] - 2026-07-26 (Real HTTP/SSE MCP transport)

Closes the other stub found while auditing the codebase for leftover
placeholders: McpTransport::Http was a real, user-configurable entry in
mcp_servers.toml's schema, but connecting to one always failed with
"HTTP/SSE transport is not implemented yet" — the config accepted it,
the runtime never delivered it.

✨ Features

• Real streamable HTTP/SSE MCP transport, built on rmcp's own
  StreamableHttpClientTransport (the same SDK already used for the stdio
  transport) — connecting to a remote MCP server over a URL now actually
  works, instead of erroring at connect time regardless of config.
• ${VAR_NAME} header resolution, matching the existing stdio env
  behavior: header values written as "${VAR_NAME}" are resolved from the
  host process environment at connect time, never stored as literal
  secrets in mcp_servers.toml.

🐛 Fixed

• Literal-secret validation now covers HTTP headers, not just stdio env
  vars. McpConfigManager::save rejected a literal-looking secret in a
  stdio server's env map, but the same check never ran against an HTTP
  server's headers map — meaning the one MCP transport where a real
  bearer token or API key is the normal case for a header value had no
  guard against saving it in plaintext. Both transports now go through the
  same rejection.

---

---

Note on this release: built locally from the tagged commit (unsigned —
no GPG key available outside CI; no SBOM — syft not available locally).
Going forward, releases published via the fixed release-artifacts.yml
workflow will include both when CI signing secrets are configured.

Ghostlink v1.11.0

[1.11.0] - 2026-07-26 (Real bearer-token auth + PQC-hybrid TLS)

Closes the last item from a gap analysis against LM Studio/vLLM: the API
server had no authentication anywhere, and the existing /api/security/*
endpoints were fully mocked — handle_gui_jwt_refresh always returned a
hardcoded "new-token-123", and the PQC endpoints always reported
enabled: true regardless of anything. Both are now real.

✨ Features

• Real bearer-token auth on every route but /health. A 256-bit API
  key is generated once on first run, persisted to api_key.txt, and
  printed to the console — the only way to learn it, since it's never
  returned by any API response. Send it directly as
  Authorization: Bearer <key>, or exchange it for a short-lived JWT via
  POST /api/security/jwt/refresh (jsonwebtoken, HS256, signed with the
  same key — genuine issuance/verification, not the old stub).
• Real HTTPS with a genuine PQC-hybrid (X25519MLKEM768) key exchange
  preference, via rustls's prefer-post-quantum feature (aws-lc-rs
  backend) — the same mechanism Chrome/Cloudflare/AWS use today, not a
  bespoke handshake. Opt-in via a new parallel_slots-style
  enable_tls setting, off by default for today's plain-localhost dev
  flow, forced on when the server binds a non-loopback address (the
  LAN/remote scenario this actually protects). A self-signed cert is
  generated once via rcgen and reused across restarts.
• /api/security/pqc/state and pqc/enable are now real: state
  reports whether this running process's listener is actually serving
  HTTPS (not the persisted setting, which only applies on next restart —
  tracked separately so the two can't be conflated); enable writes the
  setting and honestly says a restart is required rather than pretending
  it's already live.
• Go control-plane gateway now verifies the same shared secret before
  proxying — real JWT signature verification (golang-jwt/jwt/v5), not a
  shape-only check, so it doesn't reject legitimate short-lived tokens the
  GUI uses. Degrades gracefully (no extra edge rejection, not a lockout)
  if the key file isn't readable — the proxy already forwarded
  Authorization through to ghost-link's own auth either way.
• GUI now sends the token on every request — an axios interceptor plus
  the two hand-rolled fetch calls that bypassed it, reading a key the
  user pastes into a new "API Key" field on the Security tab (persisted to
  localStorage). The PQC panel's copy was also corrected — it previously
  claimed "Kyber-768/Dilithium... across all distributed nodes" and
  "AES-GCM 256-bit encryption" when disabled, neither of which was ever
  real; it now accurately describes the actual TLS/PQC-hybrid mechanism
  and states plainly that disabled means unencrypted plain HTTP.

🐛 Fixed

• The API key would only ever have been generated (and its one-time
  console banner printed) lazily on first authenticated request — a
  fresh install with zero traffic yet would have had no way to discover
  it at all. Now generated eagerly at server startup.

✅ Validation

• Live, end-to-end manual verification (not just unit tests): started
  a real server, confirmed 401 with no token, 200 with the raw key, a real
  JWT round-trip (issue → use → success), and real enabled:false →
  enable → restart → enabled:true PQC state transitions. Independently
  proved the PQC claim using openssl s_client -tls1_3 -groups X25519MLKEM768 against the running HTTPS listener — output confirmed
  Negotiated TLS1.3 group: X25519MLKEM768, and a normal client with no
  forced group still connected fine (not a hard requirement, a
  preference).
• New Rust tests: auth.rs (key generation/persistence, bearer
  verification, tampered/garbage rejection), tls.rs (loopback
  detection, idempotent cert generation with real file I/O).
• New Go tests: pkg/auth (key loading, bearer/JWT verification including
  a genuinely expired and a genuinely tampered token, full middleware
  integration via httptest).
• New frontend tests: SecurityTab.test.tsx (API key persistence, and
  that enabling PQC shows a real "restart required" message rather than
  falsely claiming it's already active).
• cargo fmt --all --check, cargo clippy --workspace --all-targets -- -D warnings, cargo test --workspace all green. go build, go vet,
  go test ./... all green. tsc --noEmit, vitest run (118 passed)
  clean.

---

Note on this release: built locally from the tagged commit (unsigned —
no GPG key available outside CI; no SBOM — syft not available locally).
Going forward, releases published via the fixed release-artifacts.yml
workflow will include both when CI signing secrets are configured.

Ghostlink v1.0.0

Ghostlink v1.0.0

Highlights

• Release binary bundle for the Ghostlink CLI
• Checksummed archive for download verification
• Updated README and docs launch experience

Assets

• ghostlink-release.tar.gz
• ghost-link
• SHA256SUMS