Skip to content

Releases

QuixiGate v0.2.1

Choose a tag to compare

@github-actions github-actions released this 14 Nov 12:56

Highlights in v0.2.1

  • 8.4 ms Cold Start & 23.7 MiB Idle RSS (14 MB physical footprint):
    • Deduplicated all outbound TLS root certificate stores into a single shared OnceLock<rustls::ClientConfig> (ring + webpki-roots) and converted all reqwest::Client pools to lazy per-core OnceLock initialization (0 HTTP pools allocated at boot).
    • Constructed the 349-route axum::Router service tree once instead of N times across per-core workers, right-sized the outer CLI runtime, and moved initial local-runtime port discovery off the startup critical path.
    • Lazy-loaded the embedded models.dev.json.gz catalog snapshot on first read.
  • Reproducible Scale Benchmark Suite (litellm_vs_quixigate_bench):
    • Added a 3-process apples-to-apples scale benchmark harness (crates/quixigate-proxy/examples/litellm_vs_quixigate_bench.rs) verifying 70,690 req/s stateless JSON, 66,605 req/s governed virtual-key + budget JSON, 3.09M tok/s governed SSE streaming, and 974,742 tok/s across 1,000 concurrent GPU streams in 156.9 MiB peak RSS with 0 errors.
  • HTTP/1.1 Keep-Alive Hygiene & Production Installers:
    • Added Connection: close on early 413 Payload Too Large rejections per RFC 9112 §9.6.
    • Native OS installers (install.sh -> /usr/local/bin/quixigate on macOS & Linux; install.ps1 -> %LOCALAPPDATA%\Programs\QuixiGate\bin\quixigate.exe on Windows) with SHA-256 verification.

Install / Upgrade

macOS & Linux:

curl -fsSL https://raw.githubusercontent.com/QuixiAI/QuixiGate/main/install.sh | sh

Windows (PowerShell):

irm https://raw.githubusercontent.com/QuixiAI/QuixiGate/main/install.ps1 | iex

Full Changelog: v0.2.0...v0.2.1

v0.2.0

Choose a tag to compare

@ehartford ehartford released this 13 Nov 17:37

Quick Install

macOS & Linux:

curl -fsSL https://raw.githubusercontent.com/QuixiAI/QuixiGate/main/install.sh | sh

Windows (PowerShell):

irm https://raw.githubusercontent.com/QuixiAI/QuixiGate/main/install.ps1 | iex

Highlights

Actix-Style Share-Nothing Per-Core Engine & Polyglot Zero-DOM Cut-Through

  • 67,751 req/s (100% parity with a bare zero-copy Rust TCP pipe, 1.76 ms p99) and 9.47M tok/s (1.56 GiB/s / 13.4 Gbps) SSE streaming throughput on an 18-core machine.
  • Lock-Free In-Memory Data Plane + Batched SQLite WAL Flusher: ArcSwap<ControlSnapshot> + SHA-256 [u8; 32] key indexing + AtomicI64 NanoUSD reservations + linearizable SAVEPOINT WAL write-behind batching (0 SQLite mutex locks on the inference hot path).
  • 3 Native Zero-DOM Cut-Through Lanes: Borrowed envelope peeks (serde::de::IgnoredAny) + SIMD memchr Bytes splicers + CutThroughSseStream (0 tokio::spawn, 0 mpsc, 0 per-token JSON DOM) for:
    • POST /v1/chat/completions -> OpenAI / OpenRouter / Copilot / Qwen / local runtimes
    • POST /v1/responses -> ChatGPT Codex (/responses)
    • POST /v1/messages -> Anthropic (anthropic / anthropic-token) & MiniMax (minimax-portal)
  • Zero-Spawn Polyglot Transcode State Machines: Direct BytesMut Stream drivers across all 5 codecs (openai_chat, openai_responses, anthropic, gemini, vercel_ui), O(1) generation-stamped CachedModelPlan, concrete zero-box Tower middleware, 64 KiB vectored writev Hyper connection builder, Linux SO_REUSEPORT multi-queue listeners, mimalloc, and full Fat LTO (lto = "fat").

100% LiteLLM Endpoint Compatibility (997 / 997 Routes)

  • Drop-in coverage across all 997 LiteLLM route decorators: native inference, embeddings, models, spend/budget management, virtual keys, teams, users, organizations, guardrails, MCP, and health endpoints—or structured 400 responses with actionable zero-paywall DIY migration guidance.

Native Windows (x86_64-pc-windows-msvc) Support

  • First-class native Windows .exe release alongside macOS (aarch64-apple-darwin) and Linux (x86_64-unknown-linux-gnu), including seamless .exe / .cmd / .bat agent shim resolution in quixigate run.

Prebuilt Archives

  • macOS (Apple Silicon): quixigate-aarch64-apple-darwin.tar.gz
  • Linux (x86_64): quixigate-x86_64-unknown-linux-gnu.tar.gz
  • Windows (x86_64 MSVC): quixigate-x86_64-pc-windows-msvc.zip

Full Changelog: v0.1.0...v0.2.0

v0.1.0

Choose a tag to compare

@github-actions github-actions released this 12 Nov 09:55