Skip to content

Releases

PrismGate v1.0.0 — Multi-Tenant LLM Inference Gateway, Speculative Cascade Router & Drift Plane

Choose a tag to compare

@nagarjunak-pixel nagarjunak-pixel released this 15 Nov 14:56

PrismGate v1.0.0

Highlights

  • TCP-Vegas / Netflix Gradient2 Adaptive Concurrency: Little's Law standing-queue estimation dynamically adjusts per-provider max_inflight ceilings (~4,666 req/s throughput, 67 µs median control-plane overhead).
  • Hierarchical Dual Token-Bucket Admission & Priority Load Shedding: Atomic RPS + TPM token buckets with progressive BATCH_BACKGROUND (>=85%) and INTERACTIVE (>=95%) shedding, preserving 97.5% CRITICAL SLOs.
  • Speculative Complexity Cascade Router & Signed-Hash Semantic Cache: Sub-50 µs structural prompt complexity scorer + Weinberger signed-hash tenant-isolated cosine cache delivering 57.16% net FinOps spend reduction.
  • Inline Atomic Claim-Faithfulness Judge: Canonicalized currency/dosage/percentage, entity, and clinical/financial antonym verification preventing RAG hallucinations from poisoning caches.
  • Statistical Drift Plane (PSI + Base-2 JSD): Decile Population Stability Index and Base-2 Jensen-Shannon Divergence formally verified via Hypothesis (97.58% test coverage, strict mypy, TypeScript 5.6 + React 18 console).