Repository navigation
Releases
Releases · nagarjunak-pixel/prism-gate
PrismGate v1.0.0 — Multi-Tenant LLM Inference Gateway, Speculative Cascade Router & Drift Plane
PrismGate v1.0.0 — Multi-Tenant LLM Inference Gateway, Speculative Cascade Router & Drift Plane
Latest
PrismGate v1.0.0
Highlights
- TCP-Vegas / Netflix Gradient2 Adaptive Concurrency: Little's Law standing-queue estimation dynamically adjusts per-provider
max_inflightceilings (~4,666 req/s throughput, 67 µs median control-plane overhead). - Hierarchical Dual Token-Bucket Admission & Priority Load Shedding: Atomic RPS + TPM token buckets with progressive
BATCH_BACKGROUND(>=85%) andINTERACTIVE(>=95%) shedding, preserving 97.5% CRITICAL SLOs. - Speculative Complexity Cascade Router & Signed-Hash Semantic Cache: Sub-50 µs structural prompt complexity scorer + Weinberger signed-hash tenant-isolated cosine cache delivering 57.16% net FinOps spend reduction.
- Inline Atomic Claim-Faithfulness Judge: Canonicalized currency/dosage/percentage, entity, and clinical/financial antonym verification preventing RAG hallucinations from poisoning caches.
- Statistical Drift Plane (
PSI+Base-2 JSD): Decile Population Stability Index and Base-2 Jensen-Shannon Divergence formally verified via Hypothesis (97.58% test coverage, strict mypy, TypeScript 5.6 + React 18 console).