Skip to content

v0.9.0 — Phase 10: serving & performance

Choose a tag to compare

View all tags
@cattolatte cattolatte released this 17 Jul 01:13
v0.9.0
5bb886f

The whole pipeline behind one API.

  • FastAPI: /health, /passages, /ask, SSE /ask/stream, /metrics; per-stage latency instrumentation (StageTimer).
  • Docker: Dockerfile + docker-compose.yml (api + demo UI + Postgres profile) and a zero-framework demo/index.html (clickable PubMed citations, GROUNDED/ABSTAIN badge, disclaimer).
  • int8 + speculative decoding already exist in Zenith. Latency numbers are earned under real load.
Assets 2
Loading