First release.
Most LLM observability tells you what happened. This tells you whether what happened was acceptable, and it exits 2 when it was not.
llmops gate --window telemetry.jsonl.gz --objectives objectives.yaml
echo $? # 0 held, 2 an objective was missed, 3 the tool could not runWhat it does
Gates on an error budget. Multi-window, multi-burn-rate alerting — four rules, two windows each, and a rule fires only when both windows exceed the threshold. The conventional table from the Google SRE workbook, unchanged, because numbers invented here would have had to be defended here and would have been wrong.
Costs a window, and checks the answer. Decimal end to end, rates per million tokens as vendors publish them, cached input as its own rate rather than a discount factor, dated and content-addressed price books. Reconciliation against the provider's own reported cost, with both a relative and an absolute tolerance.
Refuses a metric before it exists. llm_requests{provider, model, route, service} is 40 million time series and every label in it is bounded. The check is arithmetic at registration, not a list of forbidden names at emit.
Three decisions worth arguing with
- An unpriced model is a hard failure, not a zero. The model missing from the price book is by construction the newest and most expensive one. Charging it zero produces a budget gate that goes green on the exact day it should go red.
- Spend and availability are one engine. Dollars measured against a window's pro-rata share of the period budget land on the same scale as failed calls, so there is one implementation of the two-window conjunction rather than two that drift apart.
- A span has no prompt or completion field. Not a redaction rule — there is nowhere to put one. The most effective privacy control available to a telemetry pipeline, and it costs nothing.
All ten are recorded with their reasoning in ARCHITECTURE.md.
Where the method stops working, measured
docs/burnrate.md records something the workbook does not: a flat pro-rata spend budget under diurnal traffic under-reads at night. On a completely healthy platform the measured burn rate swings 6.5× across a day on a 30-minute window and 1.1× on a one-day window. A short spend window drops out of alert every night and back in every morning. The shipped objectives are sized to the cycle because of it, and the numbers are reproducible from the corpus in this repository.
The telemetry is generated, and says so
Four windows synthesised by llmops synth, a shipped and documented command — a healthy platform, a sharp outage, a latency degradation with no errors at all, and a cost spike with no availability or latency signal. Each breaks exactly one objective, and scripts/check-incidents.py asserts that the healthy window is still green at the same instant. Without that control, "the gate fired at 22:40" could be a fact about 22:40.
Quality
461 tests across five layers — unit, integration, security, end-to-end and meta — at 94.76% line coverage against a 90% gate. CI runs ruff, mypy --strict, all five layers, the coverage gate, the tool against its own corpus, every example, a wheel that is installed and invoked, gitleaks over the full history, bandit, pip-audit, CodeQL and trivy, plus a container smoke test that runs the gate inside the built image and asserts it can still fail there with exit 2 and exit 3 distinctly.
Four runtime dependencies. The OTLP exporter is a hundred lines of stdlib urllib rather than a tracing SDK.