Skip to content

v0.3.0 — OTel observability + leanctx bench CLI

Choose a tag to compare

@jia-gao jia-gao released this 26 Apr 17:18
· 48 commits to main since this release
df588a0

First release with production observability and a reproducible benchmark CLI.

Highlights

OpenTelemetry observability (opt-in via [otel] extra)

  • Every wrapper request path emits one root leanctx.compress span with provider, method, input_tokens, output_tokens, cost_usd, duration_ms.
  • 12 wrapper paths instrumented: Anthropic / OpenAI / Gemini × sync + async × stream + non-stream.
  • Per-compressor child spans (leanctx.compressor.compress) for Lingua / SelfLLM / Verbatim, with sync + async coverage.
  • 4 OTel counters + 1 histogram (leanctx.compress.{calls,input_tokens,output_tokens,cost_usd,duration_ms}), labeled by provider / method / status.
  • Closed leanctx.method taxonomy (8 documented values: passthrough, below-threshold, empty, opaque-bailout, verbatim, lingua, selfllm, hybrid).
  • API-only — leanctx never owns the OTel SDK or registers providers / exporters; the application configures OTel.
  • Bounded error attributes (no raw exception strings on spans) for cardinality + PII safety.
  • Cold-import budget preserved — import leanctx (no extras) does not import opentelemetry.

See docs/observability.md for the full attribute reference, stream-lifetime contract, and sample app-side OTel SDK setup.

leanctx bench CLI

pip install 'leanctx[bench,lingua]'
leanctx bench list                                  # six scenarios
leanctx bench run lingua-local --workload rag       # offline LLMLingua-2
leanctx bench run agent-structural --workload agent # 5 invariants enforced
leanctx bench run anthropic-e2e --workload chat     # full stack, respx-mocked
leanctx bench run selfllm-anthropic --workload rag  # live API
  • Six named scenarios: lingua-local, anthropic-e2e, selfllm-{anthropic,openai,gemini}, agent-structural.
  • Versioned JSON output (schema_version: "1") with documented field set.
  • --runs N produces N records; fresh client/middleware per run (no cross-run state leakage).
  • Clean diagnostics for missing extras / API keys (exit 3, no traceback).
  • agent-structural enforces the 5 structural-integrity invariants (tool_use_id linkage, code verbatim, error verbatim, tool_use input preserved, log compressed).

Bug fixes

  • AC-5 cost regression fixedMiddleware._aggregate now threads cost_usd through hybrid runs (was dropping SelfLLM cost in v0.2). usage.leanctx_cost_usd now attached to provider responses.
  • Gemini opaque-bailout method fixed — non-text contents now report method=opaque-bailout instead of rendering as passthrough.

Internals

  • New package leanctx.observability with API-only OTel access.
  • compression_span (class-based context manager) with detach-for-streams pattern; compressor_span (always-emit per-compressor); shared ContextVar[int] depth counter for parentage and exception unwind.
  • Span-owning iterators close at first of: iterator exhaustion / .close() / __del__ GC backstop.
  • 4 scripts/integration_test_*.py files now thin wrappers around the bench scenarios (single source of truth).

Tests + tooling

241 passed, 14 skipped, 0 failed. ruff + mypy --strict clean across 39 source files.

Verify

pip install leanctx==0.3.0
python -c "import leanctx; print(leanctx.__version__)"  # 0.3.0
leanctx bench list

Implementation built across a 5-round RLCR loop (plan).