Skip to content

v0.1.6 - Level 2 latency benchmark + get_dependencies fix

Choose a tag to compare

@knight22-21 knight22-21 released this 17 Sep 19:26
· 84 commits to main since this release
a558345

What's new in v0.1.6

Level 2 latency benchmark

  • benchmarks/run_latency_benchmark.py: measures p50/p95/p99 query latency with warmup runs
  • Engine held open across all tasks — times pure query latency, not indexing overhead
  • Results on psf/requests v2.32.3 (20 reps, 3 warmup discarded):
    • get_context p50=1.2ms, p95=1.8ms
    • get_callers p50=1.2ms, p95=1.8ms
    • get_impact p50=2.0ms, p95=3.4ms
    • get_dependencies p50=3.8ms, p95=5.1ms (after fix)
    • Overall p50=1.9ms, p95=2.5ms

All queries under 5ms p95 — noise relative to LLM API latency (500-3000ms).

Performance fix: get_dependencies

  • Was doing SELECT * FROM symbols (full table scan) on every call
  • Fixed with SELECT DISTINCT name WHERE kind != 'import' (name-only projection)
  • Latency dropped from 10.7ms to 3.8ms p50 on the requests corpus (64% faster)
  • All 341 tests pass