v0.1.36 - KV-friendly prompt assembly hotfix
This hotfix fixes a real performance regression in long local-model sessions: Successor now preserves a KV-cache-friendly request shape instead of quietly rewriting earlier prompt bytes as the conversation grows.
What changed
- made prompt assembly append-only in the places that matter for local cache reuse
- runtime-tail context is now always appended as its own synthetic turn instead of being merged into the latest real user message
- adjacent same-role API messages now preserve their original boundaries instead of being merged together later
- stabilized native tool-call serialization
- assistant
tool_calls[].function.argumentsnow use canonical JSON ordering so identical tool payloads serialize to identical bytes
- assistant
- added better cache-shape diagnostics
- turn traces and
/perfnow record request prefix hash, full request hash, and tail kind alongside the existing stable-system hash and llama timing data - this makes it possible to distinguish harness-side request-shape churn from provider-side cache dropout
- turn traces and
Verification
ruff check src testsPYTHONPATH=src pytest -q1277 passed in 25.56s
- live llama.cpp KV verification
- before fix: repeated late-turn
cache_n=0, median first-token latency ~34s, suspected KV misses across many turns - after fix:
cache_ngrows turn-to-turn, hit ratio stays roughly 0.8–0.94, and first-token latency stays sub-second in the extended local-model session
- before fix: repeated late-turn