Skip to content

v0.0.1-rc14

Pre-release
Pre-release

Choose a tag to compare

@guelfoweb guelfoweb released this 07 Jul 16:01

v0.0.1-rc14

Diagnostic and token-reduction release for KV/final prompt work.

Summary

  • Add cross-phase KV LCP diagnostics.
  • Add final/retry prompt component token diagnostics.
  • Add per-card evidence token diagnostics.
  • Reduce compact final evidence metadata for small final outputs.
  • Keep MTP diagnostics and stability gates intact.

KV / final prompt diagnostics

This RC adds diagnostics for understanding why some final/retry calls reuse only a tiny KV prefix.

The diagnostics confirmed:

  • route -> final tokenized LCP can be 4
  • the first mismatch happens at the system prompt
  • cached=4 on final/retry is prompt-view divergence, not a backend/cache failure

Final prompt token reduction

This RC reduces evaluated tokens in compact final evidence prompts by using a smaller prompt-facing evidence card for small shell/grep/unknown records.

Observed smoke results:

  • pwd_followup final_from_tool: prompt about 154, evaluated about 150, correctness correct
  • shell_error final_from_tool: prompt about 245, evaluated about 241, correctness correct
  • shell_error chat_final: intentionally mostly unchanged, correctness correct

Evidence diagnostics

This RC adds safe, diagnostic-only evidence token breakdowns:

  • final/retry component token counts
  • per-card evidence token counts
  • no raw prompt text, raw commands, raw paths, or raw excerpts are emitted

These diagnostics showed that multi-card chat_final remains dominated by evidence metadata, but no safe lineage/intent discriminator exists yet for reducing those cards without risking dual-shell correctness.

MTP status

MTP remains stable and diagnostic/experimental for performance work.

Validated:

  • MTP strict smoke: PASS
  • timeout/cancel recovery: PASS
  • final /props: mtp_enabled=true, mtp_initialized=true, mtp_failure_reason=null, in_flight=false

Validation

  • PYTHONPATH=src python3 -m orbit.native_llama.build_cli --with-mtp-shim
  • python3 -m compileall -q src tests scripts
  • targeted unit gates
  • PYTHONPATH=src python3 -m unittest discover -s tests -q
  • KV/final token reduction smoke
  • MTP strict smoke
  • timeout/cancel recovery smoke
  • git diff --check