Skip to content

v0.7.8

v0.7.8: chat_with_meta + reasoning_budget — siwen 5th batch 議 甲乙戊
@Hosuke Hosuke tagged this 21 Apr 12:21 
Direct response to siwen's 2026-04-21 post-mortem of an 11-hour wenguan
failure where the absence of finish_reason='length' detection let
truncated LLM output cache as complete across 62 books.

- 議 甲: tools.llm.chat_with_meta(...) -> (str, LLMMeta) — surfaces
  finish_reason, flattened usage (incl. reasoning_tokens from OpenAI's
  nested completion_tokens_details), attempts, and the actually-used
  model. meta.truncated is the convenience boolean for length-cut.
  Primitive does NOT raise on truncation — caller decides policy
  (Policy A, "mechanism not policy"). chat() is now a thin wrapper;
  zero break for v0.7.x callers.

- 議 乙: tools.llm.reasoning_budget(max_tokens, tokens_per_char,
  safety=0.8) -> int. Pure arithmetic, no upstream model table. Caller
  supplies tokens_per_char empirically. Matches HeadRule /
  TAXONOMY_GENERATOR "mechanism not policy" precedent.

- 議 戊: docs/pipelines.md — new "Choosing the cid" + "Sizing chunks"
  pattern sections. cid must bind to content identity, never to
  input-size knobs like CHUNK_MAX; the 11h loss's cache-wipe symptom
  was exactly this. Sizing section shows chat_with_meta + reasoning_budget
  composition that would have caught the length-cut at the point of
  failure.

- Internal: _call_llm shape changed from str -> (content, finish_reason,
  usage); private symbol, 2 test mocks updated.

- 議 丙 (aggregate_and_fallback) deliberately deferred — heuristic,
  let it mature downstream 2-3 months. 議 丁 is siwen-side integration.

Three Codex review rounds: (1) _extract_usage strict getattr silently
coerced dict-shaped payloads to None → dict-aware accessor; (2)
reasoning_budget overflowed to OverflowError on huge floats → isfinite
guard raises ValueError; (3) int*float can fail before the guard for
ints > float64 max → wrapped in try/except OverflowError→ValueError.
Each fix ships with a regression test. 419 tests pass (28 new).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Assets 2
Loading