v0.59.0.0

The LongMemEval reader now checks the evidence before giving its short answer.

When an answer depends on several old conversations, the benchmark reader now
briefly extracts the relevant facts and reasons over them before answering. A
matched study over 361 questions moved from 308 to 324 judged correct answers
with the same full sessions, dates and question text. A separate replication
using the published supporting sessions also found a gain from notes in both
natural language and JSON; changing the format to JSON alone did not help.
These are reader comparisons, not evidence that everyday gbrain think
improved. The full research record includes losses, ambiguous grades, and nine
notes responses that reached the original 512-token output limit.

How to use it

gbrain eval longmemeval DATASET --reader-mode notes --reader-max-tokens 1024 --judge --output notes.ndjson
gbrain eval longmemeval DATASET --reader-mode direct --judge --output direct.ndjson

The first command's reader settings are now the defaults. Direct keeps the
original prompt and 512-token cap, so existing baseline runs remain
reproducible. --reader-mode notes --reader-max-tokens 512 reproduces the
original treatment's output budget, but risks the same cutoffs. The larger
default cap is a mitigation, not a separately measured answer-quality gain. A
bounded nine-case replay of the prior cutoffs finished all nine naturally at
1024 tokens (445–603 output tokens), but did not remeasure answer accuracy.

What to watch

The reader still sees the same sanitized full conversations in the same order;
notes do not recover facts missing from retrieval. Output-limit, empty and
unknown completions now record an error and preserve any partial text only as
a diagnostic. They cannot be judged as complete answers or silently inflate
accuracy. Receipts pin the reader mode, prompt, model, output budget and finish
reason; resume rejects a different reader configuration even with the
retrieval-mixing override. Paid reader and judge calls remain opt-in.

To take advantage of v0.59.0.0

gbrain upgrade should apply the update. If it reports a partial migration or
gbrain doctor flags one, run gbrain apply-migrations --yes --no-autopilot-install
and then gbrain doctor. Read skills/migrations/v0.59.0.0.md for the
reader-default comparison boundary and verify the flags with
gbrain eval longmemeval --help. No database migration or new model key is
required; running a judged benchmark still needs its configured providers.

Itemized changes

Added

• Add a shared LongMemEval reader config and request builder with explicit
  direct|notes selection and output budget flags, plus per-row and summary
  configuration pins and strict resume checks. The stable package subpath
  gbrain/eval/longmemeval/reader lets companion evaluation tools use the
  same request builder instead of copying prompts. Existing invocation-guard
  and canonical-pricing modules are also exported through stable package
  subpaths so a capped companion evaluator can reuse the real cost controls.
• Preserve original transfer and oracle comparison receipts, negative excerpt
  experiments, source audits and the primary-study compendium under docs/eval/
  and docs/research/, without promoting the experimental selector.

Fixed

• Retain all provider text blocks and any incomplete partial text separately
  from a completed hypothesis. Output-limit, empty and unknown completions are
  recorded as failed reader rows, kept in the judged denominator, and fail the
  benchmark instead of passing a partial answer to the judge.
• In the inactive experimental excerpt runner, future incomplete reader
  responses also stop before judging rather than entering a completed pair;
  historical records and results remain untouched.