fix: keep a synthesis that opens with a line of chat - #21
Merged
Conversation
valid_synth_output required the very first line of the synthesizer's output to be a heading. A model that prefaces the document with "Here is the merged document:" therefore had an otherwise correct merge of a 60 KB file thrown away, and auto went on to spend another multi-minute call on the next fallback before settling for the deterministic merge. Prose above the first heading is now dropped rather than fatal, bounded to ten lines. Past that it is a refusal and not a document, and the deterministic merge already in place is the better answer. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
hibeekaey
added a commit
that referenced
this pull request
Aug 17, 2026
- `valid_synth_output` checked only that the output was non-empty and opened with a heading. A refusal that happens to start with one therefore passed, replacing the whole memory file with three lines and pushing that stub to every agent — the `.bak` was the only way back. A synthesis must now keep at least a quarter of the document's lines. - The threshold is set against measurement rather than taste: shrinking is most of the job (folding the imported sections away), a real merge measured 45% of its input, and a refusal measured 2%. 25% sits an order of magnitude above the refusal and well under the merge. - Test fixtures now answer proportionally to the prompt they are given, the way a synthesizer does, instead of returning a one-line stub. The old mocks were smaller than the floor, which is itself evidence the fixtures were not modelling the real thing. - Version bumped to 1.6.2; merging this cuts the release. Validation: falsified three ways — floor at 0 lets the stub replace the canon, floor at 60 wrongly refuses a legitimate half-document, and inverting the comparison breaks the healthy path. All nine suites and ShellCheck over `bin/agent scripts/*.sh tests/*.sh` green. This closes the hole left open deliberately in #21. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
valid_synth_outputrequired the very first line of the synthesizer's output to be a heading. A model that prefaces the document with "Here is the merged document:" therefore had an otherwise correct merge of a 60 KB file thrown away, andautowent on to spend another multi-minute call on the next fallback before settling for the deterministic merge. Found while diagnosing a realagent: claude synthesis failed; trying the next fallback.#currently replaces the whole file, recoverable only from.bak), but every threshold I could pick was arbitrary, and a legitimate consolidation measured 45% of its input. Worth a separate decision rather than a guessed constant.Validation: both new assertions fail against the pre-fix binary and pass after — the preamble case, and a fourteen-line refusal ending in a heading that must not be rescued. Falsified the bounded window by widening it to 100 (the refusal gets accepted) and by unwiring the strip (the preamble case fails again). Full suite green, nine suites, plus ShellCheck over
bin/agent scripts/*.sh tests/*.sh.🤖 Generated with Claude Code