fix(llm-worker): cap DailyReport.summary at a length Ollama can compile - #1766
Merged
Conversation
The daily-report pipeline has never produced a single real output. Every
attempt failed with
400 Failed to initialize samplers: failed to parse grammar
and the workaround was to disable it, which is where it has sat since
2026-08-22.
It is one field. Each annotation schema is handed to Ollama as a
structured-output format and compiled into a GBNF grammar, where a bounded
string becomes a repetition rule; past roughly 1200 that grammar stops
compiling and the whole schema is rejected. DailyReport declared
summary maxLength 2000 and was the only schema over the line.
Bisected against the deployed Ollama 0.32.13:
GRAMMAR full DailyReport
GRAMMAR summary alone
OK the three arrays alone
OK full schema with summary lowered to 1200
GRAMMAR full schema with smaller maxItems
GRAMMAR full schema with bounded array items
The arrays are innocent. PayloadAnalysis survives only because its summary
was already 1200.
The bound is now one shared constant, MAX_ANNOTATION_STRING, with a test that
walks every annotation schema -- including inside arrays and $defs -- and
fails if any field declares more. A single field crossing that line silently
disables an entire pipeline, and nothing caught it for the whole life of this
code, because the failure only appears when that annotation type runs against
a live model.
Worth recording how this was nearly missed twice. An earlier pass concluded
the bug no longer reproduced; that was an artefact of testing with
num_predict 16, which appears to skip grammar compilation entirely. At a
realistic output budget it reproduces every time:
DailyReport num_predict=16 OK
DailyReport num_predict=256 / 512 / 1024 / 1200 GRAMMAR
Any future test of this has to use a realistic budget or it reports a false
pass.
Verified after the change against the live model: 256, 512, 1024 and 1200 all
succeed, as does PayloadAnalysis at the same budget. 32 unit tests pass.
Closes #1748
Dependency Review✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.Scanned FilesNone |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #1748
The daily-report pipeline has never produced a single real output. Every attempt failed with
400 Failed to initialize samplers: failed to parse grammar, and the workaround was to disable it — where it has sat since 2026-08-22.It is one field
Each annotation schema is handed to Ollama as a structured-output
formatand compiled into a GBNF grammar, where a bounded string becomes a repetition rule. Past roughly 1200 that grammar stops compiling and the whole schema is rejected.DailyReportdeclaredsummarymaxLength: 2000and was the only schema over the line.Bisected against the deployed Ollama 0.32.13:
The arrays are innocent.
PayloadAnalysissurvives only because itssummarywas already 1200.The guard matters more than the fix
The bound is now one shared constant,
MAX_ANNOTATION_STRING, with a test that walks every annotation schema — including inside arrays and$defs— and fails if any field declares more.A single field crossing that line silently disables an entire pipeline, and nothing caught it for the whole life of this code, because the failure only appears when that annotation type actually runs against a live model. This one shipped broken and was only noticed when captured-data mode was finally enabled, long after it was written.
How this was nearly missed twice
An earlier pass — mine — concluded the bug no longer reproduced. That was an artefact of testing with
num_predict: 16, which appears to skip grammar compilation entirely:Any future test of this must use a realistic output budget or it reports a false pass. PR #1749's description repeats that wrong claim and should be read alongside #1748's root-cause comment.
Verified
Against the live model after the change: 256, 512, 1024 and 1200 all succeed, as does
PayloadAnalysisat the same budget. 32 unit tests pass.LLM_DAILY_REPORT_ENABLEDis alreadytrueon the live worker, so once this deploys the first real daily report should land inllm-analysis— and the container should stop reporting unhealthy, since the stage error clears.