Skip to content

fix(llm-worker): cap DailyReport.summary at a length Ollama can compile - #1766

Merged
Xore merged 1 commit into
mainfrom
llm-daily-report-grammar
Aug 23, 2026
Merged

fix(llm-worker): cap DailyReport.summary at a length Ollama can compile#1766
Xore merged 1 commit into
mainfrom
llm-daily-report-grammar

Conversation

@Xore

@Xore Xore commented Aug 23, 2026

Copy link
Copy Markdown
Owner

Closes #1748

The daily-report pipeline has never produced a single real output. Every attempt failed with 400 Failed to initialize samplers: failed to parse grammar, and the workaround was to disable it — where it has sat since 2026-08-22.

It is one field

Each annotation schema is handed to Ollama as a structured-output format and compiled into a GBNF grammar, where a bounded string becomes a repetition rule. Past roughly 1200 that grammar stops compiling and the whole schema is rejected. DailyReport declared summary maxLength: 2000 and was the only schema over the line.

Bisected against the deployed Ollama 0.32.13:

GRAMMAR  full DailyReport
GRAMMAR  summary alone
OK       the three arrays alone
OK       full schema, summary lowered to 1200
GRAMMAR  full schema, smaller maxItems
GRAMMAR  full schema, bounded array items

The arrays are innocent. PayloadAnalysis survives only because its summary was already 1200.

The guard matters more than the fix

The bound is now one shared constant, MAX_ANNOTATION_STRING, with a test that walks every annotation schema — including inside arrays and $defs — and fails if any field declares more.

A single field crossing that line silently disables an entire pipeline, and nothing caught it for the whole life of this code, because the failure only appears when that annotation type actually runs against a live model. This one shipped broken and was only noticed when captured-data mode was finally enabled, long after it was written.

How this was nearly missed twice

An earlier pass — mine — concluded the bug no longer reproduced. That was an artefact of testing with num_predict: 16, which appears to skip grammar compilation entirely:

DailyReport  num_predict=16                       OK
DailyReport  num_predict=256 / 512 / 1024 / 1200  GRAMMAR

Any future test of this must use a realistic output budget or it reports a false pass. PR #1749's description repeats that wrong claim and should be read alongside #1748's root-cause comment.

Verified

Against the live model after the change: 256, 512, 1024 and 1200 all succeed, as does PayloadAnalysis at the same budget. 32 unit tests pass.

LLM_DAILY_REPORT_ENABLED is already true on the live worker, so once this deploys the first real daily report should land in llm-analysis — and the container should stop reporting unhealthy, since the stage error clears.

The daily-report pipeline has never produced a single real output. Every
attempt failed with

    400 Failed to initialize samplers: failed to parse grammar

and the workaround was to disable it, which is where it has sat since
2026-08-22.

It is one field. Each annotation schema is handed to Ollama as a
structured-output format and compiled into a GBNF grammar, where a bounded
string becomes a repetition rule; past roughly 1200 that grammar stops
compiling and the whole schema is rejected. DailyReport declared
summary maxLength 2000 and was the only schema over the line.

Bisected against the deployed Ollama 0.32.13:

    GRAMMAR  full DailyReport
    GRAMMAR  summary alone
    OK       the three arrays alone
    OK       full schema with summary lowered to 1200
    GRAMMAR  full schema with smaller maxItems
    GRAMMAR  full schema with bounded array items

The arrays are innocent. PayloadAnalysis survives only because its summary
was already 1200.

The bound is now one shared constant, MAX_ANNOTATION_STRING, with a test that
walks every annotation schema -- including inside arrays and $defs -- and
fails if any field declares more. A single field crossing that line silently
disables an entire pipeline, and nothing caught it for the whole life of this
code, because the failure only appears when that annotation type runs against
a live model.

Worth recording how this was nearly missed twice. An earlier pass concluded
the bug no longer reproduced; that was an artefact of testing with
num_predict 16, which appears to skip grammar compilation entirely. At a
realistic output budget it reproduces every time:

    DailyReport  num_predict=16                       OK
    DailyReport  num_predict=256 / 512 / 1024 / 1200  GRAMMAR

Any future test of this has to use a realistic budget or it reports a false
pass.

Verified after the change against the live model: 256, 512, 1024 and 1200 all
succeed, as does PayloadAnalysis at the same budget. 32 unit tests pass.

Closes #1748
@github-actions

Copy link
Copy Markdown

Dependency Review

✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.

Scanned Files

None

@Xore
Xore merged commit b1e6138 into main Aug 23, 2026
86 checks passed
@Xore
Xore deleted the llm-daily-report-grammar branch August 23, 2026 15:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

llm-worker: DailyReport structured output is disabled — Ollama rejects its JSON schema with a grammar-parse error

1 participant