v1.4.0: did the run answer the question?
Forum validates each task against its own instruction, but a run can pass every task and still drift from the original request. This release witnesses that gap.
- Witnessed intent check: a completed
submit()appends anintent_checkentry, chained to the answer, with a deterministic coverage of the request's content words by the answer, the terms it missed, and whether that falls below a configurable threshold (Orchestrator(intent_threshold=...)). The payload names its own method (lexical_coverage). It is a lexical floor, not a semantic verdict: low coverage flags a run for a closer look, it does not block the run or declare the answer wrong. A grounded model intent-judge is the next rung above this floor. - In the summary and A/B:
forum ledger summaryandforum benchnow report intent checks and how many were flagged, so a prompt or model change that makes runs drift more or less is measured from the record.
Honest about its limits: a correct but paraphrased answer can still flag (the lexical floor's known blind spot). examples/run_intent.py demonstrates exactly that case.
Pure standard library, deterministic and reproducible. 186 tests, plus 2 gated real-model tests.
Install from the attached wheel: pip install forum_engine-1.4.0-py3-none-any.whl