fix(foreman/executor): route reviewer-role GO through modelDecidedResult#545
Merged
Defilan merged 1 commit intoMay 26, 2026
Merged
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
19 tasks
Reviewer-role Agents are read-only by design: their tool whitelist excludes write_file and str_replace, so they never produce workspace changes. When such an Agent submits verdict=GO (meaning APPROVE), the executor's GO branch was hitting HasChanges -> false -> noChangesResult, which (a) downgraded the verdict to NO-GO with summary "model emitted GO but produced no diff" and (b) dropped the model's submit_result.extra payload (reviewOutcome, findings, issueAsk, filesTouched, etc.) on the floor. The structured review survived only in the transcript ConfigMap. Route reviewer-role GO through the same modelDecidedResult path as non-GO verdicts. status.result preserves the verdict and surfaces extra.modelExtra unmodified. No effect on coder/verifier/planner roles: only Agent.spec.role == reviewer changes behavior. Surfaced by the M5-lite reviewer demo on 2026-05-25: review-449, review-506, review-510 all emitted GO/APPROVE with 3-5 structured findings each, all got recorded as NO-GO no-diff at the task level. Fixes defilantech#543 Signed-off-by: Christopher Maher <chris@mahercode.io>
40721bc to
04b3beb
Compare
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Routes reviewer-role Agents'
verdict=GOthroughmodelDecidedResultinstead of theHasChanges→Commit→Pushflow that coder Agents take.Why
Fixes #543
Reviewer Agents are read-only by design (
Agent.spec.toolsexcludeswrite_fileandstr_replace), so they never produce workspace changes. When such an Agent submitsverdict="GO"meaning APPROVE, the existing executor's GO branch hitHasChanges == falseand fell through tonoChangesResult, which downgraded the verdict to NO-GO with summary"model emitted GO but produced no diff"and dropped the model'ssubmit_result.extrapayload (reviewOutcome,findings,issueAsk,filesTouched, etc.) on the floor. The structured review survived only in the transcript ConfigMap.Surfaced cleanly by the M5-lite reviewer demo on 2026-05-25:
review-449,review-506,review-510all emitted GO/APPROVE with 3-5 structured findings each, all got recorded as NO-GO no-diff at the AgenticTask level. Reviewer model behavior was correct; executor was discarding the output.How
One-line change in
pkg/foreman/agent/executor_native.goat the verdict-handling branch inrunLLMPath: condition now reads "non-GO verdict OR reviewer-role Agent" and routes both tomodelDecidedResult. The non-reviewer GO path (coder commit + push) is untouched.Strict role check (not "Agent's tools lack write_file") so the contract is explicit in the CR and obvious to users modeling pipelines. Future Agent roles get their own path if they need one; this PR doesn't make assumptions about them.
Adds a regression test
TestNativeExecutor_ReviewerGoIsApproveNotCommitthat constructs a reviewer-role Agent with a read-only tool whitelist, has the fake loop submit GO with realistic structured extra (mirroring the v5 reviewer payload:reviewOutcome=APPROVE, two findings,issueAskverbatim quote,filesTouched), and asserts:res.Verdict == GO(not downgraded)res.Extra["outcome"] == "MODEL-DECIDED"(not "NO-CHANGES")res.Extra["modelExtra"]round-trips withreviewOutcome,findings,issueAskintactChecklist
make testpasses locallymake lintpasses locallygit commit -s) per DCO