Skip to content

pr-review.yml: upgrade review model to GLM-5.2 (MiniMax M3 fallback) and disable focus_only_on_problems #2

Description

@uuta

Background

On uuta/resorn PR #143, the reusable pr-review.yml workflow ran successfully but reported "No major issues detected" and "No code suggestions found", while Gemini Code Assist (reviewing the same commit fed341e1) caught a real defect: return (await lookup(id)).job; crashes on a malformed 200 response. The author fixed it in a follow-up commit (84e53a54).

Root-cause analysis of the run logs (run 29729720651) showed the miss was configuration, not tool failure:

  1. pr_code_suggestions.focus_only_on_problems was effectively true, so the improve tool is prompted to report only critical bugs and suppress defensive-programming findings.
  2. CONFIG.MODEL is a free-tier model (openrouter/nvidia/nemotron-3-ultra-550b-a55b:free), which has weaker recall than paid coding models. The model genuinely returned code_suggestions: [] — no rate limit or fallback occurred.

Gemini consumer code review is sunset as of 2026-07-17, so this workflow is becoming the only automated reviewer. Model research (OpenRouter pricing + mid-2026 coding benchmarks) selected z-ai/glm-5.2 as primary (same price band as kimi-k2.7-code, and the model behind the top Code-Review-Bench result) with minimax/minimax-m3 as fallback (frontier-adjacent coding at ~1/3 the price).

Implementation contract

In .github/workflows/pr-review.yml, in the "Review pull request with PR-Agent" step env block:

  1. Change CONFIG.MODEL to openrouter/z-ai/glm-5.2
  2. Change CONFIG.FALLBACK_MODELS to '["openrouter/minimax/minimax-m3"]'
  3. Add pr_code_suggestions.focus_only_on_problems: "false"
  4. Check the latest release of The-PR-Agent/pr-agent (gh api repos/The-PR-Agent/pr-agent/releases/latest). If it is newer than the currently pinned release, update the uses: line to pin the new release's tag commit SHA and update the trailing # vX.Y.Z comment to match. As of 2026-07-21 the latest release is v0.39.0 = 8e4d32e5497defd43c023a404f73560c62728961, which is already the pinned SHA — in that case leave the line untouched.
  5. Do NOT add pr_code_suggestions.extra_instructions (deferred; initial phase)
  6. Leave everything else unchanged: CONFIG.CUSTOM_MODEL_MAX_TOKENS, CONFIG.MAX_MODEL_TOKENS, auto_review/auto_describe/auto_improve flags, pr_actions, commitable_code_suggestions, the review-start timestamp step, and the "Verify PR-Agent published a review" step

Done when

  • pr-review.yml on main contains exactly the config changes above (three env changes, plus the action pin bump only if a newer release exists) and no other diff
  • actionlint (or equivalent YAML/workflow validation) passes on the modified file
  • Blackbox: a test PR in a consumer repo (e.g. uuta/resorn) triggers the workflow; the run succeeds; the PR receives a "PR Reviewer Guide" comment; run logs show z-ai/glm-5.2 as the model actually called (not the fallback)

Not done if

  • The pr-agent action is unpinned (tag or branch reference instead of a full commit SHA), or pinned to an untagged commit from main
  • extra_instructions keys are added anywhere
  • The verification step ("Verify PR-Agent published a review") is modified or removed
  • Model IDs are written without the openrouter/ prefix (pr-agent's LiteLLM routing requires it)

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions