Skip to content

fix(ci): prevent AI review comments from overwriting each other#104

Merged
loothero merged 1 commit into
mainfrom
fix/review-comment-overwrite
Feb 14, 2026
Merged

fix(ci): prevent AI review comments from overwriting each other#104
loothero merged 1 commit into
mainfrom
fix/review-comment-overwrite

Conversation

@loothero
Copy link
Copy Markdown
Member

@loothero loothero commented Feb 14, 2026

Summary

  • Removed use_sticky_comment: true from all 4 Claude review jobs (contracts, client, indexer-api, general) so each run creates a new comment instead of overwriting the single sticky comment
  • Simplified all 4 Codex review "Post review comment" steps to always gh pr comment instead of finding and PATCHing existing comments via HTML markers
  • Net result: each of the 8 AI review jobs now creates a fresh comment, preventing concurrent jobs from clobbering each other's output

Problem

The claude-code-action sticky comment mechanism has no per-job differentiator — it finds the most recent comment from the Claude bot and overwrites it. When multiple Claude review jobs run concurrently, the last one to finish wins and the other reviews are lost. The Codex jobs had an equivalent issue using HTML marker-based find-and-update logic.

What stays unchanged

  • All "Build review prompt" steps
  • All "Run Claude/Codex review" steps (other than removing use_sticky_comment)
  • All "Check for blocking findings" steps (Claude's API-polling check filters by heading with | last; Codex checks /tmp/review.txt directly)
  • All job dependencies and conditions

Test plan

  • Verify YAML is valid (python3 -c "import yaml; yaml.safe_load(open(...))")
  • grep -n 'use_sticky_comment' .github/workflows/pr-ci.yml returns no results
  • grep -n 'COMMENT_MARKER\|EXISTING_COMMENT_ID' .github/workflows/pr-ci.yml returns no results
  • grep -n 'PATCH --input' .github/workflows/pr-ci.yml returns no results
  • Confirm all 4 Codex "Post review comment" steps use simple gh pr comment only
  • Open a test PR touching multiple scopes and verify all review comments appear independently

🤖 Generated with Claude Code

Summary by CodeRabbit

Release Notes

  • Chores
    • Simplified pull request review comment handling by streamlining the feedback posting process. Review comments are now posted consistently across all review steps without the complexity of managing comment updates.

The 4 Claude review jobs all used `use_sticky_comment: true`, which finds
the most recent Claude bot comment and overwrites it. When jobs run
concurrently, the last to finish wins and other reviews are lost.
Similarly, 4 Codex jobs used HTML marker-based find-and-update logic.

Remove sticky/update-in-place behavior from all 8 AI review jobs so each
always creates a new comment. The "Check for blocking findings" steps
already filter by heading and continue to work correctly with multiple
comments.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings February 14, 2026 18:29
@vercel
Copy link
Copy Markdown

vercel Bot commented Feb 14, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
summit Ready Ready Preview, Comment Feb 14, 2026 6:29pm

Request Review

@gemini-code-assist
Copy link
Copy Markdown

Note

Gemini is unable to generate a summary for this pull request due to the file types involved not being currently supported.

@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented Feb 14, 2026

📝 Walkthrough

Walkthrough

The PR simplified GitHub Actions workflow PR comment handling by removing sticky-comment logic and marker-based extraction/patching across multiple review steps. Comments are now always posted fresh via gh pr comment rather than attempting to update existing bot comments.

Changes

Cohort / File(s) Summary
Workflow Comment Simplification
.github/workflows/pr-ci.yml
Removed COMMENT_MARKER logic, use_sticky_comment parameters, and conditional PATCH/POST behavior across all review steps (contracts, client, indexer-api, general, indexer). Replaced with direct gh pr comment invocations that always post new comments.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~8 minutes

Poem

🐰 No more sticky comments, no markers to find,
Just fresh new posts with a simpler mind!
The workflow hops lighter, the logic less fuss,
Each review just comments—no patching for us! ✨

🚥 Pre-merge checks | ✅ 3 | ❌ 1
❌ Failed checks (1 warning)
Check name Status Explanation Resolution
Description check ⚠️ Warning The PR description includes a clear summary of changes, problem statement, and test plan. However, it omits several required template sections: Scope checkboxes, Change Type selection, Validation commands, Risk level, and Linked Issues. Complete the missing template sections: mark scope areas touched, select change type (fix), provide validation commands, specify risk level, and add linked issues if applicable.
✅ Passed checks (3 passed)
Check name Status Explanation
Title check ✅ Passed The title 'fix(ci): prevent AI review comments from overwriting each other' accurately and specifically describes the main change in the PR—preventing concurrent AI review jobs from losing comments by removing sticky comment logic.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Merge Conflict Detection ✅ Passed ✅ No merge conflicts detected when merging into main

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch fix/review-comment-overwrite

No actionable comments were generated in the recent review. 🎉

🧹 Recent nitpick comments
.github/workflows/pr-ci.yml (1)

7-9: Consider: comment accumulation on long-lived PRs.

With cancel-in-progress: true (line 8) and if: always() on the post steps, a cancelled run may have already posted a partial comment before the next run starts and posts another. Over many pushes to the same PR, up to 8 fresh comments per push could accumulate. This is the expected tradeoff for correctness, but for noisy PRs you might eventually want a lightweight cleanup step (e.g., delete prior bot comments matching the heading before posting). Not blocking — just something to watch.

Also applies to: 511-526


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes a critical concurrency issue where multiple AI review jobs running in parallel would overwrite each other's comments, causing review output to be lost. The fix removes sticky comment mechanisms from both Claude and Codex review jobs, ensuring each review creates a fresh comment instead of updating existing ones.

Changes:

  • Removed use_sticky_comment: true parameter from all 4 Claude review jobs (contracts, client, indexer-api, general)
  • Simplified all 4 Codex review jobs to use direct gh pr comment instead of finding and updating existing comments via HTML markers and PATCH operations
  • Preserved all blocking findings checks, which continue to work correctly by filtering comments by header and selecting the most recent one

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@loothero loothero merged commit 9949149 into main Feb 14, 2026
26 checks passed
@loothero loothero deleted the fix/review-comment-overwrite branch February 14, 2026 18:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants