Research PR Fix Agents like SonarQube Gitar

[don-petry]

Research, analyze, compare emerging solutions such as:
https://www.sonarsource.com/products/gitar/

[github-actions[bot]]

🔎 Enhancement — PR Fix Agents (Gitar & peers): buy, build, or harvest?

1. Sharper problem framing

This isn't really a "should we try a new tool" question. We already run a home-built Gitar. dev-lead's fix-ci and fix-reviews intents, plus the multi-stage pr-review pipeline, cover most of what Gitar sells: diagnosing CI failures, committing fixes to the branch, applying review comments, and reviewing with one consolidated comment. So the decision is:

Does an external PR-fix agent fix more of our failing PRs, faster or cheaper, than dev-lead does now? And which of its specific capabilities should we buy, and which should we copy?

We can't answer that yet, because we don't measure our own fix rate. dev-lead-fix-ci.sh writes applied / failed / no-changes / rate-limited markers, but no report aggregates them (token_report.sh and reviewer_report.sh cover cost and third-party reviewers, not fixer outcomes). Gitar's headline number, an "84% fix rate", has nothing on our side to compare against.

2. Gap analysis: Gitar vs. what we already run

Capability	Gitar (Pro)	Ours today	Gap
CI failure → fix → commit	"Autonomous fix loop — iterates until CI passes, commits on green"	dev-lead-fix-ci.sh: action tier (sonnet-5-5 → sonnet-5 → opus-4-8), MAX_CI_CYCLES=3, MAX_FAIL_ATTEMPTS=2, then an exhaustion marker	≈ parity
Review comment → fix	On-demand fixes + interactive agent	dev-lead-fix-reviews.sh with test-tamper, regression and disposition guards, REVIEW_NONCONVERGENCE_LIMIT=3	ours is more guarded
Flaky vs. real failure triage	"Separate real breakage from flaky tests and infrastructure noise"	None. fix-ci treats every red check as a code defect, with no re-run-first step	🔴 real gap: risks LLM "fixes" for infra noise and burns MAX_PR_AUTOMATION_CYCLES
Sonar-aware fixing	Native (same vendor; SonarQube integration on the roadmap)	For external gates, collect_logs() pastes up to 1,000 lines of the PR diff with the instruction "scan for hotspots". It never asks SonarCloud which issues it actually flagged	🔴 real gap, cheap to close: SONAR_TOKEN and the project key already exist
Multiple failing checks	Whole pipeline	build_prompt() reads only CHECKS_JSON[0]	🟡 one check per cycle → extra cycles
Log context	Full CI analysis	--log-failed | tail -n 200	🟡 root cause can sit above the last 200 lines
Fix-rate analytics	"Verification analytics", time-to-merge and fix-rate dashboards	None for fixers	🔴 blocks any buy/build decision
Independent review	Gitar reviews its own fixes	Rubber-duck on a different engine + tier-3 security audit	ours is stronger (separate engine)

3. Competitive landscape (as of Oct 2026)

Tool	How fixes land	Pricing fit for 1 seat, ~223 PRs/mo, mostly agent-authored (see #1348)	Notes
Sonar Gitar (acquired 21 May 2026)	Commits directly to the PR branch ("commits on green")	Core $20–25/user/mo; Pro $40–50/user/mo; Enterprise priced per PR. Unknown: whether bot-authored PRs (donpetry-bot) count as a seat or fall outside the plan	Runs on Anthropic models; BYO key only on Enterprise. Claims zero data retention, SOC 2 Type II. Free Pro tier for OSS (doesn't apply to private repos)
SonarQube Remediation Agent (GA 30 Jun 2026)	One new PR per Sonar issue	Bundled with SonarQube Cloud tiers	Narrow scope (deterministic Sonar findings), so low risk and directly relevant to the gap above
CodeRabbit Autofix (Apr 2026) + @coderabbitai fix-ci (Jul 2026)	Commit to the branch or a stacked PR; fix-ci always uses a stacked PR	We already pay for CodeRabbit (the rate limit is the binding constraint, #1348)	Lowest incremental cost to pilot; stacked PRs connect to #792
Cursor Bugbot Autofix (GA 25 Feb 2026)	Cloud agents in a VM propose fixes	Usage-billed at ~$1–1.50 per run, so it scales badly at our volume	The only public efficacy number: >35% of Autofix changes merged; bug resolution up from 52% to 76%
GitHub "Fix with Copilot" (Jun–Jul 2026)	Stacked PR on top of the failing PR	Copilot Pro was cancelled (#1348), so it's out	Confirms this is now a platform-level, table-stakes feature
Codex CLI autofix cookbook	Self-hosted in GitHub Actions	Model tokens only	Same architecture as our dev-lead, which supports staying build-first

Efficacy reality check: in the AIDev dataset (Copilot, Devin, Cursor and Claude PRs), about 46% of agent-proposed fixes were rejected. CI/test failures and incorrect implementation were the leading causes. Vendor fix-rate claims (Gitar's 84% comes from a single "featured example") need verifying on our own PRs.

4. Options & trade-offs

Option	Cost	Upside	Main risk
A. Buy Gitar Pro and let it auto-commit	~$40–50/mo plus seat ambiguity	Flake triage and analytics without building them	Two writers on one branch. Gitar's commits race dev-lead's pushes and don't count against MAX_PR_AUTOMATION_CYCLES (scripts/lib/pr-automation-budget.sh:33), so runaway loops can slip past our budget guard. Also adds a third-party App with contents: write on self-hosting infra
B. Gitar Core in advisory mode (diagnosis + on-demand fixes only, no autonomous loop)	~$20–25/mo	Gives a second opinion and benchmark data without write contention	Adds noise to our review stack; needs advisory-gate wiring like #1349
C. Harvest the ideas into dev-lead	Engineering time only	Closes the 🔴 gaps on our own guardrails, engines and audit trail	Build effort. Flake detection is subtle
D. CodeRabbit fix-ci as an A/B arm	~$0 incremental	The stacked-PR model avoids the double-writer race	Still capped by CodeRabbit's rate limits

Recommendation: C first, then B (or D) as a measured A/B. Avoid A on .github-private itself: an external auto-committer on the repo that hosts the agent harness breaks our ring-rollout model.

5. Risks and unknowns

• Seat definition: does Gitar bill per human seat or per PR author? With ~100% of volume through one human plus one bot, this decides whether it's $20/mo or "contact sales". Ask during the trial.
• Roadmap churn: Sonar is folding Gitar into SonarQube. Pricing, plan names and the standalone App could change within 2–3 quarters. Avoid annual commitments.
• Prompt-injection surface: fix agents read CI logs and PR text, both attacker-influenceable (CWE-1427; PromptPwnd and Clinejection showed this in 2026). An external agent sits outside agent-shield and our 💡 GitInject-Class PR Content Sanitization Layer for Review Pipeline Defense #1157 sanitisation layer.
• Data egress: code leaves the org for a Sonar/Anthropic processor. Their zero-retention claim is credible but adds a new subprocessor.
• Attribution: fixes from an external bot muddy 💡 Independent Agent Disposition Verification to Catch False Success Claims #2024 (independent disposition verification) unless that bot's commits are tagged and checked.

6. Phased validation path

Phase	Duration	Work	Exit gate
0 — Baseline	~1 wk	Add a fixer_report.sh (or extend fleet_report.sh) that aggregates dev-lead-fix-ci and fix-reviews markers: outcome mix, cycles to green, human-touch rate, $ per green (join with token-metrics.sh)	We know our own fix rate
1 — Cheap in-house wins	1–2 wk	(a) For app_slug=sonarcloud, call api/issues/search?componentKeys=petry-projects_.github-private&pullRequest=$PR with SONAR_TOKEN and inject the structured issues instead of the raw diff. (b) Rerun-once before fixing (gh run rerun --failed); if it goes green, label the failure flaky and skip the LLM. (c) Iterate over all entries in CHECKS_JSON, not just [0]. (d) Use tail plus a grep window around error|FAIL rather than a flat 200 lines	Fix rate improves over the Phase 0 baseline
2 — External A/B	14 days (Gitar trial length)	Install Gitar Core (advisory) or CodeRabbit fix-ci on one product repo (e.g. markets), not on .github-private. On matched failing PRs, compare its proposed fix with dev-lead's	Data for the metrics below
3 — Decide	—	Buy B or D only if it beats the post-Phase-1 dev-lead by the margins below; otherwise retire this thread with findings	Written ADR

7. Success metrics (decision thresholds)

• Fix rate: % of first CI failures reaching green with no human commit within ≤3 cycles. Adopt an external tool only if it beats dev-lead by ≥10 pp.
• Time-to-green: median minutes from first red to green.
• Spurious-fix rate: % of fix commits made on failures a re-run would have cleared. Target <5% after Phase 1b.
• Revert / rework rate: fix commits reverted or rewritten within 7 days. Target ≤ the AIDev-implied ~46% rejection rate, ideally <20%.
• $ per green PR: model tokens plus prorated subscription.
• Budget-exhaustion rate: % of PRs hitting MAX_PR_AUTOMATION_CYCLES. This should fall once flake triage stops the wasted cycles.

Related threads

#1348 (reviewer bake-off; the same 1-seat throughput constraint) · #1349 (advisory-gate wiring pattern) · #2024 (disposition verification) · #792 (stacked PRs, the delivery model CodeRabbit and Copilot use) · #1157 (PR content sanitisation) · #1838 (non-required check circuit breaker; related to flake handling)

Sources

• Sonar — Gitar product page · Gitar pricing · Sonar acquires Gitar (press release) · SiliconANGLE coverage, 21 May 2026
• SonarQube Remediation Agent docs
• CodeRabbit Autofix docs · CodeRabbit fix-ci release
• Cursor — Bugbot Autofix
• GitHub Changelog — Fix with Copilot for failing Actions · Fix failing checks from GitHub Mobile
• OpenAI Cookbook — Codex CLI CI autofix
• Understanding the Rejection of Fixes Generated by Agentic PRs (AIDev), arXiv 2606.13468
• CSA research note — Prompt injection in AI-powered GitHub Actions (May 2026)
• Internal: scripts/dev-lead-fix-ci.sh (collect_logs, build_prompt, cycle limits), scripts/lib/pr-automation-budget.sh, scripts/engine.sh, .github/workflows/sonarcloud.yml

— Mary (BMAD Analyst), single-idea enhancement run, 2026-10-09

[don-petry]

@petry-projects/solution-architect - Review and advise.