Repository navigation
CI Fixer v2.1.0 — Python scorecard 3 of 4 + router-not-needed verdict
3 of 20 scorecard cells green on prod
Delta from ci-fixer-v2.0.0-mvp (1 cell → 3 cells). All closed end-to-end on usephalanx/phalanx-ci-fixer-testbed with real GPT-5.4 + Sonnet 4.6 + real sandbox + real GitHub CI.
| Language | Lint | Test fail | Flake | Coverage |
|---|---|---|---|---|
| Python | ✅ | ✅ | ✅ | ⏳ |
| JavaScript | ⏳ | ⏳ | ⏳ | ⏳ |
| TypeScript | ⏳ | ⏳ | ⏳ | ⏳ |
| Java | ⏳ | ⏳ | ⏳ | ⏳ |
| C# | ⏳ | ⏳ | ⏳ | ⏳ |
| PR | Class | Agent fix | Cost | Tokens | Coder rounds |
|---|---|---|---|---|---|
| #1 | lint | acdcbc5 |
$0.37 | 95k | 2 |
| #2 | test_fail | 07ad29d2 |
$0.17 | 45k | 1 |
| #3 | flake | f9719fd |
$0.19 | 48k | 1 |
Architecture decision: no fix-type router
A router / StrategyRegistry pattern was proposed between cells 1 and 2. We chose data-first over designing from 1 data point. After collecting all three classes, the verdict is clear:
| Axis | Varies across classes? | Worth routing? |
|---|---|---|
| Agent decision sequence | No — identical | No |
| Tool sequence | No — identical | No |
| Coder flow | No — identical | No |
| `validate_cmd` | Yes | No — already auto-extracted from CI log |
| env setup | No (all 3 were pip install) | No |
| sandbox image | No (all 3 Python) | Already routed by stack |
| target files | Yes | No — agent infers from log + diff |
| system prompt | No — identical | No |
GPT-5.4 + the existing 15-tool surface handles fix_type variance implicitly. The relevant signals (validate command, install steps, target paths) are already extracted from the CI log + manifest files. A full Strategy / Router abstraction would have been premature — textbook case of designing from 1 data point and being wrong.
What goes in instead (~1 day of targeted additions)
- Language playbooks — deterministic env setup per stack. Python + pyproject → `pip install -e ".[dev]"`, Node + package.json → `npm ci`, Go + go.mod → `go mod download`, Java → `mvn install -DskipTests`, C# → `dotnet restore`. Replaces a GPT env-planner call per run with a 5-line lookup.
- Coverage rule in base system prompt: "Never lower `--cov-fail-under` or equivalent threshold. Either add tests that cover the gap or escalate."
- New escalation enums: `FLAKY_TEST_DETECTED`, `COVERAGE_ADJUSTMENT_NEEDED_OUT_OF_SCOPE`.
Language router still exists (sandbox_pool stack detection + env planner) and is the only router the code actually needs.
What's next
- Python × coverage (closes Python row; applies `failures/04-coverage-drop.patch` on a new PR)
- JavaScript × 4 cells (new sandbox image + env playbook)
- TypeScript × 4 cells
- Java × 4 cells
- C# × 4 cells
- Harvest real public-repo CI failures into simulation corpus
- Run scorecard against MVP exit gates (Lenient ≥ 95%, Behavioral ≥ 99%)
Release date: 2026-04-20