PatchLeague

One issue in. Several agents compete. One tested patch out.

Run the same coding task with Codex, Claude Code, Gemini CLI, or OpenCode in separate Git worktrees. Verify every patch, compare the evidence, and apply the winner.

中文文档 · Contributing · Security

 #  AGENT           SCORE   CHECKS    LINES  STATUS
 1  codex            96.4       3/3       41  verified
 2  claude           88.7       2/3       29  checks_failed
 3  gemini           81.2       3/3      186  verified

 Winner: codex (96.4/100)
 Report: .patchleague/runs/20260724-120000-a1b2c3/report.html

Important

PatchLeague is an early alpha. It isolates changes with Git worktrees, which are not security sandboxes. Run only agent CLIs and repositories you trust.

Why PatchLeague?

Choosing a coding agent from anecdotes is hard. PatchLeague gives several agents the exact same repository, task, time budget, and verification commands, then preserves the artifacts needed for a real comparison:

• one detached Git worktree per agent;
• parallel, non-interactive agent execution;
• auto-detected or explicit test, lint, and build commands;
• deterministic scoring with visible components;
• patch-size and risk-signal analysis;
• complete patches and local logs;
• a self-contained offline HTML report;
• a guarded, preview-first path for applying the winner.

PatchLeague uses your existing CLI subscriptions and authentication. PatchLeague itself does not proxy prompts through its own service or upload source code. However, the agent CLIs you configure may send prompts, repository context, or source code to their providers. Review each provider's data-handling policy before running it on private or sensitive repositories.

Quick start

Requirements: Node.js 20+ and Git.

git clone https://github.com/269394628/PatchLeague.git
cd PatchLeague
npm link
patchleague --json doctor

Create a task:

# Fix duplicate retries

`requestWithRetry` performs one extra request after a successful retry.

Acceptance criteria:

- Add a regression test.
- Preserve the public API.
- Run the existing test suite.

Then run the competition from the target repository:

patchleague run issue.md \
  --agents codex,claude \
  --check "npm test" \
  --check "npm run lint"

On PowerShell, use a backtick or place the command on one line.

Commands

patchleague doctor
patchleague init
patchleague agents list
patchleague agents probe <agent>

patchleague run <task-file> [options]

patchleague runs list
patchleague runs show <run-id>
patchleague runs report <run-id>

patchleague winners apply <run-id> [--agent <agent>]
patchleague winners apply <run-id> --confirm

patchleague agent exec <agent> -- <raw args...>

Every read and workflow command supports --json. Progress goes to stderr, leaving stdout parseable.

Configuration

Run patchleague init in a Git repository to create .patchleague/config.json:

{
  "version": 1,
  "defaultAgents": ["codex", "claude"],
  "maxParallel": 2,
  "timeoutMs": 900000,
  "setup": [],
  "checks": "auto",
  "agents": {}
}

Built-in presets exist for codex, claude, gemini, and opencode. Override or add any CLI without writing a plugin:

{
  "version": 1,
  "defaultAgents": ["my-agent"],
  "maxParallel": 1,
  "timeoutMs": 600000,
  "setup": ["npm ci"],
  "checks": ["npm test"],
  "agents": {
    "my-agent": {
      "command": "my-agent",
      "args": ["run", "--json"],
      "prompt": "stdin",
      "versionArgs": ["--version"]
    }
  }
}

Agent prompts can be passed through stdin or through an argument containing {prompt}. Adapter arguments may also use {worktree} and {task_file}.

Use setup for commands that prepare each detached worktree before an agent runs, such as npm ci or uv sync --frozen. Setup commands must be non-interactive and should not change tracked files. They can be overridden with repeatable --setup flags or disabled with --no-setup.

Verification and scoring

When --check is omitted, PatchLeague looks for conventional project commands:

Project signal	Default check
package.json	npm test, then lint or build scripts when available
Cargo.toml	cargo test
go.mod	go test ./...
pyproject.toml / pytest.ini	python -m pytest
Makefile	make test

Before agents start, PatchLeague runs the configured checks once against the base commit in a separate worktree. The saved baseline result distinguishes pre-existing failures from failures introduced by a candidate. Setup commands run before both the baseline checks and every agent.

The default score is deliberately simple and inspectable:

Component	Weight
Verification commands	70
Agent completed successfully	10
Focused patch size	15
Static risk signals	5

No checks means only partial validation credit. An empty patch or failed agent receives a zero score. A patch can be selected as the winner only when the agent completed successfully and every configured check passed. Scores help sort candidates; they do not replace reviewing the diff.

Safe patch application

winners apply is preview-first:

# Runs git apply --check only
patchleague --json winners apply <run-id>

# Modifies the current worktree
patchleague winners apply <run-id> --confirm

PatchLeague refuses to apply when the current worktree is dirty or when HEAD differs from the competition's base commit.

JSON contract

Success:

{
  "schema_version": "1",
  "ok": true,
  "command": "agents list",
  "data": {}
}

Error:

{
  "schema_version": "1",
  "ok": false,
  "command": "run",
  "error": {
    "code": "DIRTY_REPOSITORY",
    "message": "The repository has uncommitted changes."
  }
}

JSON stdout never contains provider credentials. Agent and verification output is saved as local log files and represented by paths in run metadata.

How it works

flowchart LR
    T[Task + checks] --> W[Detached worktrees]
    W --> C[Codex]
    W --> A[Claude Code]
    W --> G[Other agents]
    C --> V[Patch capture + checks]
    A --> V
    G --> V
    V --> R[Ranking + offline report]
    R --> P[Preview / apply winner]

Loading

Run artifacts live under .patchleague/runs/<run-id>/. Temporary worktrees are removed after patch capture unless --keep-worktrees is passed.

Roadmap

• Parse provider-native token and cost telemetry.
• Container and OS-level sandbox backends.
• GitHub Issue and pull-request URLs as task sources.
• Flaky-test detection and retry policies.
• Replayable agent/tool event timelines.
• CI mode and shareable redacted reports.

Development

The project intentionally has no runtime dependencies.

npm run verify
node bin/patchleague.js --help
node bin/patchleague.js --json doctor

See CONTRIBUTING.md for the project layout and pull-request expectations.

License

MIT