PatchTwin

PatchTwin is an agent framework for SWE-bench / SWT-Bench-style tasks. It models bug fixing as the joint generation of a pair of twin patches:

• code_patch: a minimal code fix for the issue.
• test_patch: a new reproduction test patch that demonstrates the issue and validates the fix.

The project supports two execution modes: PatchTwin, where the core agent generates the twin patches on its own, and PatchTwin+Guard, where Guard adds post-test review and corrective feedback. This setup is designed to study how a coding agent coordinates, misaligns, and recovers across the two goals of fixing code and writing reproduction tests.

Environment setup, model configuration, execution commands, evaluation commands, and output formats are documented separately in RUNNING.md.

Note: We use SWE-bench Docker images for all runs. As a result, SWT-Bench evaluation results may differ slightly from those obtained using the official SWT-Bench images.

Project Positioning

Standard SWE-bench evaluation focuses on whether the final code patch passes the official tests. PatchTwin treats reproduction testing as a first-class objective as well. This makes it possible to study whether the agent actually understands the issue, rather than only producing a code-side patch that happens to pass.

PatchTwin is implemented by agents/RepoAgent, which handles repository exploration, code editing, reproduction test creation, and test execution. In PatchTwin+Guard, Guard is invoked after test execution to review the current reproduction test and fix. If Guard rejects the current state, it returns feedback that the agent uses in the next edit.

Design Highlights

1. Joint Code Fix and Reproduction Test Generation

PatchTwin requires the agent to submit both a code fix and a new reproduction test file. The test is not a secondary artifact; it is recorded, extracted, and evaluated alongside the code patch.

This design lets the system distinguish several important outcomes:

• The code is fixed, but the test does not truly reproduce the issue.
• The test reproduces the issue, but the code fix is incomplete.
• The code and test pass together, but both drift away from the original issue.
• Both code and test are correct and form a reliable validation loop.

2. PatchTwin+Guard as Process-Level Review

Guard does not only score the final result. It is invoked after test runs and mainly checks two questions:

• Does the reproduction test fail on the original code for an issue-aligned reason?
• Does the code fix fully resolve the issue without only overfitting the current test or adding unsupported assumptions?

When Guard rejects the current test/fix state, it returns concrete feedback for the next edit. This lets the agent revise the reproduction test, the code fix, or both after observing test results and Guard's diagnosis.

3. Single-Test-File Boundary

PatchTwin constrains the test artifact to one newly created reproduction test entry file. The agent is not allowed to modify existing test files. This boundary is enforced through tools such as create_test_file, move_test_file, and str_replace.

The constraint helps:

• Prevent the agent from modifying existing tests to avoid the real issue.
• Keep the test patch easy to locate, replay, and evaluate independently.
• Cleanly separate the code patch from the test patch.

4. Controlled Repository Tooling

The agent does not receive arbitrary shell access. Instead, it uses repository tools for search, file reading, precise edits, test-file creation, test-file movement, and test execution. The edit tool requires exact string replacement and records each action together with the patch state at that moment.

This design provides:

• Auditability: every tool call is written to actions/ for later trajectory analysis.
• Control: exact replacement, single-test-file limits, and test-run classification reduce unrelated changes.

5. Dual Evaluation View

The project separates final evaluation into three views:

• Code patch evaluation: checks whether the issue is resolved using the official SWE-bench logic.
• Test patch evaluation: checks whether the new test fails on the original code and passes after the official fix.
• Code/test agreement evaluation: applies the agent's final code patch and test patch together and checks whether they agree.

Agreement evaluation is especially useful for detecting false agreement: the code and test may pass together even when one or both are not aligned with the original issue.

Repository Structure

agents/
  base_agent.py      Shared message loop, tool calls, compression, and recovery
  repo_agent.py      Core PatchTwin implementation for fixing code, writing tests,
                    and consuming Guard feedback
  guard.py           Guard reviewer for reproduction-test and code-fix checks

tools/
  read_file.py       Read repository files
  grep_search.py     Search code
  str_replace.py     Precise string-replacement edits
  create_test_file.py / move_test_file.py
                    Create and move the new reproduction test file
  run_test.py        Run tests inside the container

utils/
  docker_container.py Docker repository environment, patch extraction, and evaluation
  litellm_api.py      OpenAI-compatible model API wrapper

evaluate_*.py        Code patch, test patch, and agreement evaluation
run_agent.py         PatchTwin / PatchTwin + Guard execution entry point

Use Cases

PatchTwin is useful for:

• Studying whether coding agents can jointly perform bug fixing and test construction.
• Comparing PatchTwin and PatchTwin + Guard on final success rates and false-agreement behavior.
• Analyzing agent edit decisions across intermediate code/test states.
• Generating SWE-bench / SWT-Bench-style prediction files for official evaluation.

Quick Links

• Running and configuration: RUNNING.md
• Model configuration examples: config/
• Main execution entry point: run_agent.py

Acknowledgements

PatchTwin builds on ideas and tooling from the broader agentic software engineering ecosystem. We thank the OpenHands/OpenHands and aorwall/moatless-tools projects for their open-source contributions.