PSearch

This is the official implementation of PSearch: Search-based Patch Generation in the Era of LLM-based Automated Program Repair, Accepted to IEEE/ACM International Conference on Automated Software Engineering (ASE 2026).

Requirements

• Gradle: 8.11.1

wget https://services.gradle.org/distributions/gradle-8.11.1-bin.zip
unzip gradle-8.11.1-bin.zip
sudo vim ~/.bashrc
# add following line to .bashrc
export GRADLE_HOME=/root/autodl-tmp/gradle-8.11.1
export PATH=${PATH}:${GRADLE_HOME}/bin

source ~/.bashrc
# check gradle version
gradle -v

• HuggingFace

pip install -U huggingface_hub
export HF_ENDPOINT=https://hf-mirror.com
huggingface-cli download --token $HF_TOKEN --resume-download $MODEL_NAME --local-dir $LOCAL_DIR

• Defects4j

git clone https://github.com/rjust/defects4j
cd defects4j
cpanm --installdeps .
./init.sh
export PATH=$PATH:"path2defects4j"/framework/bin
# results under ./data

• SWE-bench

git clone git@github.com:princeton-nlp/SWE-bench.git
cd SWE-bench
pip install -e .
# results under ./SWE

• ConDefects

  • paper
  • code
  • git clone & replace the original files with ConDefects_Replace to support multi-threaded testing.

• VUL4J

  • repository
  • results under ./VUL4J

• other artifacts needed to install

  • less
  • expect
  • svn

Run

• Defects4J (PSearch)

python mcts.py \
    --policy_model gpt-4o-mini \
    --max_rollout 16 \
    --max_expansion 3 \
    --branch 1 \
    --exploration_constant 0.7 \
    --logger logs/mcts.log \
    --output_file results/result.jsonl

• Defects4J (Beam Search + Patch Pool)

python beam_search.py \
    --policy_model qwen2.5-coder-32b \
    --reward_model qwen2.5-coder-32b \
    --output_path ./data/beam_search_results.jsonl \
    --plausible_save_path ./plausible \
    --beam_width 5 \
    --max_tokens 16000 \
    --max_iter 4 \
    --pool_size 4 \

• QuixBugs

Python quixbugs_repair.py

• ConDefects

python condefects_mcts.py \
    --policy_model gpt-4o-mini \
    --max_rollout 32 \
    --max_expansion 3 \
    --branch 10 \
    --exploration_constant 0.7 \
    --program_id xxx \
    --task_id xxx\
    --mask_mode unmasked \
    --logger logs/mcts.log \
    --output_file results/condefects_result.jsonl

• SWE-bench lite

# prepare data
python swe_utils.py
# run swe
python swe_mcts.py

• VUL4J

bash scripts/vul_mcts.sh

Results

On Defects4J and QuixBugs

Category	Method	Model	Patch Size	Defects4J-v1.2	Defects4J-v2	Total	QuixBugs
APR	SelfAPR	T5	150	65/74	45/47	110/121	-
APR	ITER	T5	1000	59/89	19/36	78/125	-
APR	CURE	GPT-2	5000	57/-	19/-	76/-	26
APR	RAPGen	CodeT5	<=100	72/-	53/-	125/-	-
APR	RewardRepair	Transformer	200	45/-	45/-	90/-	20
APR	Recoder	TreeGen	100	53/-	19/-	72/-	31
APR	Repatt	NA	1200	40/70	35/68	75/138	-
APR	ChatRepair	GPT-3.5	<=500	114/-	48/-	162/-	40
APR	RepairAgent	GPT-3.5	117	74/96	90/90	164/186	-
APR	GiantRepair	GPT-3.5	<=200	86/-	84/-	170/-	-
APR	D4C	GPT-4	10	84/96	96/114	180/210	-
LLM	Stable-Code-3B	Stable-Code-3B	16	31/49	27/50	58/99	20
LLM	Calme-3.1-3B	Calme-3.1-3B	16	25/44	20/42	45/86	19
LLM	Starcoder2-3B	Starcoder2-3B	16	19/35	24/44	43/79	18
LLM	Qwen2.5-Coder-3B	Qwen2.5-Coder-3B	16	44/68	43/70	87/138	27
LLM	Llama-3.2-3B	Llama-3.2-3B	16	32/53	27/42	59/95	21
LLM	Phi-3.5-mini	Phi-3.5-mini	16	28/52	29/53	57/105	19
LLM	DeciLM-7B	DeciLM-7B	16	23/42	22/41	45/83	19
LLM	Falcon-7B	Falcon-7B	16	8/21	10/25	18/46	4
LLM	Yi-Coder-9B	Yi-Coder-9B	16	48/73	58/93	106/166	31
LLM	Llama-3.1-8B	Llama-3.1-8B	16	43/71	43/68	86/139	25
LLM	Qwen2.5-Coder-7B	Qwen2.5-Coder-7B	16	38/66	41/70	79/132	25
LLM	GPT-4o-mini	GPT-4o-mini	16	67/89	61/81	128/170	35
LLM	GPT-3.5	GPT-3.5	16	69/92	63/84	132/176	36
Ours	PSearch (Ours)	GPT-3.5	16	86/112	73/104	159/216	40
Ours	PSearch (Ours)	GPT-3.5	32	106/146	95/134	201/280	40

On ConDefects

ChatRepair	GPT-3.5	AlphaRepair	PSearch (48 patch)	PSearch (96 patch)
241/249	165/171	142/160	204/211	264/287

On Vul4J

Metric	PSearch-Vul (Ours)	FSV-Codex	FSV-finetuned	NTR	VRPILOT	APR4Vul	ChatRepair
CF	27/79	10.9/79	9/79	14/79	14/79	16/79	15/79
RCR	34.17%	13.79%	11.39%	17.72%	17.72%	20.25%	18.98%

On SWE-bench Lite

SWE System	Base Model	Resolved	%Resolved	Date
Refact.ai Agent	NA	180	60%	2025-06-25
SWE-agent	Claude-4 Sonnet	170	56.67%	2025-05-26
PSearch (Ours)	Qwen3-Coder-480B	164	54.67%	2026-03-22
KGCompass	Claude-3.5 Sonnet	138	46%	2025-06-19
ChatRepair	Qwen3-Coder-480B	129	43%	2026-03-22
OpenHands	Claude-3.5 Sonnet	125	41.67%	2024-10-25
Vanilla LLMs	Qwen3-Coder-480B	113	37.67%	2026-03-22

Cite

@misc{hu2026psearchsearchbasedpatchgeneration,
      title={PSearch: Search-based Patch Generation in the Era of LLM-based Automated Program Repair}, 
      author={Haichuan Hu and Ye Shang and Weifeng Sun and Quanjun Zhang},
      year={2026},
      eprint={2507.01827},
      archivePrefix={arXiv},
      primaryClass={cs.SE},
      url={https://arxiv.org/abs/2507.01827}, 
}