Healer

This repository contains the code for the Healer paper.

Requirements

pip install openai

Fill your OpenAI API key in the responser.py file.

Data

The instruction-tuning dataset is available here The four benchmarks used in the paper are available here

Usage

The scripts to run the experiments on each benchmark are available in the scripts folder. The parameters are introduced in the following:

• --handler: the LLM to use. Options: gpt4, gpt35, qwen
• --max_attempt: the maximum times to invoke the LLM for one code snippet
• --benchmark: the benchmark to run the experiment on. Options: debugbench, codenet, humaneval_{MODEL}, mbpp_{MODEL} where {MODEL} is the model name including gpt4, gpt35, codellama, mistral, starcoder, codet5p.
• --save_dir: the directory to save the results