Diffusion LLMs For Code Generation

Offical implementation of our paper "Exploring the Potential of Diffusion Large Language Models in Code Generation".

📰 News

• [Sep, 2025]: We've released the code for our paper "Exploring the Potential of Diffusion Large Language Models in Code Generation"

👋 Overview

Our work is an empirical study on exploring the potential of Diffusion LLMs in Code Generation

As an emerging model architecture, Diffusion LLMs have achieved performance levels comparable to those of Autoregressive LLMs, and in some cases, even surpass them.

🚀 Environment Setup

To conduct tests on HumanEval and MBPP, execute the following command:

pip install -r requirements.txt

For testing on LiveCodeBench, first configure the environment as specified in its official repository. Then, within the established Python environment, run:

pip install -r requirements_LiveCodeBench.txt

For RepoQA, first set up the environment following the instructions in its official repository. Subsequently, replace the file repoqa/provider/hf.py (from the official RepoQA repository) with the counterpart provided in our repository, located at RepoQA/utils/hf.py.

Please note that conflicts may arise between different environments. Plase create separate, independent environments for each benchmark. You may refer to Conda (a popular package/environment manager) or UV (a fast Python package/environment tool) to streamline environment isolation.

💽 Usage

For HumanEval, MBPP, and LiveCodeBench, simply locate the model you wish to run and execute the corresponding command:

python HumanEval/humaneval_dreamCoder.py  # For HumanEval
python mbpp_mercury.py  # For MBPP
python LiveCodeBench/Livecb_llada_1_5.py  # For LiveCodeBench

The hyperparameters are specified in the file, you may locate and modify them as needed.

For RepoQA, testing Diffusion LLMs follows the same procedure as testing Autoregressive LLMs. Specifically:

# For Hugging Face Models
repoqa.search_needle_function --model "Dream-org/Dream-Coder-v0-Instruct-7B" --backend hf --code-context-size 2048 --trust-remote-code

# For Models with Available API
OPENAI_BASE_URL=https://api.inceptionlabs.ai/v1 \ 
OPENAI_API_KEY=<your_api_key> \
repoqa.search_needle_function --model "merucry-coder" --backend openai --code-context-size 16384

We also present all results generated by the diffusion LLMs for Research Question 1 in the results/ directory.