RIFT: Group-Relative RL Fine-Tuning for Realistic and Controllable Traffic Simulation

Left: AV View (SparseDrive as AV)   Right: Scenario View (RIFT as CBV)

✨ News

• 2025-07-22 Support for E2E AV is available now, with doc📄!
• 2025-05-07 Explore our project page, now live here🔗!
• 2025-05-07 Our paper is available on arXiv📄!

This repository contains the implementation of the paper.

RIFT: Group-Relative RL Fine-Tuning for Realistic and Controllable Traffic Simulation

Keyu Chen¹ , Wenchao Sun¹, Hao Cheng¹, Sifa Zheng¹

¹School of Vehicle and Mobility, Tsinghua University

If you find our work useful, Please give us a star 🌟!

💫 RIFT achieve realistic and controllable traffic simulation by combining IL pre-training in a data-driven simulator for realism with RL fine-tuning in a physics-based simulator for controllability.

Method

TODO List

• full model checkpoint
• E2E AV (UniAD, VAD, SparseDrive)
• training code
• initial repo & paper

Outline

• Setup
• Data and Ckpt
• Usage
  • Fine Tune CBV Policy
  • Evaluation
  • Visualization

• Citation

• Acknowledgement

Setup

Recommended system: Ubuntu 20.04 or 22.04

Step 1: Install Carla

mkdir carla
cd carla
wget https://carla-releases.s3.us-east-005.backblazeb2.com/Linux/CARLA_0.9.15.tar.gz
tar -xvf CARLA_0.9.15.tar.gz
cd Import && wget https://carla-releases.s3.us-east-005.backblazeb2.com/Linux/AdditionalMaps_0.9.15.tar.gz
cd .. && bash ImportAssets.sh

Update your system's PYTHONPATH with the following paths:

export CARLA_ROOT=YOUR_CARLA_PATH
export PYTHONPATH=$PYTHONPATH:${CARLA_ROOT}/PythonAPI
export PYTHONPATH=$PYTHONPATH:${CARLA_ROOT}/PythonAPI/carla

Step 2: Setup conda environment

conda create -n rift python=3.8
conda activate rift
echo "$CARLA_ROOT/PythonAPI/carla/dist/carla-0.9.15-py3.7-linux-x86_64.egg" >> YOUR_CONDA_PATH/envs/rift/lib/python3.8/site-packages/carla.pth # python 3.8 also works well, please set YOUR_CONDA_PATH

Step 3: Clone this git repo in an appropriate folder

git clone git@github.com:CurryChen77/RIFT.git
cd RIFT

Step 4: Install main packages

pip install -r requirements.txt
pip install -e .

Step 5: Install E2E packages

Follow the documentation here to install E2E-related packages.

Data and Ckpt

• Carla Map Data

Doc of HD Map

Name	Google Drive	Approx. Size	Storage Place
HD Map Data	Link	714 MB	Folder
Speed Limits Data	Link	79 MB	Folder

• AV Checkpoint

AV Name	Google Drive	Approx. Size	Storage Place
PlanT_medium	Link	695 MB	Folder

For the E2E AV Checkpoint, please check the documentation.

• CBV Checkpoint

Name	Google Drive	Approx. Size	Storage Place
Pluto & Fine-Tuned Variants	Link	166.2 MB	Folder

Usage

Fine Tune CBV Policy

Run with Python

Parallel execution of Python scripts is allowed as long as each script is assigned a different CUDA_VISIBLE_DEVICES.

# Train rift pluto
CUDA_VISIBLE_DEVICES=0 python scripts/run.py --ego_cfg pdm_lite.yaml --cbv_cfg rift_pluto.yaml --mode train_cbv

Run with Bash (with Resume)

Multiple Runs for Resume (Carla will crash for some reasons in Fine-tuning)

bash scripts/run_multi.sh \
  -t 3 \                         # max try
  -e pdm_lite.yaml \             # ego file
  -c rift_pluto.yaml \           # cbv file
  -m train_cbv \                 # run mode (train_cbv, train_ego, eval, collect_data)
  -r 2 \                         # scenario repeat time
  -s 0 \                         # random seed
  -g 0 \                         # GPU_ID
  -v                             # render or not

Evaluation

Evaluating for specific CBV and AV

# Eval rift pluto
CUDA_VISIBLE_DEVICES=0 python scripts/run.py --ego_cfg pdm_lite.yaml --cbv_cfg rift_pluto.yaml --mode eval -rep 1

Plot Evaluation Result

• Plot Speed and Acceleration Distribution

python tools/plot/plot_distribution.py

• Plot Evaluation Results

python tools/plot/plot_eval_results.py

Evaluation results of the paper are provided here.

Visualization

# Eval rift pluto
CUDA_VISIBLE_DEVICES=0 python scripts/run.py --ego_cfg pdm_lite.yaml --cbv_cfg rift_pluto.yaml --mode eval --render -rep 1  # add --render

Citation

If you find our paper useful, please kindly cite us via:

@misc{chen2025riftgrouprelativerlfinetuning,
      title={RIFT: Group-Relative RL Fine-Tuning for Realistic and Controllable Traffic Simulation}, 
      author={Keyu Chen and Wenchao Sun and Hao Cheng and Sifa Zheng},
      year={2025},
      eprint={2505.03344},
      archivePrefix={arXiv},
      primaryClass={cs.RO},
      url={https://arxiv.org/abs/2505.03344}, 
}

Acknowledgement

This implementation is based on code from several repositories. We sincerely thank the authors for their awesome work.

• Pluto
• FREA
• Bench2Drive
• DriveLM-CARLA
• PlanT
• nuPlan