• 2025.06.15 🌟 We are proud to introduce BridgeVLA, a model designed to bridge the gap between VLM backbone and VLA by aligning input and output within a shared 2D space during both pre-training and fine-tuning.

• Model Overview
• Installation
• Training
• Evaluation
• Experimental Results
• TODO
• Acknowledgement
• Contact
• Citation

As illustrated in the following figure, BridgeVLA employs a dual-phase training recipe. During pre-training, it is trained to predict 2D heatmaps on object detection datasets. During fine-tuning, point clouds are projected into multiple 2D images as inputs to the VLM backbone. The model is trained to predict 2D heatmaps for estimating the translational action and other action components. This design aligns the input and output within a shared 2D space in both pre-training and fine-tuning.

1. Clone this repository and navigate to the BridgeVLA folder:

git clone https://github.com/BridgeVLA/BridgeVLA.git
cd BridgeVLA

2. Install the required package:

conda create -n bridgevla python=3.9 -y
conda activate bridgevla

# For pre-training
cd pretrain
bash ./pretrain_install.sh

# For RLBench fine-tuning
cd finetune/RLBench
bash ./install_rlbench.sh

# For Colosseum fine-tuning
cd finetune/Colosseum
bash ./install_colosseum.sh

# For GemBench fine-tuning
cd finetune/GemBench
bash ./install_gembench.sh

3. Note: To avoid potential conflicts between different simulation benchmarks, we suggest creating separate virtual environments for each benchmark. Also, our model is built upon Paligemma, which is a gated repo. Therefore, you should first be authenticated to access it.

If you want to reproduce our results, please use the same training hyperparameters in the config file. Do not forget to modify the corresponding saving path in the file before running the following code.

1. Pre-training: We use the object detection data in the RoboPoint dataset to pre-train the model. We upload the data and checkpoints here. With the pretrain/pretrain.py file, you can do three things:

• visualiztion: This function is used to visualize the pre-training dataset.
• pre-training: This function is used to pre-train the Paligemma model on the dataset .
• evaluation: This function is used to test the pre-trained checkpoints. What you need to do is to modify the checking branch in the file and then run the following code:

cd pretrain
bash pretrain.sh --branches BRANCH_OPTION --config_path PATH_TO_CONFIG_FILE --json_detection_path PATH_TO_DETECTION_JSON --image_folder PATH_TO_IMAGE_FOLDER

1. RLBench Fine-tuning: To finetune on RLBench, you should first prepare the dataset. You can generate the train and test data yourself by following the instructions in PerAct repository. Or you can directly download the dataset we generate to fully reproduce our results here. To improve the data loading speed, we will first convert the raw data into replay buffer. The training code will automatically do that if it does not find the replay buffer. Meanwhile, you can also directly download the replay buffer we preprocess here. After the data is ready, you can use the finetune/RLBench/train.sh file to finetune the model. Please run the following code:

cd finetune/RLBench
bash train.sh --exp_cfg_path  configs/rlbench_config.yaml \
              --exp_note debug \
              --freeze_vision_tower \
              --log_dir PATH_TO_LOG_DIR \
              --load_pretrain \
              --pretrain_path  PATH_TO_PRETRAINED_MODEL 

1. COLOSSEUM Fine-tuning: For COLOSSEUM, we fine-tune the model with the training dataset provided by the COLOSSEUM challenge. Similarly, our training code will first convert the raw data into replay buffer. You can also directly download the replay buffer we preprocess here. Then, you can use the finetune/Colosseum/train.sh file to finetune the model. Please run the following code:

cd finetune/Colosseum
bash train.sh --exp_cfg_path  configs/colosseum_config.yaml \
              --exp_note debug \
              --freeze_vision_tower \
              --log_dir PATH_TO_LOG_DIR \
              --load_pretrain \
              --pretrain_path  PATH_TO_PRETRAINED_MODEL

1. GemBench Fine-tuning: To finetune on GemBench, you should first download the dataset from here. The structure of GemBench is different from RLBench and COLOSSEUM. We did not use replay buffer and did not do demo augmentation. You can use the finetune/GemBench/train.sh file to finetune the model. Please run the following code:

cd finetune/GemBench
bash train.sh --exp_cfg_path  configs/gembench_config.yaml \
              --exp_note debug \
              --freeze_vision_tower \
              --log_dir PATH_TO_LOG_DIR \
              --load_pretrain \
              --pretrain_path  PATH_TO_PRETRAINED_MODEL

1. RLBench Evaluation: To evaluate on RLBench, you can just run the following code:

cd finetune/RLBench
bash eval.sh # Please modify the evaluated tasks and the checkpoint path in the file.

1. COLOSSEUM Evaluation: To evaluate on COLOSSEUM, you should first preprocess the eval data as the original format is not suitable for our data loading. Run the following code to preprocess them. Or you can directly download the cleaned data we have tided from here.

cd finetune/Colosseum
python3   cleanup_script.py   /PATH/TO/COLOSSEUM_EVAL_DATA/

After cleaning the eval dataset, you can run the following code to evaluate the model:

cd finetune/Colosseum
bash eval.sh  VARIATION LOG_NAME MODEL_EPOCH MODEL_FOLDER

COLOSSEUM requires to evaluate on all the variation factors. We provide the Colosseum/cal_statics.py to compute the per task success rate on each variation factor. Just replace the results folder path in the file and run the following code:

cd finetune/Colosseum
python3 cal_statics.py

Note: During the evaluation of Variations 1 and 6, three tasks—“close laptop lid,” “wipe desk,” and “insert onto peg”—occasionally encountered errors in certain evaluation episodes. These issues stem from problems within the evaluation data itself. I have contacted the COLOSSEUM authors, who have confirmed the issue and plan to address it in a future update. In the meantime, I adopted the following workaround: I recorded only the successful trials and repeated the evaluation until I had collected 25 successful runs for each of these tasks. These 25 valid trials were then used to compute the final performance metrics.

3. GemBench Evaluation: To evaluate on GemBench, you should first launch the server. Run the following code:

cd finetune/GemBench
bash run_server.sh  MODEL_EPOCH  MODEL_BASE_PATH

After lanuching the server, you can run the following code to evaluate the model:

cd finetune/GemBench
bash run_client.sh  SEED MODEL_EPOCH

The results are saved as results.json, which record the success status of each trial. We provide the GemBench/cal_results.py to compute the average success rates of each task in each setting for each seed. Just replace the results folder path in the file and run the following code:

cd finetune/GemBench
python3 cal_results.py

BridgeVLA's performance on three simulation benchmarks is shown in the following table:

• Release the pre-training code
• Release the training & evaluation code of RLBench
• Release the training & evaluation code of COLOSSEUM
• Release the training & evaluation code of GemBench
• Release the pre-training data
• Release the checkpoints
• Release a documentation for conducting real-robot experiments

We stand on the shoulders of giants, and our work in developing BridgeVLA has been inspired and empowered by the remarkable open source projects in the field. We would like to extend our heartfelt gratitude to each of these initiatives and their dedicated developers.

• PerAct
• RVT-2
• Palligemma
• RLBench
• GemBench
• COLOSSEUM
• RoboPoint

If you have any questions about the code, please contact peiyan.li@cripac.ia.ac.cn.

@misc{li2025bridgevla,
    title={BridgeVLA: Input-Output Alignment for Efficient 3D Manipulation Learning with Vision-Language Models},
    author={Peiyan Li and Yixiang Chen and Hongtao Wu and Xiao Ma and Xiangnan Wu and Yan Huang and Liang Wang and Tao Kong and Tieniu Tan},
    year={2025},
    eprint={2506.07961},
    archivePrefix={arXiv},
    primaryClass={cs.RO}
}