RoRA: Role-Oriented Regional Allocation for Visual Token Pruning in MLLMs

This repository contains the official implementation of RoRA, a training-free visual token pruning method for multimodal large language models (MLLMs).

RoRA performs pruning during inference by allocating the retained visual-token budget across complementary roles:

• a semantic core selected from debiased attention;
• attention-anchored regional context;
• optional detail-repair tokens for fine-grained evidence.

The current release includes the LLaVA/LLaVA-NeXT implementation, utilities for building RoRA attention priors, POPE evaluation helpers, and the Qwen2.5-VL patch used in our experiments.

News

• 2026-08-07: Initial public release.

Installation

git clone https://github.com/LukieLuu/RoRA.git
cd RoRA
conda create -n rora python=3.10 -y
conda activate rora
pip install -r requirements.txt
pip install -e .

Install PyTorch according to your CUDA version before running large-model experiments. This repository does not include model weights or datasets.

Repository Structure

rora/                       # RoRA pruning patch and graph utilities
llava/                          # LLaVA-compatible inference modules
qwen_patch/                     # Qwen2.5-VL RoRA patch
tools/rora/                 # prior construction tools
scripts/eval_pope_yesno_f1.py   # POPE yes/no accuracy and F1 evaluation
scripts/generate_pope_llava_next.py
run_llava_rora.py   # LLaVA VQA generation entry point

Build a RoRA Attention Prior

RoRA can use a positional attention prior. The script below builds an average attention prior from a calibration JSONL file.

python tools/rora/build_rora_attention_prior.py \
  --model-path /path/to/llava-v1.5-7b \
  --image-list /path/to/calibration.jsonl \
  --image-root /path/to/images \
  --out-dir outputs/rora_prior

The resulting prior can be passed to RoRA with --rora-prior-path.

Run LLaVA / LLaVA-1.5

python run_llava_rora.py \
  --model-path /path/to/llava-v1.5-7b \
  --question-file /path/to/questions.jsonl \
  --image-folder /path/to/images \
  --answers-file outputs/rora_answers.jsonl \
  --use-rora-pruner \
  --use-graph-attn \
  --use-rora \
  --fast-v-sys-length 35 \
  --fast-v-image-token-length 576 \
  --fast-v-keep-k 192 \
  --fast-v-agg-layer 2 \
  --rora-prior-path outputs/rora_prior/avg_attn/layer_1.pt \
  --rora-core-ratio 0.7 \
  --rora-aar-anchor-count 20 \
  --rora-aar-radius 2 \
  --rora-aar-weight 0.05 \
  --rora-use-graph-mask

For LLaVA-NeXT, use the same patching path with the appropriate visual-token length and model loader. scripts/generate_pope_llava_next.py provides a minimal POPE generation example.

Evaluate POPE

python scripts/eval_pope_yesno_f1.py \
  --question-file /path/to/pope.jsonl \
  --result-file outputs/rora_answers.jsonl \
  --output-json outputs/rora_pope_f1.json

Qwen2.5-VL

The Qwen2.5-VL implementation used in our experiments is provided in qwen_patch/. It is intended to be applied to the corresponding Qwen2.5-VL evaluation environment. In this path, RoRA is selected with pruner_method=rora.

Citation

@article{lu2026rora,
  title={RoRA: Role-Oriented Regional Allocation for Visual Token Pruning in MLLMs},
  author={Lu, Qiyanhui and Wu, Han and Xu, Rongjian and Luo, Tingzhang and Fan, Cheng and Chen, Xinghao and Dong, Minjing and Yang, Jufeng and Guo, Jianyuan},
  journal={arXiv preprint},
  year={2026}
}

Acknowledgements

This codebase builds on the LLaVA ecosystem and follows the public release style of recent training-free MLLM token-pruning projects. We thank the authors of these projects for their open-source efforts.

License

This project is released under the Apache-2.0 license.