LiST: Local-Simplex Test-Time LoRA Fusion (Accepted by Findings of Conference on Empirical Methods in Natural Language Processing 2026)

English | 中文

Structure

LiST/
├── checkpoints/  # required LoRA checkpoints
└── src/
    ├── deepseek_7b/
    ├── qwen3_8b/
    ├── qwen3_vl_8b/
    └── llava_v1_5_7b/
        ├── model/   # LiST
        ├── config/  # config.yaml
        ├── eval/    # validation code
        └── tools/   # offline construction and utilities

Installation

conda create -n LiST python=3.10
conda activate LiST
pip install -r requirements.txt

Backbones

Backbone	Link
DeepSeek-7B	DeepSeek 7B
Qwen3-8B	Qwen3 8B
Qwen3-VL-8B	Qwen3-VL 8B
LLaVA-v1.5-7B	LLaVA-v1.5 7B

Configuration

src/deepseek_7b/config/config.yaml
src/qwen3_8b/config/config.yaml
src/qwen3_vl_8b/config/config.yaml
src/llava_v1_5_7b/config/config.yaml

LoRA Checkpoints

Place the required task-specific LoRA checkpoints under checkpoints/, following the paths.checkpoints_root value in each model configuration:

checkpoints/
├── deepseek_7b/output_<task>/
├── qwen3_8b/output_<task>_qwen3_8b/
├── qwen3_vl_8b/output_<task>/
└── llava_v1_5_7b/output_<task>/

Each task directory must contain adapter_config.json and adapter_model.safetensors (or adapter_model.bin), either directly or inside a checkpoint-* subdirectory.

Offline Artifacts

python src/qwen3_8b/tools/generate_task_vectors.py
python src/qwen3_8b/tools/build_offline_artifacts.py

Replace qwen3_8b with the corresponding model directory for another backbone.

Labeled Validation Sets

python src/deepseek_7b/eval/scripts/run_tta_eval.py --split validation --task sst2
python src/qwen3_8b/eval/scripts/run_tta_eval.py --split validation --task mrpc
python src/qwen3_vl_8b/eval/scripts/run_tta_eval.py --split validation --task scienceqa
python src/llava_v1_5_7b/eval/scripts/run_tta_eval.py --split validation --task vqav2

Use --all-tasks to evaluate all configured tasks.

References

• Pinned RobustMerge revision
• Official LLaVA repository
• LLaVA-v1.5
• RobustMerge

@inproceedings{liu2024improved,
  title={Improved Baselines with Visual Instruction Tuning},
  author={Liu, Haotian and Li, Chunyuan and Li, Yuheng and Lee, Yong Jae},
  booktitle={CVPR},
  year={2024}
}

@inproceedings{zeng2025robustmerge,
  title={RobustMerge: Parameter-Efficient Model Merging for MLLMs with Direction Robustness},
  author={Zeng, Fanhu and Guo, Haiyang and Zhu, Fei and Shen, Li and Tang, Hao},
  booktitle={NeurIPS},
  year={2025}
}