SHIELD

[ICLR 2026🔥] SHIELD: Suppressing Hallucinations In LVLM Encoders via Bias and Vulnerability Defense

This is the official implementation for SHIELD: Suppressing Hallucinations In LVLM Encoders via Bias and Vulnerability Defense

by Yiyang Huang, Liang Shi, Yitian Zhang, Yi Xu, Yun Fu.

Status: Code will be released soon.

SHIELD is a training-free framework that mitigates hallucinations in Large Vision-Language Models (LVLMs) by tracing the issue back to visual encoders and addressing three factors: statistical bias, inherent bias, and vulnerability. It achieves strong performance across multiple hallucination benchmarks (CHAIR, POPE, MME, AMBER), demonstrating its effectiveness for reliable vision-language understanding.

Table of contents

• Getting Started
  • Installation
  • Data Preparation
• Inference and Evaluation
  • Output Structures
• Acknowledgement
• Citations

Getting Started

Installation

• The code is developed with Python >= 3.10, PyTorch >= 2.1.0

  0. [Optional but recommended] Create a new conda environment.

     conda create -n shield python=3.10
     

     And activate the environment.

     conda activate shield
     

  1. Install the requirements.

     pip install -r requirements.txt
     

Note: requirements.txt will be finalized upon code release.

Data Preparation

We will provide detailed instructions for downloading and organizing evaluation datasets upon release.

Evaluation benchmarks used in the paper:

• CHAIR
• POPE (COCO / A-OKVQA / GQA)
• MME
• AMBER

Inference and Evaluation

SHIELD is a training-free method, so we can directly do the inference and evaluation without model training.

# Code will be released soon.
# python scripts/run_eval.py --config configs/xxx.yaml

Output Structures

• The inference outputs will be stored under outputs/predictions/.
• The evaluation results will be stored under outputs/logs/.
• All paths can be changed in the config file.

Acknowledgement

We extend our gratitude to the following awesome projects: LLaVA, VCD, OPERA, and Qwen-VL.

Citations

If you find this work useful, please cite our paper:

@inproceedings{huang2026shield,
    title     = {SHIELD: Suppressing Hallucinations In LVLM Encoders via Bias and Vulnerability Defense},
    author    = {Huang, Yiyang and Shi, Liang and Zhang, Yitian and Xu, Yi and Fu, Yun},
    booktitle = {Proceedings of the International Conference on Learning Representations (ICLR)},
    year      = {2026},
}

arXiv version:

@misc{huang2025shield,
    title         = {SHIELD: Suppressing Hallucinations In LVLM Encoders via Bias and Vulnerability Defense},
    author        = {Huang, Yiyang and Shi, Liang and Zhang, Yitian and Xu, Yi and Fu, Yun},
    year          = {2025},
    eprint        = {2510.16596},
    archivePrefix = {arXiv},
    primaryClass  = {cs.CV},
    url           = {https://arxiv.org/abs/2510.16596}
}