Training-free and Zero-shot Regeneration for Hallucination Mitigation in MLLMs: Representation Understanding Perspective Training-free and Zero-shot Regeneration for Hallucination Mitigation

---

Hallucinations in multimodal large language models (MLLMs) are urgent problems to be solved in the new era of artificial general intelligence (AGI). Compared with traditional large language models (LLMs), besides handling language understanding and modeling, we also need to consider the detection and position determination of objects in vision. Therefore, to tackle the hallucination issues, the existing studies attempt to employ few-shot learning on the following perspectives: 1) limit the length of the generated response, 2) iteratively generate multiple candidates or select from multiple candidates via beam search, 3) locally edit the possible parts of primary response, and 4) leverage external knowledge to augment the generation capability. To address the above potential weaknesses, this paper proposes a multimodal training-free and zero-shot regeneration approach by obatain various multimodal evidences and globally improving the raw response to alleviate hallucinations in MLLMs (Mtzr). Specifically, we first extract the entity-level evidences by object-based pre-trained models with in-context learning. Then, we mine the attribute-level evidences inside each entity and cross different entities with heterogeneous in-context learning based on both uni- and multimodal pre-trained models. Finally, towards the obtained multimodal evidences, we regenerate the response with augmented context by residually connecting both the input text and image. For better understanding, we provide theoretical explanations with universal approximation to support why our approach can bring about smaller hallucination. Detailed experimental results and extensive analysis demonstrate that our approach is very suitable for mitigating hallucination in MLLMs.

POPE Result

This part focuses on object-level hallucinations.

MME Result

This part focuses on both object- and attribute-level hallucinations.

Core Model File Descriptions

1. util.py

This file provides a set of general utility functions to support various data processing and computing operations in the project.

• compute_iou: Calculates the Intersection over Union (IoU) of two bounding boxes, which is used to measure the overlap degree between bounding boxes.
• extract_boxes: Extracts bounding box coordinates from text by matching specific formatted bounding box data with regular expressions.
• find_matching_boxes: Identifies entities and their corresponding bounding box information that match the extracted bounding boxes.
• annotate: Annotates images by drawing bounding boxes and adding labels on the image based on the given bounding boxes and entity phrases.

2. answerer.py

Responsible for generating answers to questions, especially those involving positional relationships between entities.

• get_res: Invokes the OpenAI GPT-3.5-turbo model to generate results related to the positional relationships between entities according to the given prompt template.
• get_answer: Processes images and questions using the BLIP2 model to generate answers.
• get_single_answer: Handles questions involving a single entity, generates corresponding answers, and stores them.
• get_multiple_answer: Addresses questions involving multiple entities, particularly positional relationship problems, generates answers, and performs subsequent processing.
• process_position: Processes positional relationship results, determines the actual positional relationship between entities, and returns "Yes" or "No".
• Answerer Class: Initializes model processors and models, and provides methods for generating answers related to single-entity and multi-entity questions.

3. entity_extractor.py

Extracts entities from sentences, laying the foundation for subsequent tasks such as object detection and question generation.

• get_res: Calls the OpenAI GPT-3.5-turbo model to extract entities from sentences based on prompt templates.
• EntityExtractor Class: Initializes OpenAI-related configurations, provides methods for entity extraction, and stores the extracted entities in samples.

4. questioner.py

Generates fact-verification questions based on sentences and entities, covering aspects such as entity attributes and positional relationships.

• remove_duplicates: Eliminates duplicate questions to ensure the uniqueness of generated questions.
• get_res_template: Generates questions according to specified templates (positional relationship or basic attribute templates).
• get_res: Generates questions by combining positional relationship and basic attribute templates, followed by filtering and deduplication.
• Questioner Class: Initializes OpenAI-related configurations and spaCy models, generates questions for sentences and entities, and classifies them into single-entity questions and multi-entity questions.

5. detector.py

Performs object detection based on the GroundingDINO model to identify entities in images and obtain their related information.

• in_dict: Determines whether a bounding box already exists in the entity dictionary to avoid duplicate annotations.
• extract_detection: Extracts entity information from detection results, including bounding boxes, cropped image paths, etc., and updates the entity dictionary.
• find_most_similar_strings: Uses spaCy models to find the most similar entities to the detected phrases, improving the accuracy of entity matching.
• Detector Class: Initializes the GroundingDINO model and spaCy models, provides object detection methods, and stores the detected entity information in samples.

6. claim_generator.py

Generates claims based on questions and answers, integrating entity counting information, attribute information, and other related content.

• format_inputs: Formats questions and answers to prepare for claim generation.
• get_claim: Converts questions and answers into claims using a pre-trained seq2seq model.
• get_single_claim_string and get_multiple_claim_string: Generate claim strings related to single entities and multiple entities, respectively.
• ClaimGenerator Class: Initializes models and tokenizers, and provides methods for claim generation, including single-entity claims and multi-entity claims.

Preliminary

1. Create conda environment

conda create -n mtzr python=3.10
conda activate mtzr
pip install -r requirements.txt

2. Install required packages and models

• Install spacy and relevant model packages, following the instructions in Link. This is used for some text processing operations.

pip install -U spacy
python -m spacy download en_core_web_lg
python -m spacy download en_core_web_md
python -m spacy download en_core_web_sm

3. Install GroundingDINO required packages

• Install GroundingDINO following the instructions in Link.

Usage

1. Inference

Run the following inference code to correct the MLLM output:

for task in  mme_existence mme_count mme_position mme_color pope_adversarial pope_popular pope_random
do
for mode in entity_extract detecte question answer
do
    CUDA_VISIBLE_DEVICES=1 python all_claim_generate.py \
        --task $task \
        --model_name minigpt \
        --device 0 \
        --seed 13 \
        --method mtzr \
        --mode $mode \
        --rewrite
done
done

Acknowledgement

This repository benefits from Woodpecker, mPLUG-Owl, GroundingDINO, BLIP-2, and LLaMA-Adapter. Thanks for their awesome works.