Unsolvable Problem Detection: Evaluating Trustworthiness of Vision Language Models

🤗 Dataset | 📖 arXiv | GitHub

Atsuyuki Miyai¹  Jingkang Yang²  Jingyang Zhang³  Yifei Ming⁴ 
Qing Yu^1,5  Go Irie⁶  Sharon Yixuan Li⁴  Hai Li³  Ziwei Liu²
Kiyoharu Aizawa¹

¹The University of Tokyo  ²S-Lab, Nanyang Technological University  ³Duke University  ⁴University of Wisconsin-Madison  
⁵LY Corporation  ⁶Tokyo University of Science 

News

2024.03: The short version (4p) of this paper has been accepted by ICLR 2024 R2-FM Workshop.

Introduction

This paper introduces a novel and significant challenge for Vision Language Models (VLMs), termed Unsolvable Problem Detection (UPD). UPD examines the VLM's ability to withhold answers when faced with unsolvable problems in the context of Visual Question Answering (VQA) tasks. UPD encompasses three distinct settings: Absent Answer Detection (AAD), Incompatible Answer Set Detection (IASD), and Incompatible Visual Question Detection (IVQD). To deeply investigate the UPD problem, extensive experiments indicate that most VLMs, including GPT-4V and LLaVA-Next-34B, struggle with our benchmarks to varying extents, highlighting significant room for the improvements. To address UPD, we explore both training-free and training-based solutions, offering new insights into their effectiveness and limitations. We hope our insights, together with future efforts within the proposed UPD settings, will enhance the broader understanding and development of more practical and reliable VLMs.

Requirements

Installation

We mainly follow the LLaVA's environment for the Installation.
For the implementations, we utlize Nvidia A100 GPUs with 80G. We utilize single GPU for the VLMs' inference and two GPUs for instruction tuning.

conda create -n upd_en python=3.10 -y
conda activate upd_en
pip install --upgrade pip  # enable PEP 660 support
pip install -e .
pip install -e ".[train]"
pip install flash-attn --no-build-isolation --no-cache-dir

Data

MM-UPD Bench (Unnecessary)

We provide all benchmarks via huggingface 🤗. All benchmarks are automatically downloaded by running scripts. Please refer to our huggingface Dataset for more details.

Instruction tuning data (Optional)

We instruction tuning data (.json) via this url.
Please download, unzip and put it to ~/data.
As for the images for the instruction tuning data, we used the images for the official LLaVA's instruction tuning. Please download the images from constituting datasets:

• COCO: train2017
• GQA: images
• OCR-VQA: download script, we save all files as .jpg
• TextVQA: train_val_images
• VisualGenome: part1, part2

Also, we provide the checkpoint for the instruction tuning via this url. Please download, unzip and put it to ~/checkpoints.

The overall file structure is as follows:

UPD
|-- checkpoints
    |-- llava-v1.6-vicuna-13b-task-lora
    |-- llava-v1.6-34b-task-lora
|-- data
    |-- inst_tuning
        |-- upd_tuning_data_20240303.json
        ├── coco
        │   └── train2017
        ├── gqa
        │   └── images
        ├── ocr_vqa
        │   └── images
        ├── textvqa
        │   └── train_images
        └── vg
            ├── VG_100K
            └── VG_100K_2

Note that the setting of these files is not necessary if you just want to inference VLMs.

API KEY

We need to create API keys for OpenAI and Gemini. With API keys, we can run the following commands:

export OPENAI_API_KEY='your-api-key-here'

export GEMINI_API_KEY='your-api-key-here'

About the setup of API keys, please refer to the OpenAI official page as for more detail.

Quick Start

1. Inference of VLMs

We put each script in ~/scripts/inference/<VLM name>/<UPD>. For example, to implement LLaVA-1.5 13B for the base setting, you can implement the following commands for each AAD and Standared senario:

base aad

bash scripts/inference/llava1.5/aad/base_aad.sh

By implelenting the above code, the result is automatically created under output/aad/answers_upload/llava1.5/base/mmaad_aad_base/llava1.5-13b_<time_stamp>.xlsx

base standard

bash scripts/inference/llava1.5/aad/base_standard.sh

By implelenting the above code, the result is automatically created under output/aad/answers_upload/llava1.5/base/mmaad_standard_base/llava1.5-13b_<time_stamp>.xlsx

2. Evaluation

We put each evaluation script in ~/scripts/evaluation/<UPD>.

For example, to evaluate the performance of LLaVA-1.5 13B for the base setting, you can implement the following commands:

bash scripts/evaluation/aad/eval_base.sh <UPD_RESULT_PATH> <STANDARD_RESULT_PATH> 

• <UPD_RESULT_PATH> is output/aad/answers_upload/llava1.5/base/mmaad_aad_base/llava1.5-13b_<time_stamp>.xlsx in this example.
• <STANDARD_RESULT_PATH> is output/aad/answers_upload/llava1.5/base/mmaad_standard_base/llava1.5-13b_<time_stamp>.xlsx in this example.

By implelenting the above code, the result is automatically created in each RESULT_PATH folder.

3. Instruction Tuning

We put each script in ~/scripts/inst_tuning. For example, to implement LLaVA-1.6 34B, you can implement the following commands:

bash scripts/inst_tuning/llava1.6_34b_lora_tuning.sh

As of March 2024, LLaVA1.6 has not yet released official LoRA tuning code. Therefore, please be aware that the our instruction tuning code may differ from the official LLaVA implementation.

Model Results

We provide a Google Sheet for the detailed results on each senario (Fig. 3, 4 ,5 ,6 in the paper). You can access the sheet here and can easily draw radar charts!

Acknowledgement

We adopt these codes to create this repository.

• Visual Instruction Tuning, in NeurIPS, 2023.
• OpenCompass
• Otter

Contact

If you have questions, please open an issue mentioning @AtsuMiyai or send an email to miyai[at]hal.t.u-tokyo.ac.jp

Ads

If you are interested in this work, please refer to our other projects.

• OpenOOD, in NeurIPS Dataset and Benchmark Track, 2022
• OpenOOD v1.5 in NeurIPSW, 2023
• LoCoOp, in NeurIPS, 2023.

Citaiton

If you find our work interesting or use our code/models, please consider citing:

@article{miyai2024upd,
  title={Unsolvable Problem Detection: Evaluating Trustworthiness of Vision Language Models},
  author={Miyai, Atsuyuki and Yang, Jingkang and Zhang, Jingyang and Ming, Yifei and Yu, Qing and Irie, Go and Li, Yixuan and Li, Hai and Liu, Ziwei and Aizawa, Kiyoharu},
  journal={arXiv preprint arXiv:2403.20331},
  year={2024}
}