MA-AGIQA：Large Multi-modality Model Assiste AI-Generated Image Quality Assessment

This repository is the official PyTorch implementation of MA-AGIQA:Large Multi-modality Model Assisted AI-Generated Image Quality Assessment.

Network Architecture

Requirements

Our experiments are based on two conda environment. One for Semantic Feature Extraction and another for Train and Test.

git clone https://github.com/Q-Future/Q-Align.git
cd Q-Align
pip install -e .

git clone https://github.com/wangpuyi/MA-AGIQA.git
cd MA-AGIQA
pip install -r requirements.txt

Usage

Semantic Feature Extraction

We use official mPLUG-Owl2 to extract semantic features. The feature extraction codes are based on Q-Align, great thanks to them!

First, download and transfer root to Q-Align (You should download their repository as said in Requirements.)

cd Q-Align

then

• put json files containing information of training data to Q-Align/playground/data/test_jsons like Q-Align/playground/data/test_jsons/AGIQA_3k.json.
• put getFeature.py like Q-Align/q_align/evaluate/getFeature.py.

You can find them under q_align file in this repository and get semantic feature by

python "q_align/evaluate/getFeature.py"

if you have error when connect to Hugging Face, we recommand you use

HF_ENDPOINT=https://hf-mirror.com python "q_align/evaluate/getFeature.py"

Train and Test

Download and transfer root to MA-AGIQA. If you've download this repository, just implement the "cd" code.

cd MA-AGIQA

Train and Test

python train.py

Performance

TODO

• release the checkpoints
• simplify codes for friendly usage

Citation

If you find our code or model useful for your research, please cite:

@misc{wang2024large,
      title={Large Multi-modality Model Assisted AI-Generated Image Quality Assessment}, 
      author={Puyi Wang and Wei Sun and Zicheng Zhang and Jun Jia and Yanwei Jiang and Zhichao Zhang and Xiongkuo Min and Guangtao Zhai},
      year={2024},
      eprint={2404.17762},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}

Acknowledgement

Part of our code are based on MANIQA and Q-Align. Thanks for their awesome work!