Global-Attention-fused-Image-Cropping

This is the official repository for the following paper:

Global Attention-fused Image Cropping with Attention-guided and Global-aligned Crop Evaluator

Haotian Yang, Zhile Yang, Kin-Man Lam, Patrick Le Callet and Xin Sun

Our study focuses on image cropping which aims to improve the aesthetic quality of an image by capturing important content. To this end, we propose Global Attention-Fused Image Cropping (GAFIC), a novel framework, to address the issues of coarse attention distribution and poor perception of regions near cropping boundaries during feature representation. As illustrated in the figure below, the proposed framework is composed of two key components: Attention-Guided Feature Fusion (AGFF) and Global-Aligned Crop Evaluator (GACE). AGFF generates one global feature to represent the importance of each local region. GACE aligns and fuses the features of the cropping regions with the global feature to make it sensitive to the boundary of the cropping box.

Results

In the below figure, we present the existing state-of-the-art methods side by side with our method, which demonstrates that our method outperforms other existing methods in terms of expression of the importance of different local regions and perception ability of the boundary areas of the cropping boxes. For images with clear objects in columns (A), (D), and (E), our results on different types of images are close to the true values. For example, foot movement is truncated by VEN [36] when a person is in the center of the image (A), while our method retains the details of the foot movement while highlighting the object.

Usage

Here we not only release the code of our method, but also provide the selected human-centric samples in these frequently used image cropping datasets, as well as their human bounding boxes under the folder human_bboxes.

1. Download the source code and related image cropping datasets including CPC, GAICD datasets. CPC GAICD.

2. Change the pathes to above datasets and annotation files in config_GAICD.py and config_CPC.py.

3. Run generate_heatmap.py to generate pseudo heatmaps for GAICD or CPC dataset.

4. Run train_GAFIC.py to train a new model on GAICD dataset (resp., CPC dataset).

Requirements

Please see requirement.txt.