A Channel-Spatial Hybrid Attention Mechanism using Channel Weight Transfer Strategy

Authors: Jie Pan, Haigen Hu, Aizhu Liu, Qianwei Zhou, Qiu Guan

Abstract

Attention is one of the most valuable breakthroughs in the deep learning community, and how to effectively utilize the attention information of channel and spatial is still one of the hot research topics. In this work, we integrate the advantages of channel and spatial mechanism to propose a Channel-Spatial hybrid Attention Module (CSHAM). Specifically, max-average fusion Channel Attention Module and Spatial Attention Neighbor Enhancement Module are firstly proposed, respectively. Then the connection between the two modules is analyzed and designed, and an alternate connection strategy with the transformation of channel weights is proposed. The key idea is to repeatedly use the channel weight information generated by the channel attention module, and to reduce the negative impact of the network complexity caused by the addition of the attention mechanism. Finally, a series of comparison experiments are conducted on CIFAR100 and Caltech-101 based on various backbone models. The results show that the proposed methods can obtain the best Top-1 performance among the existing popular methods, and can rise by nearly 1% in accuracy while basically maintaining the parameters and FLOPs.

Requirements

Please make sure your Python version>=3.6. Use pip or conda to install those Packages:

tensorflow-gpu>=2.0.3
tqdm
pandas
glob3
matplotlib
numpy
tqdm

CNNs for Caltech101 classification

We provide some CNNs not used in the paper. If necessary, you can use them yourself.

• VGG16
• MobileNetV2
• ResNet
• ResNext
• LeNet
• DenseNet

Prepare datasets

You need download Caltech101 dataset and unzip it,and make sure the folder architecture as follows:

./CSHAM
├── assets
├── attention
├── callbacks
├── data
│   ├── Caltech101
├── models
├── my_scripts
├── nets
├── Results
└── trainer

Train model

If you want to trainmodel,you need enter a training folder directory,then run train.py.If you want to train this model from scratch,please delete checkpoints and log folders first,then run train.py All log files and training results will be saved in the Results folder.

Use GPU

• CUDA 8.0.61
• CUDNN 5.1

# Chose GPU to use
$ CUDA_VISIBLE_DEVICES=0 python train.py (optional patameters...)

Reference

Please cite our paper if you find the work useful:

@INPROCEEDINGS{CS_Attention_2022,
  author={Pan, Jie and Hu, Haigen and Liu, Aizhu and Zhou, Qianwei and Guan, Qiu},
  booktitle={2022 26th International Conference on Pattern Recognition (ICPR)}, 
  title={A Channel-Spatial Hybrid Attention Mechanism using Channel Weight Transfer Strategy}, 
  year={2022},
  pages={2524-2531},
  doi={10.1109/ICPR56361.2022.9956146}}