Illumination Adaptive Transformer (IAT) (paper)

2022.7.11: Upload the low-light object detection code. See detection.

What is IAT?

For Vision Tasks under various lighting conditions, towards both Human Vision 😄 and Machine Vision 📷

5 Tasks Under Various Lighting Conditions: 1. Low-light Enhancement (LOL, MIT5K) // 2. Exposure Correction // 3. Low-Light Object Detection // 4. Low-Light Semantic Segmentation // 5. Various-Light Object Detection

Figure 1: IAT (illumination-adaptive-transformer) for multi light conditions vision tasks.

Model Structure:

Figure 2: Model Structure of Illumination Adaptive Transformer.

Our IAT model consist of two individual branches, the local branch is for pixel-wise adjustment and ouputs two feature map for add and multiply. The global branch is for global-wise adjustment and outpus the color matrix and gamma value, global branch is inspired by DETR, the network would updates color matrix and gamma value by a dynamic query learning way. BTW, the total model is only over 90k+ parameters and the inference speed is only 0.004s per image on LOL dataset (single Nvidia-3090 GPU).

Usage:

Enviroment (install pytorch 1.7.1 or later, following pytorch.):

$ conda install --yes -c pytorch pytorch=1.7.1 torchvision cudatoolkit=11.0
$ pip install timm matplotlib IQA_pytorch tqdm

For low-level vision (low-light enhancement, exposure correction):

cd IAT_enhance

For high-level vision (low-light detection, low-light semantic segmentation):

cd IAT_high

Demo:

Figure 3: IAT in low-light enhancement (LOL dataset, MIT-5K dataset).

Figure 4: IAT in exposure correction (Exposure dataset).

Figure 5: IAT in low-light detection (EXDark Dataset). Background image is the image generated by IAT.

Citation:

Detection and Segmentation are use mmdetection and mmsegmentation, some of the code are borrow from Zero-DCE and UniFormer, thanks them both so much!

Citation:

@misc{Illumination_Adaptive_Transformer,
  doi = {10.48550/ARXIV.2205.14871},
  url = {https://arxiv.org/abs/2205.14871},
  author = {Cui, Ziteng and Li, Kunchang and Gu, Lin and Su, Shenghan and Gao, Peng and Jiang, Zhengkai and Qiao, Yu and Harada, Tatsuya},
  keywords = {Computer Vision and Pattern Recognition (cs.CV), FOS: Computer and information sciences, FOS: Computer and information sciences},
  title = {Illumination Adaptive Transformer},
  publisher = {arXiv},
  year = {2022},
  copyright = {Creative Commons Attribution 4.0 International}
}

We also have another work about the low-light object detection, ICCV 2021: Multitask AET with Orthogonal Tangent Regularity for Dark Object Detection (code) (paper), please read if you interest!