MetaWeather: Few-shot Weather-Degraded Image Restoration (ECCV 2024)

Project Page  |  Paper

The official code for MetaWeather: Few-shot Weather-Degraded Image Restoration (ECCV 2024).

Setup

    MetaWeather
    ├ data
    │   ├ BID
    │   │  ├ case1
    │   │  ├ case2
    │   │  ├ ...
    │   │  ├ case6
    │   │  └ gt
    │   ├ SPAData/real_test_100
    │   │  ├ input
    │   │  └ gt
    │   ├ realsnow
    │   │  ├ train
    │   │  │  ├ input
    │   │  │  └ gt
    │   │  └ test
    │   │     ├ input
    │   │     └ gt
    │   ├ BID-case1-train.txt
    │   ├ BID-case1-test.txt
    │   ├ BID-case2-train.txt
    │   ├ BID-case2-test.txt
    │   ├ ...
    │   ├ BID-case6-test.txt
    │   ├ spadata-train.txt
    │   ├ spadata-test.txt
    │   ├ realsnow-train.txt
    │   ├ realsnow-test.txt
    │   └ simmim_pretrain__swin_base__img192_window6__800ep.pth
    └ checkpoints
        ├ metaweather_meta_train.pth
        ├ metaweather_1shot_case1.pth
        ├ metaweather_1shot_case2.pth
        ├ ...

Datasets

Download the datasets below:

• BID Dataset [Link]
• SPA-Data [Link]
• RealSnow [Link]

Pretrained MetaWeather Model

Download checkpoints [Link]

• Place them in ./checkpoints/

Prerequisite

Use conda environment:

conda env create -f env.yaml

Swin Transformer

If you want to train the model from the scratch, download simmim_pretrain__swin_base__img192_window6__800ep.pth from [Link]

• Place the file in data.

Usage

Meta-Train

python3 train.py --stage=0 --meta_train=True --exp_name=test

Meta-Test

python3 train.py --stage=1 --case=<case> --meta_train=True --exp_name=test --checkpoint=<path-to-checkpoint>

• case: (1-6) BID Task II.A Case 1-6, (7) SPA-Data, (8) RealSnow

Evaluation

python3 test.py --case=<case> --exp_name=test --checkpoint=<path-to-checkpoint>

• case: (1-6) BID Task II.A Case 1-6, (7) SPA-Data, (8) RealSnow

Examples

python3 test.py --case=1 --checkpoint=./checkpoints/metaweather_1shot_case1.pth

python3 test.py --case=3 --checkpoint=./checkpoints/metaweather_1shot_case3.pth

Tested on Ubuntu 20.04, NVIDIA RTX 3090, CUDA 11.7, PyTorch 2.0.1, Python 3.10

References

Our code is built upon the following works:

• Visual Token Matching
• Transweather
• NAFNet
• Swin Transformer

Citation

TODO

Acknowledgements

This work was supported by Institute of Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (No.2019-0-01126, Self-learning based Autonomic IoT Edge Computing).