Deep Learning for Earth Observation: Chapter Domain Adaptation

Code base for reproducing the experiments in the book "Deep Learning for the Earth Sciences: A Comprehensive Approach to Remote Sensing, Climate Science, and Geosciences", Chapter 7: "Deep Domain Adaptation in Earth Observation", Section 7.3.1: "Adapting the inner representation."

Overview

This repository contains code to train, adapt, and evaluate ResNet-based models on and to the following two datasets:

• UC-Merced
• WHU-RS19

The following domain adaptation methodologies are supported:

• MMD
• DeepCORAL
• DeepJDOT

Installation

Setup code base

The following instructions apply to Conda.

    conda create -y -n da4dl4eo python=3.7
    conda activate da4dl4eo
    conda install -c anaconda scikit-learn
    conda install -c conda-forge tqdm matplotlib pot
    conda install pytorch torchvision cudatoolkit=10.1 -c pytorch

Download datasets

1. Create folders in the repository root: mkdir -p datasets/UCMerced; mkdir -p datasets/WHU-RS19
2. Download the UC-Merced dataset from here and extract it into the folder datasets/UCMerced
3. Repeat the same for the WHU-RS19 dataset and extract it into the folder datasets/WHU-RS19 (note: you might need a .rar extraction utility to do so).
4. Create train/val/test splits: conda activate da4dl4eo; python datasets.py

Run

The code snippets below can be used to train, adapt, and test a model, respectively.

Make sure to cd to the repository root and conda activate da4dl4eo before running one of the commands below.

Train

Example: train ResNet-18 on UC-Merced:

    python 1_train.py --dataset UCMerced --backbone resnet18

Perform domain adaptation

Example: adapt a ResNet-18 from UC-Merced to WHU-RS19 using DeepCORAL:

    python 2_adapt.py --dataset_source UCMerced --dataset_target WHU-RS19 --daMethod DeepCORAL --backbone resnet18

Test

Note: Models that have undergone domain adaptation will be saved in the sub-folder of the target dataset.

Example 1: test a ResNet-18, adapted from UC-Merced to WHU-RS19 using DeepCORAL, on WHU-RS19:

    python 3_test.py --dataset_target WHU-RS19 --dataset_model UCMerced --daMethod DeepCORAL --backbone resnet18

Example 2: test a ResNet-18, trained on UC-Merced without domain adaptation, on WHU-RS19. Also save the resulting statistics to LaTeX files:

    python 3_test.py --dataset_target WHU-RS19 --dataset_model UCMerced --backbone resnet18 --saveResults 1

Replicate book chapter results

To replicate the results, the following experiments need to be conducted:

    # train base models
    python 1_train.py --dataset UCMerced
    python 1_train.py --dataset WHU-RS19
    python 1_train.py --dataset both


    # adapt
    python 2_adapt.py --dataset_source UCMerced --dataset_target WHU-RS19 --daMethod MMD
    python 2_adapt.py --dataset_source UCMerced --dataset_target WHU-RS19 --daMethod DeepCORAL
    python 2_adapt.py --dataset_source UCMerced --dataset_target WHU-RS19 --daMethod DeepJDOT

    python 2_adapt.py --dataset_source WHU-RS19 --dataset_target UCMerced --daMethod MMD
    python 2_adapt.py --dataset_source WHU-RS19 --dataset_target UCMerced --daMethod DeepCORAL
    python 2_adapt.py --dataset_source WHU-RS19 --dataset_target UCMerced --daMethod DeepJDOT


    # test (order: source only, MMD, DeepCORAL, DeepJDOT, target only; for both UCMerced and WHU-RS19 target datasets)
    python 3_test.py --dataset_target UCMerced --dataset_model WHU-RS19
    python 3_test.py --dataset_target UCMerced --dataset_model WHU-RS19 --daMethod MMD
    python 3_test.py --dataset_target UCMerced --dataset_model WHU-RS19 --daMethod DeepCORAL
    python 3_test.py --dataset_target UCMerced --dataset_model WHU-RS19 --daMethod DeepJDOT
    python 3_test.py --dataset_target UCMerced --dataset_model UCMerced
    python 3_test.py --dataset_target UCMerced --dataset_model all

    python 3_test.py --dataset_target WHU-RS19 --dataset_model UCMerced
    python 3_test.py --dataset_target WHU-RS19 --dataset_model UCMerced --daMethod MMD
    python 3_test.py --dataset_target WHU-RS19 --dataset_model UCMerced --daMethod DeepCORAL
    python 3_test.py --dataset_target WHU-RS19 --dataset_model UCMerced --daMethod DeepJDOT
    python 3_test.py --dataset_target WHU-RS19 --dataset_model WHU-RS19
    python 3_test.py --dataset_target WHU-RS19 --dataset_model all