Skip to content


Switch branches/tags

Name already in use

A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch?

Latest commit


Git stats


Failed to load latest commit information.
Latest commit message
Commit time

Building a Parallel Universe - Image Synthesis from Land Cover Maps and Auxiliary Raster Data


This repository contains the code for our paper Building a Parallel Universe - Image Synthesis from Land Cover Maps and Auxiliary Raster Data


Just clone this repository and also make sure you have a recent version of PIL with JPEG2000 support.

git clone


We employ two datasets of high and medium resolution. Both are freely available at the IEEE DataPort.


The GeoNRW dataset consists of 1m aerial photographs, digital elevation models and land cover maps. It can additionally be augmented with TerraSAR-X spotlight acquisitions, provided you have the corresponding data. In that case, please visit to check how to process that data.


This dataset was used for the IEEE data fusion contest in 2020 and consists of Sentinel-1 and Sentinel-2 patches of 256x256pixels, together with semantic maps. All of roughly 10m resolution. You can get the dataset from here, where you need to download

Training new models

  1. Training the models as in the paper requires a multiple GPUs due to memory constraints. Alternatively, the batch size or model capacity can be adjusted using command line parameters.

  2. We advise to use multiple workers when training the NRW dataset. Reading the JPEG2000 files of the GeoNRW's aerial photographs seems to be CPU intensive, and only a single worker bottlenecks the GPU.

  3. Both datasets have different types of input and output that the generator can consume or produce. These can be set using the corresponding command line parameters. The following table lists all of them.

    Dataset Input Output
    NRW dem, seg rgb, sar
    DFC2020 seg rgb, sar
  4. The crop and resize parameters are ignored for the DFC2020 dataset.

  5. The following two examples show how to use the GeoNRW dataset to generate RGB images from digital elevation models and land cover maps

    python -m torch.distributed.launch --nproc_per_node=8 \
        --crop 256 \
        --resize 256 \
        --epochs 200 \
        --batch_size 32 \
        --model_cap 64 \
        --lbda 5.0 \
        --num_workers 4 \
        --dataset 'nrw' \
        --dataroot './data/geonrw' \
        --input 'dem' 'seg' \
        --output 'rgb'

    and dual-pol SAR images from land cover maps alone using the dataset of the 2020 IEEE GRSS data fusion contest

    python -m torch.distributed.launch --nproc_per_node=8 \
    --epochs 200 \
    --batch_size 32 \
    --model_cap 64 \
    --lbda 5.0 \
    --num_workers 0 \
    --dataset 'dfc' \
    --dataroot './data/DFC_Public_Dataset' \
    --input 'seg' \
    --output 'sar

    The training script creates a directory named something like nrw_seg_dem2rgb_bs32_ep200_cap64_2020_09_05_07_58, where the training configuration, logs and the generator and discriminator models will be stored.


Run the testing script with a generator model as a command line argument.

python results/nrw_seg_dem2rgb_bs32_ep200_cap64_2020_09_05_07_58/

This goes through the testing set and plots the results in the corresponding directory.

Computing FID scores

  1. Train a U-Net segmentation network
    python \
        --crop 256  \
        --resize 256 \
        --epochs 100 \
        --batch_size 32 \
        --num_workers 4 \
        --dataset 'nrw' \
        --dataroot './data/geonrw' \
        --input 'rgb'
  2. Compute FID scores by passing the generator to be tested and the just trained U-Net as arguments
    python \
            path_to_generator/ \
  3. Compute intersection-over-union and pixel accuracy
    python \
            path_to_generator/ \
  4. You can optionally also compute the segmentation results
    python \
    which stores the segmentation results in the model's directory.

Codes structure

GAN training and image synthesis

  • and for training and testing image synthesis.
  • options/ defines command line arguments for training the GANs and U-Net.
  • datasets/ contains data loaders and transforms.
  • models/ contains the various network architectures.
  • defines the GAN loss functions.
  • our general GAN trainer.
  • utility functions.

Numerical analysis

  • and for training and testing U-Net..
  • calculates Fréchet inception distances using a pretrained U-Net
  • calculates pixel accuarcy and IoU using a pretrained U-Net


In case you use this code in your research please consider citing

      title={Building a Parallel Universe - Image Synthesis from Land Cover Maps and Auxiliary Raster Data}, 
      author={Gerald Baier and Antonin Deschemps and Michael Schmitt and Naoto Yokoya},


No description, website, or topics provided.






No releases published


No packages published