Skip to content

PierfrancescoArdino/SGINet

main
Switch branches/tags

Name already in use

A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch?
Code

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
src
 
 
 
 
 
 
 
 

SGINet

Pytorch implementation of our paper Semantic-Guided Inpainting Network for Complex UrbanScenes Manipulation In ICPR 2020. Please cite with the following Bibtex code:

@INPROCEEDINGS{9412690,
  author={Ardino, Pierfrancesco and Liu, Yahui and Ricci, Elisa and Lepri, Bruno and de Nadai, Marco},
  booktitle={2020 25th International Conference on Pattern Recognition (ICPR)}, 
  title={Semantic-Guided Inpainting Network for Complex Urban Scenes Manipulation}, 
  year={2021},
  volume={},
  number={},
  pages={9280-9287},
  doi={10.1109/ICPR48806.2021.9412690}}

Please follow the instructions to run the code.

Scripts

1. Installation

  • See the sgi_net.yml configuration file. We provide an user-friendly configuring method via Conda system, and you can create a new Conda environment using the command:
conda env create -f sgi_net.yml
conda activate sgi_net
  • Install cityscapesscripts with pip
cd cityscapesScripts
pip install -e .
  • Install nvidia-apex using
sh install_nvidia_apex.sh

2. Data Preprocessing

Cityscapes

  • Please download the Cityscapes dataset from the official website (registration required). After downloading, please put these files under the ~/datasets/cityscapes/ folder and run the following command in order to generate the correct segmentation maps

    cd cityscapesScripts
    CITYSCAPES_DATASET=~/datasets/cityscapes/
    python cityscapesscripts/preparation/createTrainIdLabelImgs.py
    

    If you want to use a different number of labels for the segmentation you can change them in the cityscapesScripts/cityscapesscripts/helpers/labels.py file.

    You should end up with the following structure:

    datasets
    ├── cityscapes
    │   ├── leftImg8bit_sequence
    │   │   ├── train
    │   │   │   ├── aachen
    │   │   │   │   ├── aachen_000003_000019_leftImg8bit.png
    │   │   │   │   ├── ...
    │   │   ├── val
    │   │   │   ├── frankfurt
    │   │   │   │   ├── frankfurt_000000_000294_leftImg8bit.png
    │   │   │   │   ├── ...
    │   ├── gtFine
    │   │   ├── train
    │   │   │   ├── aachen
    │   │   │   │   ├── aachen_000003_000019_gtFine_trainIds.png
    │   │   │   │   ├── aachen_000003_000019_gtFine_polygons.json
    │   │   │   │   ├── aachen_000003_000019_gtFine_instanceIds.png
    │   │   │   │   ├── ...
    │   │   ├── val
    │   │   │   ├── frankfurt
    │   │   │   │   ├── frankfurt_000000_000294_gtFine_trainIds.png
    │   │   │   │   ├── frankfurt_000000_000294_gtFine_polygons.json
    │   │   │   │   ├── frankfurt_000000_000294_gtFine_instanceIds.png
    │   │   │   │   ├── ...
    
  • Then run the script src/preprocess_city.py in order to prepare the dataset.

    Usage

    The script takes as input three parameters:

    • dataroot: Folder where the Cityscape dataset has been extracted.
    • resize_size: New size of the images (width,height). By default the images will not be resized. Default value: (2048,1024)
    • use_multiprocessing: Run the preprocessing in parallel. By default is disabled

    Example

    cd src
    python preprocess_city.py --dataroot ~/datasets/cityscapes/ --resize_size 512,256 --use_multiprocessing
    
  • Copy the train list and the evaluation list from file_list/cityscapes into the dataroot ~/datasets/cityscapes/

    cp file_list/cityscapes/* ~/dataset/cityscapes/

    You should end up with the following structure:

    datasets
    ├── cityscapes
    │   ├── train_img
    │   │   ├── aachen_000003_000019_leftImg8bit.png
    │   │   ├── ...
    │   ├── val_img
    │   │   ├── frankfurt_000000_000294_leftImg8bit.png
    │   │   ├── ...
    │   ├── train_label
    │   │   ├── aachen_000003_000019_gtFine_trainIds.png
    │   │   ├── ...
    │   ├── val_label
    │   │   ├── frankfurt_000000_000294_gtFine_trainIds.png
    │   │   ├── ...
    │   ├── train_inst
    │   │   ├── aachen_000003_000019_gtFine_data.json
    │   │   ├── aachen_000003_000019_gtFine_instanceIds.png
    │   │   ├── ...
    │   ├── val_inst
    │   │   ├── frankfurt_000000_000294_gtFine_data.json
    │   │   ├── frankfurt_000000_000294_gtFine_instanceIds.png
    │   │   ├── ...
    │   ├── train.txt
    │   ├── val.txt
    

Indian Driving Dataset

TODO

3. Train

Single-GPU Train

  • Train a model at 256 x 256 resolution with cropping and Pixel Shuffle in the decoder
    cd src
    sh script/train_paper_cityscapes_pixel_shuffle.sh
  • Train a model at 256 x 256 resolution with cropping and deconvolution in the decoder
    cd src
    sh script/train_paper_cityscapes.sh

Multi-GPU Train

  • Train a model at 256 x 256 resolution with cropping and Pixel Shuffle in the decoder
    cd src
    sh script/train_paper_cityscapes_multigpu_pixel_shuffle.sh
  • Train a model at 256 x 256 resolution with cropping and deconvolution in the decoder
    cd src
    sh script/train_paper_cityscapes_multigpu.sh

The example consider a scenario with a single node and two gpus per node. Please change according to your needs. For more information check the DDP example

Training with Automatic Mixed Precision (AMP) for faster speed

  • Train a model at 256 x 256 resolution with cropping and Pixel Shuffle in the decoder
    cd src
    sh script/train_paper_cityscapes_fp16_multigpu_pixel_shuffle.sh
  • Train a model at 256 x 256 resolution with cropping and deconvolution in the decoder
    cd src
    sh script/train_paper_cityscapes_fp16_multigpu.sh

The example consider a scenario with a single node and two gpus per node. Please change according to your needs. For more information check the DDP example

4. Test

WORK IN PROGRESS

More Training/Test Details

  • Flags: see options/train_options.py and options/base_options.py for all the training flags; see options/test_options.py and options/base_options.py for all the test flags.

Acknowledgments

This code borrows heavily from pix2pixHD.

About

Code for the paper Semantic-Guided Inpainting Network for Complex UrbanScenes Manipulation

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published