Pytorch implementation of our paper Semantic-Guided Inpainting Network for Complex UrbanScenes Manipulation In ICPR 2020. Please cite with the following Bibtex code:
@INPROCEEDINGS{9412690,
author={Ardino, Pierfrancesco and Liu, Yahui and Ricci, Elisa and Lepri, Bruno and de Nadai, Marco},
booktitle={2020 25th International Conference on Pattern Recognition (ICPR)},
title={Semantic-Guided Inpainting Network for Complex Urban Scenes Manipulation},
year={2021},
volume={},
number={},
pages={9280-9287},
doi={10.1109/ICPR48806.2021.9412690}}
Please follow the instructions to run the code.
- See the
sgi_net.yml
configuration file. We provide an user-friendly configuring method via Conda system, and you can create a new Conda environment using the command:
conda env create -f sgi_net.yml
conda activate sgi_net
- Install
cityscapesscripts
withpip
cd cityscapesScripts
pip install -e .
- Install
nvidia-apex
using
sh install_nvidia_apex.sh
-
Please download the Cityscapes dataset from the official website (registration required). After downloading, please put these files under the
~/datasets/cityscapes/
folder and run the following command in order to generate the correct segmentation mapscd cityscapesScripts CITYSCAPES_DATASET=~/datasets/cityscapes/ python cityscapesscripts/preparation/createTrainIdLabelImgs.py
If you want to use a different number of labels for the segmentation you can change them in the
cityscapesScripts/cityscapesscripts/helpers/labels.py
file.You should end up with the following structure:
datasets βββ cityscapes β βββ leftImg8bit_sequence β β βββ train β β β βββ aachen β β β β βββ aachen_000003_000019_leftImg8bit.png β β β β βββ ... β β βββ val β β β βββ frankfurt β β β β βββ frankfurt_000000_000294_leftImg8bit.png β β β β βββ ... β βββ gtFine β β βββ train β β β βββ aachen β β β β βββ aachen_000003_000019_gtFine_trainIds.png β β β β βββ aachen_000003_000019_gtFine_polygons.json β β β β βββ aachen_000003_000019_gtFine_instanceIds.png β β β β βββ ... β β βββ val β β β βββ frankfurt β β β β βββ frankfurt_000000_000294_gtFine_trainIds.png β β β β βββ frankfurt_000000_000294_gtFine_polygons.json β β β β βββ frankfurt_000000_000294_gtFine_instanceIds.png β β β β βββ ...
-
Then run the script
src/preprocess_city.py
in order to prepare the dataset.The script takes as input three parameters:
dataroot
: Folder where the Cityscape dataset has been extracted.resize_size
: New size of the images (width,height). By default the images will not be resized. Default value: (2048,1024)use_multiprocessing
: Run the preprocessing in parallel. By default is disabled
cd src python preprocess_city.py --dataroot ~/datasets/cityscapes/ --resize_size 512,256 --use_multiprocessing
-
Copy the train list and the evaluation list from
file_list/cityscapes
into the dataroot~/datasets/cityscapes/
cp file_list/cityscapes/* ~/dataset/cityscapes/
You should end up with the following structure:
datasets βββ cityscapes β βββ train_img β β βββ aachen_000003_000019_leftImg8bit.png β β βββ ... β βββ val_img β β βββ frankfurt_000000_000294_leftImg8bit.png β β βββ ... β βββ train_label β β βββ aachen_000003_000019_gtFine_trainIds.png β β βββ ... β βββ val_label β β βββ frankfurt_000000_000294_gtFine_trainIds.png β β βββ ... β βββ train_inst β β βββ aachen_000003_000019_gtFine_data.json β β βββ aachen_000003_000019_gtFine_instanceIds.png β β βββ ... β βββ val_inst β β βββ frankfurt_000000_000294_gtFine_data.json β β βββ frankfurt_000000_000294_gtFine_instanceIds.png β β βββ ... β βββ train.txt β βββ val.txt
TODO
- Train a model at 256 x 256 resolution with cropping and Pixel Shuffle in the decoder
cd src sh script/train_paper_cityscapes_pixel_shuffle.sh
- Train a model at 256 x 256 resolution with cropping and deconvolution in the decoder
cd src sh script/train_paper_cityscapes.sh
- Train a model at 256 x 256 resolution with cropping and Pixel Shuffle in the decoder
cd src sh script/train_paper_cityscapes_multigpu_pixel_shuffle.sh
- Train a model at 256 x 256 resolution with cropping and deconvolution in the decoder
cd src sh script/train_paper_cityscapes_multigpu.sh
The example consider a scenario with a single node and two gpus per node. Please change according to your needs. For more information check the DDP example
- Train a model at 256 x 256 resolution with cropping and Pixel Shuffle in the decoder
cd src sh script/train_paper_cityscapes_fp16_multigpu_pixel_shuffle.sh
- Train a model at 256 x 256 resolution with cropping and deconvolution in the decoder
cd src sh script/train_paper_cityscapes_fp16_multigpu.sh
The example consider a scenario with a single node and two gpus per node. Please change according to your needs. For more information check the DDP example
WORK IN PROGRESS
- Flags: see
options/train_options.py
andoptions/base_options.py
for all the training flags; seeoptions/test_options.py
andoptions/base_options.py
for all the test flags.
This code borrows heavily from pix2pixHD.