Merge pull request #183 from CSAILVision/hang-dev

Use configuration files to store most options which were in argument parser

Author: hangzhaomit

README.md

@@ -5,6 +5,8 @@ This is a PyTorch implementation of semantic segmentation models on MIT ADE20K s
 ADE20K is the largest open source dataset for semantic segmentation and scene parsing, released by MIT Computer Vision team. Follow the link below to find the repository for our dataset and implementations on Caffe and Torch7:
 https://github.com/CSAILVision/sceneparsing
 
+If you simply want to play with our demo, please try this link: http://scenesegmentation.csail.mit.edu You can upload your own photo and segment it!
+
 All pretrained models can be found at:
 http://sceneparsing.csail.mit.edu/model/pytorch
 
@@ -15,6 +17,10 @@ http://sceneparsing.csail.mit.edu/model/pytorch
 Color encoding of semantic categories can be found here:
 https://docs.google.com/spreadsheets/d/1se8YEtb2detS7OuPE86fXGyD269pMycAWe2mtKUj2W8/edit?usp=sharing
 
+## Updates
+- We use configuration files to store most options which were in argument parser. The definitions of options are detailed in ```config/defaults.py```.
+
+
 ## Highlights
 
 ### Syncronized Batch Normalization on PyTorch
@@ -43,9 +49,6 @@ Encoder:
 - ResNet50dilated
 - ResNet101dilated
 
-***Coming soon***:
-- ResNeXt101dilated
-
 Decoder:
 - C1 (1 convolution module)
 - C1_deepsup (C1 + deep supervision trick)
@@ -144,12 +147,6 @@ IMPORTANT: We use our self-trained base model on ImageNet. The model takes the i
         <td>Yes</td><td>42.66</td><td>81.01</td><td>61.84</td>
         <td>2.3</td>
     </tr>
-    <tr>
-        <td>UPerNet-ResNext101 (coming soon!)</td>
-        <td>-</td><td>-</td><td>-</td><td>-</td>
-        <td>-</td>
-        <td>-</td>
-    </tr>
 </tbody></table>
 
 The training is benchmarked on a server with 8 NVIDIA Pascal Titan Xp GPUs (12GB GPU memory), ***except for*** ResNet101dilated, which is benchmarked on a server with 8 NVIDIA Tesla P40 GPUS (22GB GPU memory), because of the insufficient memory issue when using dilated conv on a very deep network. The inference speed is benchmarked a single NVIDIA Pascal Titan Xp GPU, without visualization.
@@ -158,6 +155,7 @@ The training is benchmarked on a server with 8 NVIDIA Pascal Titan Xp GPUs (12GB
 The code is developed under the following configurations.
 - Hardware: 1-8 GPUs (with at least 12G GPU memories) (change ```[--gpus GPUS]``` accordingly)
 - Software: Ubuntu 16.04.3 LTS, ***CUDA>=8.0, Python>=3.5, PyTorch>=0.4.0***
+- Dependencies: numpy, scipy, opencv, yacs, tqdm
 
 ## Quick start: Test on an image using our trained model 
 1. Here is a simple demo to do inference on a single image:
@@ -167,86 +165,61 @@ chmod +x demo_test.sh
 ```
 This script downloads a trained model (ResNet50dilated + PPM_deepsup) and a test image, runs the test script, and saves predicted segmentation (.png) to the working directory.
 
-2. To test on multiple images or a folder of images, you can simply do something as the following (```$PATH_IMG1, $PATH_IMG2, $PATH_IMG3```are your image paths):
+2. To test on an image or a folder of images (```$PATH_IMG```), you can simply do the following:
 ```
-python3 -u test.py \
-  --model_path $MODEL_PATH \
-  --test_imgs $PATH_IMG1 $PATH_IMG2 $PATH_IMG3 \
-  --arch_encoder resnet50dilated \
-  --arch_decoder ppm_deepsup
+python3 -u test.py --imgs $PATH_IMG --gpu $GPU --cfg $CFG
 ```
 
-3. See full input arguments via ```python3 test.py -h```.
-
 ## Training
 1. Download the ADE20K scene parsing dataset:
 ```bash
 chmod +x download_ADE20K.sh
 ./download_ADE20K.sh
 ```
-2. Train a model (default: ResNet50dilated + PPM_deepsup). During training, checkpoints will be saved in folder ```ckpt```.
+2. Train a model by selecting the GPUs (```$GPUS```) and configuration file (```$CFG```) to use. During training, checkpoints by default are saved in folder ```ckpt```.
 ```bash
-python3 train.py --gpus GPUS
+python3 train.py --gpus $GPUS --cfg $CFG 
 ```
-
 - To choose which gpus to use, you can either do ```--gpus 0-7```, or ```--gpus 0,2,4,6```.
 
-For example:
+For example, you can start with our provided configurations: 
 
 * Train MobileNetV2dilated + C1_deepsup
 ```bash
-python3 train.py --gpus GPUS \
-    --arch_encoder mobilenetv2dilated --arch_decoder c1_deepsup \
-    --fc_dim 320
+python3 train.py --gpus GPUS --cfg config/ade20k-mobilenetv2dilated-c1_deepsup.yaml
 ```
 
-* Train ResNet18dilated + PPM_deepsup
+* Train ResNet50dilated + PPM_deepsup
 ```bash
-python3 train.py --gpus GPUS \
-    --arch_encoder resnet18dilated --arch_decoder ppm_deepsup \
-    --fc_dim 512
+python3 train.py --gpus GPUS --cfg config/ade20k-resnet50dilated-ppm_deepsup.yaml
 ```
 
 * Train UPerNet101
 ```bash
-python3 train.py --gpus GPUS \
-    --arch_encoder resnet101 --arch_decoder upernet \
-    --segm_downsampling_rate 4 --padding_constant 32
+python3 train.py --gpus GPUS --cfg config/ade20k-resnet101-upernet.yaml
 ```
 
-3. See full input arguments via ```python3 train.py -h ```.
+3. You can also override options in commandline, for example  ```python3 train.py TRAIN.num_epoch 10 ```.
 
 
 ## Evaluation
-1. Evaluate a trained model on the validation set. ```--id``` is the folder name under ```ckpt``` directory. ```--suffix``` defines which checkpoint to use, for example ```_epoch_20.pth```. Add ```--visualize``` option to output visualizations as shown in teaser.
-```bash
-python3 eval_multipro.py --gpus GPUS --id MODEL_ID --suffix SUFFIX
-```
+1. Evaluate a trained model on the validation set. Add ```VAL.visualize True``` in argument to output visualizations as shown in teaser.
 
 For example:
 
 * Evaluate MobileNetV2dilated + C1_deepsup
 ```bash
-python3 eval_multipro.py --gpus GPUS \
-    --id MODEL_ID --suffix SUFFIX --arch_encoder mobilenetv2dilated --arch_decoder c1_deepsup \
-    --fc_dim 320
+python3 eval_multipro.py --gpus GPUS --cfg config/ade20k-mobilenetv2dilated-c1_deepsup.yaml
 ```
 
-* Evaluate ResNet18dilated + PPM_deepsup
+* Evaluate ResNet50dilated + PPM_deepsup
 ```bash
-python3 eval_multipro.py --gpus GPUS \
-    --id MODEL_ID --suffix SUFFIX --arch_encoder resnet18dilated --arch_decoder ppm_deepsup \
-    --fc_dim 512
+python3 eval_multipro.py --gpus GPUS --cfg config/ade20k-resnet50dilated-ppm_deepsup.yaml
 ```
 
 * Evaluate UPerNet101
 ```bash
-python3 eval_multipro.py --gpus GPUS \
-    --id MODEL_ID --suffix SUFFIX --arch_encoder resnet101 --arch_decoder upernet \
-    --padding_constant 32
-```
-
-2. See full input arguments via ```python3 eval_multipro.py -h ```.
+python3 eval_multipro.py --gpus GPUS --cfg config/ade20k-resnet101-upernet.yaml
 
 ## Reference
 

config/__init__.py

@@ -0,0 +1 @@
+from .defaults import _C as cfg

config/ade20k-mobilenetv2dilated-c1_deepsup.yaml

@@ -0,0 +1,44 @@
+DATASET:
+  root_dataset: "./data/"
+  list_train: "./data/training.odgt"
+  list_val: "./data/validation.odgt"
+  num_class: 150
+  imgSizes: (300, 375, 450, 525, 600)
+  imgMaxSize: 1000
+  padding_constant: 8
+  segm_downsampling_rate: 8
+  random_flip: True
+
+MODEL:
+  arch_encoder: "mobilenetv2dilated"
+  arch_decoder: "c1_deepsup"
+  weights_encoder: ""
+  weights_decoder: ""
+  fc_dim: 320
+
+TRAIN:
+  batch_size_per_gpu: 2
+  num_epoch: 20
+  start_epoch: 0
+  epoch_iters: 5000
+  optim: "SGD"
+  lr_encoder: 0.02
+  lr_decoder: 0.02
+  lr_pow: 0.9
+  beta1: 0.9
+  weight_decay: 1e-4
+  deep_sup_scale: 0.4
+  fix_bn: False
+  workers: 16
+  disp_iter: 20
+  seed: 304
+
+VAL:
+  visualize: False
+  suffix: "_epoch_20.pth"
+
+TEST:
+  suffix: "_epoch_20.pth"
+  result: "./"
+
+DIR: "ckpt/ade20k-mobilenetv2dilated-c1_deepsup"

config/ade20k-resnet101-upernet.yaml

@@ -0,0 +1,44 @@
+DATASET:
+  root_dataset: "./data/"
+  list_train: "./data/training.odgt"
+  list_val: "./data/validation.odgt"
+  num_class: 150
+  imgSizes: (300, 375, 450, 525, 600)
+  imgMaxSize: 1000
+  padding_constant: 32
+  segm_downsampling_rate: 4
+  random_flip: True
+
+MODEL:
+  arch_encoder: "resnet101"
+  arch_decoder: "upernet"
+  weights_encoder: ""
+  weights_decoder: ""
+  fc_dim: 2048
+
+TRAIN:
+  batch_size_per_gpu: 2
+  num_epoch: 40
+  start_epoch: 0
+  epoch_iters: 5000
+  optim: "SGD"
+  lr_encoder: 0.02
+  lr_decoder: 0.02
+  lr_pow: 0.9
+  beta1: 0.9
+  weight_decay: 1e-4
+  deep_sup_scale: 0.4
+  fix_bn: False
+  workers: 16
+  disp_iter: 20
+  seed: 304
+
+VAL:
+  visualize: False
+  suffix: "_epoch_40.pth"
+
+TEST:
+  suffix: "_epoch_40.pth"
+  result: "./"
+
+DIR: "ckpt/ade20k-resnet101-upernet"

config/ade20k-resnet101dilated-ppm_deepsup.yaml

@@ -0,0 +1,44 @@
+DATASET:
+  root_dataset: "./data/"
+  list_train: "./data/training.odgt"
+  list_val: "./data/validation.odgt"
+  num_class: 150
+  imgSizes: (300, 375, 450, 525, 600)
+  imgMaxSize: 1000
+  padding_constant: 8
+  segm_downsampling_rate: 8
+  random_flip: True
+
+MODEL:
+  arch_encoder: "resnet50dilated"
+  arch_decoder: "ppm_deepsup"
+  weights_encoder: ""
+  weights_decoder: ""
+  fc_dim: 2048
+
+TRAIN:
+  batch_size_per_gpu: 2
+  num_epoch: 20
+  start_epoch: 0
+  epoch_iters: 5000
+  optim: "SGD"
+  lr_encoder: 0.02
+  lr_decoder: 0.02
+  lr_pow: 0.9
+  beta1: 0.9
+  weight_decay: 1e-4
+  deep_sup_scale: 0.4
+  fix_bn: False
+  workers: 16
+  disp_iter: 20
+  seed: 304
+
+VAL:
+  visualize: False
+  suffix: "_epoch_20.pth"
+
+TEST:
+  suffix: "_epoch_20.pth"
+  result: "./"
+
+DIR: "ckpt/ade20k-resnet50dilated-ppm_deepsup"

config/ade20k-resnet18dilated-ppm_deepsup.yaml

@@ -0,0 +1,44 @@
+DATASET:
+  root_dataset: "./data/"
+  list_train: "./data/training.odgt"
+  list_val: "./data/validation.odgt"
+  num_class: 150
+  imgSizes: (300, 375, 450, 525, 600)
+  imgMaxSize: 1000
+  padding_constant: 8
+  segm_downsampling_rate: 8
+  random_flip: True
+
+MODEL:
+  arch_encoder: "resnet18dilated"
+  arch_decoder: "ppm_deepsup"
+  weights_encoder: ""
+  weights_decoder: ""
+  fc_dim: 512
+
+TRAIN:
+  batch_size_per_gpu: 2
+  num_epoch: 20
+  start_epoch: 0
+  epoch_iters: 5000
+  optim: "SGD"
+  lr_encoder: 0.02
+  lr_decoder: 0.02
+  lr_pow: 0.9
+  beta1: 0.9
+  weight_decay: 1e-4
+  deep_sup_scale: 0.4
+  fix_bn: False
+  workers: 16
+  disp_iter: 20
+  seed: 304
+
+VAL:
+  visualize: False
+  suffix: "_epoch_20.pth"
+
+TEST:
+  suffix: "_epoch_20.pth"
+  result: "./"
+
+DIR: "ckpt/ade20k-resnet18dilated-ppm_deepsup"

config/ade20k-resnet50dilated-ppm_deepsup.yaml

@@ -0,0 +1,44 @@
+DATASET:
+  root_dataset: "./data/"
+  list_train: "./data/training.odgt"
+  list_val: "./data/validation.odgt"
+  num_class: 150
+  imgSizes: (300, 375, 450, 525, 600)
+  imgMaxSize: 1000
+  padding_constant: 8
+  segm_downsampling_rate: 8
+  random_flip: True
+
+MODEL:
+  arch_encoder: "resnet50dilated"
+  arch_decoder: "ppm_deepsup"
+  weights_encoder: ""
+  weights_decoder: ""
+  fc_dim: 2048
+
+TRAIN:
+  batch_size_per_gpu: 2
+  num_epoch: 20
+  start_epoch: 0
+  epoch_iters: 5000
+  optim: "SGD"
+  lr_encoder: 0.02
+  lr_decoder: 0.02
+  lr_pow: 0.9
+  beta1: 0.9
+  weight_decay: 1e-4
+  deep_sup_scale: 0.4
+  fix_bn: False
+  workers: 16
+  disp_iter: 20
+  seed: 304
+
+VAL:
+  visualize: False
+  suffix: "_epoch_20.pth"
+
+TEST:
+  suffix: "_epoch_20.pth"
+  result: "./"
+
+DIR: "ckpt/ade20k-resnet50dilated-ppm_deepsup"