Fully convolutional networks for semantic segmentation
Branch: master
Clone or download
Latest commit 405f57c Sep 10, 2018
Type Name Latest commit message Commit time
Failed to load latest commit information.
.gitignore Add data folders to gitignore Jun 22, 2017
LICENSE.md Add code Jun 22, 2017
README.md Add separate bias lr/weight decay Jul 18, 2017
data.py Modularise code Jun 27, 2017
main.py Fix batch norm key when modifying optim params Jul 18, 2017
model.py Fix bilinear interp weight init Sep 10, 2018



Simple end-to-end semantic segmentation using fully convolutional networks [1]. Takes a pretrained 34-layer ResNet [2], removes the fully connected layers, and adds transposed convolution layers with skip connections from lower layers. Initialises upsampling convolutions with bilinear interpolation filters and zeros the final (classification) layer.

Uses an independent cross-entropy loss per class. Trained with SGD with momentum, plus weight decay only on convolutional weights. Calculates and plots class-wise and mean intersection-over-union. Checkpoints the network every epoch.

Note: This code does not achieve great results (achieves ~40 IoU fairly quickly, but converges there). Contributions to fix this are welcome! The goal of this repo is to provide strong, simple and efficient baselines for semantic segmentation using the FCN method, so this shouldn't be restricted to using ResNet 34 etc.



  1. Install all of the required software. To feasibly run the training, CUDA is needed. The crop size and batch size can be tailored to your GPU memory (the default crop and batch sizes use ~10GB of GPU RAM).
  2. Register on the Cityscapes website to access the dataset.
  3. Download and extract the training/validation RGB data (leftImg8bit_trainvaltest) and ground truth data (gtFine_trainvaltest).
  4. Run python main.py <options>.

First a Dataset object is set up, returning the RGB inputs, one-hot targets (for independent classification) and label targets. During training, the images are randomly cropped and horizontally flipped. Testing calculates IoU scores and produces a subset of coloured predictions that match the coloured ground truth.


[1] Fully convolutional networks for semantic segmentation
[2] Deep Residual Learning for Image Recognition