Skip to content
Realtime Semantic Segmentation (ICNet) for Self-Driving Vehicles
Python Jupyter Notebook Shell
Branch: master
Clone or download
Latest commit bc04a59 Mar 21, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.idea cross fusion model guidance labels added Mar 21, 2019
.ipynb_checkpoints icnet training started Jan 2, 2019
logs/ICNet cleaned up some logs Mar 21, 2019
media removed old models, updated README Mar 18, 2019
models cross fusion model guidance labels added Mar 21, 2019
monodepth mid fusion model developed, testing... Mar 15, 2019
testing_imgs Cross fusion model v2 implemented, trained. Mar 20, 2019
.DS_Store
.gitignore monodepth added to repo, simple test completed. Mar 3, 2019
Data Preprocessing.ipynb productive training on new img size Jan 13, 2019
LICENSE Initial commit May 13, 2018
NOTES.md Cross fusion model v2 implemented, trained. Mar 20, 2019
README.md removed old models, updated README Mar 18, 2019
configs.py early fusion implemented, training Mar 9, 2019
configs.pyc early fusion implemented, training Mar 9, 2019
new_labels.csv
new_val_labels.csv early fusion implemented, training Mar 9, 2019
segmentor.py initial commit May 13, 2018
test_fusion.py Cross fusion model v2 implemented, trained. Mar 20, 2019
test_icnet.py removed old models, updated README Mar 18, 2019
train_fusion.py cross fusion model guidance labels added Mar 21, 2019
train_icnet.py
utils.py cross fusion model guidance labels added Mar 21, 2019
utils.pyc cross fusion model guidance labels added Mar 21, 2019
validation.py Cross fusion model v2 implemented, trained. Mar 20, 2019
validation_icnet.py cross fusion model guidance labels added Mar 21, 2019
visualization.py initial commit May 13, 2018

README.md

Semantic Segmentation

🚧 This project still under development. Some scripts might be incomplete. 🏗

Semantic segmentation has become an important component of self-driving vehicles. It allows the car to understand the surroundings by classifying every pixel of the input image.

🏃🏻‍♂️ Running...

To run inference on the pre-trained models, please use segmentor.py.

from segmentor import Segmentor
seg = Segmentor()
classes_output, img_viz = seg.semantic_segmentation(image=image, visualization=True)

classes_output is the pixel-wise classification result for all the categories. img_viz is a RGB image generated based on classes_output

The best way to run some actual test is using test.py. You must specify the image path by changing the path variable.

The pre-trained weights are stored in the ./weights directory.

The Cityscape Dataset 💼

In order to train the model, please download the cityscape dataset, which can be found here.

Remeber to preprocess the data using this jupyter notebook: Data Preprocessing.ipynb. The script will generate train_labels.csv and val_labels.csv

My data is organized as such:

Cityscape
│   train_labels.csv
│   val_labels.csv 
└─── training
│   └─── aachen
│   └─── augsburg
│   .
│ 	.
└─── training_gt
│   └─── aachen
|   └─── augsburg
|	.
|	.
└─── val
│   └─── frankfurt
│   └─── lindau
└─── val_gt
|   └─── frankfurt
|   └─── lindau

Training

There are two training scripts:

  • train_icnet.py
  • train_fusion.py

train.py is the ICNet training script. utils.py contains all the categories (classes). You can modify them based on your dataset.

Models

An overview of the different segmentation models in this project.

ICNet

ICNet (or image cascade network) is a realtime semantic segmentation model developed by Zhao et al. at The Chinese University of Hong Kong. Their paper shows that ICNet can achieve mIoU of ~70% with the Cityscape dataset, while running at ~30 FPS. After some testing, ICNet became a great choice for self-driving applications. (I am currently using the network on my self-driving golf cart project)

Here is a simple benchmark comparison between ICNet and other popular semantic segmentation models. These images and visualizations are from the original ICNet paper, which can be found here.

image

FusionNet 🤔

Coming soon...

About

This project is created for the self-driving golf cart project that I have been working on. For more information on that, please refer to the Github page, or my website.

If you have questions, comments or concerns, please contact me at contact@neilnie.com.

You can’t perform that action at this time.