Skip to content
Go to file

Latest commit


Git stats


Failed to load latest commit information.
Latest commit message
Commit time

The Lyft Perception Challenge

Challenge Overview


Your goal in this challenge is pixel-wise identification of objects in camera images. The segmentation targets are other cars and the drivable area of the road. Images from the CARLA 0.8.2 simulator are used for training and testing in the challenge.

For details see my website:

The solution is based on a LinkNet neural network for semantic segmentation. Loss function is based on a weighted pixel-wise F-eta scores. The approach contains some inference speed up techniques as FPS was an essential part of the competition.


PyTorch 0.4 framework with torchvision was used for the project. Modern Nvidia GPU with CUDA 9.1 with 3 patches installed is required for correct and fast training.

Python 3.6 and several Python modules:


How to train

The whole training pipeline is implemented in Just list train and val folders in the train_dirs and val_dirs lists and run python to recreate the final submission model. Several sample images are provided with the repo, see data. Each dataset folder should contain "CameraRGB" folder with images and a "CameraSeg" folder with segmentation masks from the simulator.

Note: It may take about two days to train on 15601 images in the train dataset and 1500 images in the val dataset with a single Nvidia GTX 1080 Ti GPU.

For experiments and data visualization, use the train.ipynb. It also may be useful to save the hood mask.

How to predict

A client-server approach was used for the inferencestage, that is why one should start a server with correct model path:


One may need to adjust batch size (batch) and number of threads for parallel prediction encoding (n_jobs) for performance maximization on a given hardware setup.

And after it is ready, the client can be used:

# In another terminal
python path/to/the/test/video

It will print out the prediction results as base64 masks encoded with png. To save the predictions as a json file, uncomment lines 12-13 in the Current implementation takes 800x600 px video as an input file.


The 4th place and the fastest solution of the Lyft Perception Challenge (Image semantic segmentation with PyTorch)




No releases published


No packages published
You can’t perform that action at this time.