Skip to content

Commit

Permalink
removed old models, updated README
Browse files Browse the repository at this point in the history
  • Loading branch information
NeilNie committed Mar 18, 2019
1 parent 2cb4a2f commit d46c029
Show file tree
Hide file tree
Showing 44 changed files with 52 additions and 647 deletions.
Binary file modified .DS_Store
Binary file not shown.
74 changes: 52 additions & 22 deletions README.md
@@ -1,58 +1,88 @@
# Semantic Segmentation

🚧 This is still under development 🏗
🚧 This project still under development. Some scripts might be incomplete. 🏗

Semantic segmentation has become an important component of self-driving vehicles. It allows the car to understand the surroundings by classifying every pixel of the input image.

![](./media/image-1.png)

## Running...
## 🏃🏻‍♂️ Running...
To run inference on the pre-trained models, please use `segmentor.py`.

from segmentor import Segmentor
seg = Segmentor()
cls_output, img_viz = seg.semantic_segmentation(image=image, visualization=True)
classes_output, img_viz = seg.semantic_segmentation(image=image, visualization=True)

`cls_output` is the pixel-wise classification result for all the categories. `img_viz` is a RGB image generated based on `cls_output`
`classes_output` is the pixel-wise classification result for all the categories. `img_viz` is a RGB image generated based on `classes_output `

The best way to run some actual test is using `test.py`. You must specify the image path by changing the `path` variable.

The pre-trained weights are stored in the `./weights` directory.

## The Cityscape Dataset 💼

In order to train the model, please download the cityscape dataset, which can be found [here](https://www.cityscapes-dataset.com/).

Remeber to preprocess the data using this jupyter notebook: `Data Preprocessing.ipynb`. The script will generate `train_labels.csv` and `val_labels.csv`

My data is organized as such:

```
Cityscape
│ train_labels.csv
│ val_labels.csv
└─── training
│ └─── aachen
│ └─── augsburg
│ .
│ .
└─── training_gt
│ └─── aachen
| └─── augsburg
| .
| .
└─── val
│ └─── frankfurt
│ └─── lindau
└─── val_gt
| └─── frankfurt
| └─── lindau
```

![](./media/screenshot.png)

## Training

`train.py` is the training script. `utils.py` contains all the categories (classes). You can modify them based on your dataset.
There are two training scripts:

- `train_icnet.py`
- `train_fusion.py`

*training doesn't work very well.* I trying to fix the issue.
`train.py` is the ICNet training script. `utils.py` contains all the categories (classes). You can modify them based on your dataset.

## Models

The goal is to achieve real time semantic segmentation. Therefore, choosing & developing the appropriate model is critical.
I experimented with UNet and ENet, but realized drawbacks to both of these architectures. I eventually landed on ICNet, which
stands for image cascade network
An overview of the different segmentation models in this project.

### ICNet

ICNet (or image cascade network) is a realtime semantic segmentation model developed by Zhao et al. at The Chinese University of Hong Kong. Their paper shows that ICNet can achieve mIoU of ~70% with the Cityscape dataset, while running at ~30 FPS. After some testing, ICNet became a great choice for self-driving applications. (I am currently using the network on my [self-driving golf cart project](https://github.com/sigmaai/self-driving-golf-cart))

![](./media/model_overview.png)

Here is a simple benchmark comparison between ICNet and other popular semantic segmentation models. These images and visualizations
are from the original ICNet paper, which can be found [here](https://arxiv.org/abs/1704.08545).

<center>
<img src="./media/model_comparison.png" alt="image" width="500">
<img src="./media/model_comparison.png" alt="image" width="350">
</center>

## About
### FusionNet 🤔

This project is created for the self-driving golf cart project that I have been working on. For more information on that, please refer to the [Github page](https://github.com/xmeng17/self-driving-golf-cart), or my [website](https://neilnie.com/the-self-driving-golf-cart-project/).
Coming soon...

If you have questions, comments or concerns, please contact me at [contact@neilnie.com](mailto:contact@neilnie.com).

## Others

#### Details about trained models
`./weights/new-enet-5.h5`: 512x512x26

`./weights/enet-c-v1-2.h5`: 640x320x16
## About

`./weights/enet-c-v2-1.h5`: 1024x512x16
This project is created for the self-driving golf cart project that I have been working on. For more information on that, please refer to the [Github page](https://github.com/xmeng17/self-driving-golf-cart), or my [website](https://neilnie.com/the-self-driving-golf-cart-project/).

The channel value correspond to the number of classes.
If you have questions, comments or concerns, please contact me at [contact@neilnie.com](mailto:contact@neilnie.com).
Binary file modified logs/ICNet/.DS_Store
Binary file not shown.
Binary file modified media/.DS_Store
Binary file not shown.
Binary file modified media/image-1.png 100755 → 100644
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added media/screenshot.png
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
3 changes: 0 additions & 3 deletions models/enet_naive_upsampling/LICENSE

This file was deleted.

1 change: 0 additions & 1 deletion models/enet_naive_upsampling/__init__.py

This file was deleted.

Binary file removed models/enet_naive_upsampling/__init__.pyc
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
49 changes: 0 additions & 49 deletions models/enet_naive_upsampling/decoder.py

This file was deleted.

Binary file removed models/enet_naive_upsampling/decoder.pyc
Binary file not shown.
87 changes: 0 additions & 87 deletions models/enet_naive_upsampling/encoder.py

This file was deleted.

Binary file removed models/enet_naive_upsampling/encoder.pyc
Binary file not shown.
47 changes: 0 additions & 47 deletions models/enet_naive_upsampling/model.py

This file was deleted.

Binary file removed models/enet_naive_upsampling/model.pyc
Binary file not shown.
3 changes: 0 additions & 3 deletions models/enet_unpooling/LICENSE

This file was deleted.

Empty file removed models/enet_unpooling/__init__.py
Empty file.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
54 changes: 0 additions & 54 deletions models/enet_unpooling/decoder.py

This file was deleted.

0 comments on commit d46c029

Please sign in to comment.