This repository is a PyTorch implementation of Memory-based semantic segmentation for off-road unstructured natural environments. This work is based on semseg.
The codebase mainly uses ResNet18, ResNet50 and MobileNet-V2 as backbone with ASPP module and can be easily adapted to other basic semantic segmentation structures.
Sample experimented dataset is RUGD.
Hardware: >= 11G GPU memory
Software: PyTorch>=1.0.0, python3
For installation, follow installation steps below or recommend you to refer to the instructions described here.
For its pretrained ResNet50 backbone model, you can download from URL.
- Clone this repository.
git clone https://github.com/youngsjjn/MemSeg.git
- Install Python dependencies.
pip install -r requirements.txt
Download data list of RUGD here.
- Inference
If you want to inference on pretrained models, download pretrained network in my drive and save them in
./exp/rugd/
.
Inference "ResNet50 + Deeplabv3" without the memory module
sh tool/test.sh rugd deeplab50
Inference "ResNet50 + Deeplabv3" with the memory module
sh tool/test_mem.sh rugd deeplab50mem
Network | mIoU |
---|---|
ResNet18 + PSPNet | 33.42 |
ResNet18 + PSPNet (Memory) | 34.13 |
ResNet18 + Deeplabv3 | 33.48 |
ResNet18 + Deeplabv3 (Memory) | 35.07 |
ResNet50 + Deeplabv3 | 36.77 |
ResNet50 + Deeplabv3 (Memory) | 37.71 |
- Train (Evaluation is included at the end of the training) Train "ResNet50 + Deeplabv3" without the memory module
sh tool/train.sh rugd deeplab50
Train "ResNet50 + Deeplabv3" without the memory module
sh tool/train_mem.sh rugd deeplab50mem
Here, the example is for training or testing on "ResNet50 + Deeplabv3". If you want to train other networks, please change "deeplab50" or "deeplab50mem" as a postfix of a config file name.
For example, train "ResNet18 + PSPNet" with the memory module:
sh tool/train_mem.sh rugd pspnet18mem
If you like our work and use the code or models for your research, please cite our work as follows.
@article{DBLP:journals/corr/abs-2108-05635,
author = {Youngsaeng Jin and
David K. Han and
Hanseok Ko},
title = {Memory-based Semantic Segmentation for Off-road Unstructured Natural
Environments},
journal = {CoRR},
volume = {abs/2108.05635},
year = {2021},
url = {https://arxiv.org/abs/2108.05635},
eprinttype = {arXiv},
eprint = {2108.05635},
timestamp = {Wed, 18 Aug 2021 19:45:42 +0200},
biburl = {https://dblp.org/rec/journals/corr/abs-2108-05635.bib},
bibsource = {dblp computer science bibliography, https://dblp.org}
}