Multi-Task Training Example
In this example, we are going to train Multi-Task Light-Weight RefineNet for joint semantic segmentation and depth estimation. Note that inference examples together with pre-trained weights can be found in the official repository.
The hyperparameters set here are not the same as used in the corresponding paper, hence the results will differ. Please refer to the paper below for more information on the model and the training regime.
Real-Time Joint Semantic Segmentation and Depth Estimation Using Asymmetric Annotations Vladimir Nekrasov, Thanuja Dharmasiri, Andrew Spek, Tom Drummond, Chunhua Shen, Ian Reid In ICRA 2019
Considering that you successfully installed the
DenseTorch package, the next step is to download the NYUDv2 dataset with segmentation and depth masks. The dataset can be downloaded by following the link.
After downloading and unpacking the archive, create the
datasets folder and link the
nyudv2 directory in the archive
to the just created folder:
mkdir datasets ln -s /path/to/nyudv2 ./datasets/
Now you are ready to run the example script. To do so, simply execute
python train.py. After it is finished, the
best model will be stored in the corresponding
pth.tar file. Note that it would the model that improves upon the
previous checkpoint both in terms of mean IoU (for segmentation) and linear RMSE (for depth estimation).