LVS-Net: Line Voting with Segmentation for Visual Localization

Introduction

This work is the extension of VS-Net for line segment landmarks. The code is built upon VS-Net.

Briefly, we trained the deep NN to predict both the line segmentation map and the attraction field map from input camera image. We tested our model on Cambridge Landmark Dataset and achieved similar accuracy with VS-Net on 3 out of 5 scenes. Future work includes the improvement of 3D line segment quality and the integration of point and line landmarks for establishing 2D-3D correspondences.

Requirements

We recommend Ubuntu 18.04, cuda 10.0, python 3.7.

conda env create -f environment.yaml
conda activate lvsnet
cd scripts
sh environment.sh
cd squeeze
python setup.py build_ext --inplace

In order to test the model, you also need to install Python bindings of PoseLib.

Data

For reimplementation convenice, we share the pre-processed data on KingsCollege. You can also download the raw Cambrdige Landmarks Dataset from here.

python scripts/cambridge_line_preprocess.py --scene scene_name --root_dir /path/to/data

Training

python train.py --dataset {dataset}_loc --scene {scene_name} --use-aug true --gpu-id gpu_device_id --experiment {experiment_name}

Evaluation

We provide the pretrained model for KingsCollege here.

python test_all.py

Acknowledgements

Thanks Rémi Pautrat and Shaohui Liu for their guidance and supervision.