HPointLoc: open dataset and framework for indoor visual localization based on synthetic RGB-D images

This repository provides a novel framework PNTR for exploring the capabilities of a new indoor dataset - HPointLoc, specially designed to explore detection and loop closure capabilities in Simultaneous Localization and Mapping (SLAM).

HPointLoc is based on the popular Habitat simulator from 49 photorealistic indoor scenes from the Matterport3D dataset and contains 76,000 frames.

When forming the dataset, considerable attention was paid to the presence of instance segmentation of scene objects, which will allow it to be used in new emerging semantic methods for place recognition and localization

The dataset is split into two parts: the validation HPointLoc-Val, which contains only one scene, and the complete HPointLoc-All dataset, containing all 49 scenes, including HPointLoc-Val

Download links

HPointLoc dataset is available by the following links (updated on March 3, 2023):

• Whole dataset (HPointLoc-All): disk.yandex
• Val subsection only (HPointLoc-Val): disk.yandex

Experimental results

The experiments were held on the HPointLoc-Val and HPointLoc-ALL datasets.

Quick start to evaluate PNTR pipeline

git clone --recurse-submodules https://github.com/cds-mipt/HPointLoc.git

Download models from https://drive.google.com/drive/folders/192c_XEn12Pz0pmD3aEwV8t3QBqwlBPo1?usp=sharing to PNTR folder
cd HPointLoc
conda env create -f environment.yml
conda activate PTNR_pipeline 
python /path/to/HPointLoc_repo/pipelines/utils/exctracting_dataset.py --dataset_path /path/to/dataset/HPointLoc_dataset
python pipelines/pipeline_evaluate.py --dataset_root /path/to/extracted_dataset --image-retrieval patchnetvlad --keypoints-matching superpoint_superglue --optimizer-cloud teaser