Skip to content


Switch branches/tags

Name already in use

A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch?

Latest commit


Git stats


Failed to load latest commit information.


This is a PyTorch implementation of "MultiPoint: Cross-spectral registration of thermal and optical aerial imagery"


This software requires a base python install of 3.6 or higher. Requirements can be installed with:

pip3 install -r requirements.txt

The repository provides models with pretrained weights for MultiPoint but for training the dataset needs to be downloaded separately (see Dataset).

The multipoint python package can be locally installed by executing:

pip3 install -e . --user

(You can remove the --user flag if operating in a virtual environment)


The dataset is hosted at the Autonomous Systems Lab dataset website. The page also provides some basic information about the data itself.

The dataset can be downloaded by running (from the multipoint directory):


A different target directory can be specified with the -d flag, but please remember to update the configuration yaml files to point to the new location (or create symlinks). Force overwrite of existing files can be set with -f flag. Note that the files are quite large (over 36 GB total), so may take some time to download.


In the following section the scripts to train and visualize the results of MultiPoint are explained. For each script, additional help on the input paramaters and flags can be found using the -h flag (e.g. python -h).

Generating Keypoint Labels

Keypoint labels for a given set of image pairs can be generated using:

python3 -o /tmp/labels.hdf5

where the -o flag defines the output filename. The base detector and the export settings can be modified by making a copy of the configs/config_export_keypoints.yaml config file, editing the desired parameters, and specifying your new config file with the -y flag.

python3 -y configs/custom_export_keypoints.yaml -o /tmp/labels.hdf5

Visualizing Keypoint Labels

The generated keypoint labels can be inspected by executing the script:

python3 -d /tmp/training.hdf5 -k /tmp/labels.hdf5 -n 100

The -d flag specifies the dataset file, the -k flag the labels file, and the -n flag the index of the sample which is shown.

Visualizing Synthetic Shapes Samples

A randomly generated sample of the synthetic shapes dataset can be visualized by executing the script (any key to exit).

Visualizing Samples from Datasets

By executing the following command:

python3 -i /tmp/test.hdf5 -n 100

the 100th image pair of the /tmp/test.hdf5 dataset is shown.

Training MultiPoint

MultiPoint can be trained by executing the script. All that script requires is a path to a yaml file with the training parameters:

python3 -y configs/config_multipoint_training.yaml

The hyperparameter for the training, e.g. learning rate, can be modified in the yaml file.

Predicting Keypoints

Predicting only keypoints can be done executing the script. The results are plotted by adding the -p flags and the metrics for the whole dataset are computed by adding the -e flag.

Predicting the Homography

Predicting the alignment of an image pair can be done using the script. The resulting keypoints and matches can be visualized by adding the -p flag. The metrics over the full dataset are computed when adding the -e flag.


This work is presented in the the following paper, please use this citation to cite our work:

    author={Achermann, Florian and Kolobov, Andrey and Dey, Debadeepta and Hinzmann, Timo and Chung, Jen Jen and Siegwart, Roland and Lawrance, Nicholas},
    title={{MultiPoint}: Cross-spectral registration of thermal and optical aerial imagery},
    booktitle={Conference on Robot Learning (CoRL)},


This research was funded by the Microsoft Swiss Joint Research Center under Project Altair: Infrared Vision and AI Decision-Making for Longer Drone Flights.

This code is based on previous implementations of SuperPoint by Rémi Pautrat and You-Yi Jau


MultiPoint: Feature detector and descriptor for optical/thermal image alignment






No releases published


No packages published