Skip to content

Learning to match local patches/keypoints from image-level annotations

Notifications You must be signed in to change notification settings

nenadmarkus/wlrn

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Learning Local Descriptors from Weakly-Labeled Data

This repo contains the official implementation of the method from the following paper (arXiv):

{
	skar,
	author = {Nenad Marku\v{s} and Igor S. Pand\v{z}i\'c and J\"{o}rgen Ahlberg},
	title = {{Learning Local Descriptors by Optimizing the Keypoint-Correspondence Criterion: Applications to Face Matching, Learning from Unlabeled Videos and 3D-Shape Retrieval}},
	journal = {IEEE Transactions on Image Processing},
	year = 2019
}

Link to the official version: https://doi.org/10.1109/TIP.2018.2867270.

Some basic information (abstract) is available here.

This work expands our previous ICPR2016 paper (https://doi.org/10.1109/ICPR.2016.7899992) and introduces novel hard-negative mining strategy which significantly improves the discriminative ability of the learned descriptors.

Requirements

To run the code, you will need:

  • PyTorch;
  • a CUDA-capable GPU;
  • OpenCV for Python (needed for training-data preparation).

Training

Follow these steps to learn a descriptor for matching ORB keypoints.

1. Prepare the training data

Run datasets/prepare-ukb-dataset.sh:

bash datasets/prepare-ukb-dataset.sh

This script will download the UKBench dataset.

2. Specify the descriptor-extraction structure

The default model-specification class is models/3x32x32_to_128.py.

You are encouraged to try different architectures as the default one does not perform very well in all settings. However, to learn their parameters, some parts of training scripts might need additional tweaking, such as learning rates.

3. Start the learning script

Finally, learn the parameters of the network by running the traininig script:

python wlrn.py models/3x32x32_to_128.py datasets/tripletgen-ukbench.py --writepath models/3x32x32_to_128.pth

The training should finish in a day or two on a modern GPU (using the --dataparallel flag can reduce training time drastically on a multi-GPU systems).

Pretrained models

Pretrained models can be downloaded here.

License

MIT.

Contact

For any additional information contact me at nenad.markus@fer.hr.

Copyright (c) 2019, Nenad Markus. All rights reserved.

About

Learning to match local patches/keypoints from image-level annotations

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published