This directory contains a Sonnet implementation of the Transporter architecture and a notebook explaining how the model can be used for keypoint inference. To launch the notebook in Google colab, click here.
The Transporter is a neural network architecture for discovering concise geometric object representations in terms of keypoints or image-space coordinates. Our method learns from raw video frames in a fully unsupervised manner, by transporting learnt image features between video frames using a keypoint bottleneck. The discovered keypoints track objects and object parts across long time-horizons more accurately than recent similar methods.
For details, see our paper Unsupervised Learning of Object Keypoints for Perception and Control.
If you use the code here please cite this paper.
Tejas Kulkarni, Ankush Gupta, Catalin Ionescu, Sebastian Borgeaud, Malcolm Reynolds, Andrew Zisserman, Volodymyr Mnih. Unsupervised Learning of Object Keypoints for Perception and Control. NeurIPS 2019. [arXiv].
- Tejas Kulkarni tkulkarni@google.com
- Ankush Gupta ankushgupta@google.com
- Catalin Ionescu
- Sebastian Borgeaud
- Malcolm Reynolds
- Andrew Zisserman
- Volodymyr Mnih
This is not an official Google product.