initial commit

ignacio-rocco · Dec 11, 2018 · 8f6839c · 8f6839c
1 parent e7ec716
commit 8f6839c
Show file tree

Hide file tree

Showing 35 changed files with 17,268 additions and 0 deletions.
diff --git a/.gitignore b/.gitignore
@@ -0,0 +1,3 @@
+.ipynb_checkpoints
+__pycache__
+**/.DS_Store
diff --git a/README.md b/README.md
@@ -0,0 +1,82 @@
+# Neighbourhood Consensus Networks
+
+![](https://www.di.ens.fr/willow/research/ncnet/images/teaser.png)
+
+
+## About
+
+This is the implementation of the paper "Neighbourhood Consensus Networks" by I. Rocco, M. Cimpoi, R. Arandjelović, A. Torii, T. Pajdla and J. Sivic.
+
+For more information check out the project [[website](http://www.di.ens.fr/willow/research/ncnet/)] and the paper on [[arXiv](https://arxiv.org/abs/1810.10510)].
+
+
+## Getting started
+
+### Dependencies
+
+The code is implemented using Python 3 and PyTorch 0.3. All dependencies should be included in the standard Anaconda distribution.
+
+### Getting the datasets
+
+The PF-Pascal dataset can be downloaded and unzipped by browsing to the `datasets/pf-pascal/` folder and running `download.sh`.
+
+The IVD dataset (used for training for the InLoc benchmark) can be downloaded and unzipped by browsing to the `datasets/ivd/` folder and first running `make_dirs.sh` and then `download.sh`.
+
+The InLoc dataset (used for evaluation) an be downloaded by browsing to the `datasets/inloc/` folder and running `download.sh`. 
+
+### Getting the trained models
+
+The trained models trained on PF-Pascal (`ncnet_pfpascal.pth.tar`) and IVD (`ncnet_ivd.pth.tar`) can be dowloaded by browsing to the `trained_models/` folder and running `download.sh`.
+
+### Keypoint transfer demo
+
+The demo Jupyter notebook file `point_transfer_demo.py` illustrates how to evaluate the model and use it for keypoint transfer on the PF-Pascal dataset. For this, previously download the PF-Pascal dataset and trained model as indicated above.
+
+### Training
+
+To train a model, run `train.py` with the desired model architecture and the path to the training dataset.
+
+Eg. For PF-Pascal:
+
+```bash
+python main.py --ncons_kernel_sizes 5 5 5 --ncons_channels 16 16 1 --dataset_image_path datasets/pf-pascal --dataset_csv_path datasets/pf-pascal/image_pairs/ 
+```
+
+Eg. For InLoc: 
+
+```bash
+python main.py --ncons_kernel_sizes 3 3 --ncons_channels 16 1 --dataset_image_path datasets/ivd --dataset_csv_path datasets/ivd/image_pairs/ 
+```
+
+### Evaluation on PF-Pascal
+
+Evaluation for PF-Pascal is implemented in the `eval_pf_pascal.py` file. You can run the evaluation in the following way: 
+
+```bash
+python eval_pf_pascal.py --checkpoint trained_models/[checkpoint name]
+```
+
+### Evaluation on InLoc
+
+Evaluation for PF-Pascal is implemented in the `eval_inloc.py` file. You can run the evaluation in the following way: 
+
+```bash
+python eval_inloc.py --checkpoint trained_models/[checkpoint name]
+```
+
+This will generate a series of matches files in the `matches/` folder that then need to be fed to the InLoc evaluation Matlab code. 
+
+
+## BibTeX 
+
+If you use this code in your project, please cite our paper:
+````
+@InProceedings{Rocco18b,
+        author       = "Rocco, I. and Cimpoi, M. and Arandjelovi\'c, R. and Torii, A. and Pajdla, T. and Sivic, J."
+        title        = "Neighbourhood Consensus Networks",
+        booktitle    = "Proceedings of the 32nd Conference on Neural Information Processing Systems",
+        year         = "2018",
+        }
+````
+
+
diff --git a/datasets/inloc/LICENCE b/datasets/inloc/LICENCE
@@ -0,0 +1,35 @@
+InLoc dataset is made available under the Open Database License: http://opendatacommons.org/licenses/odbl/1.0/. Any rights in individual contents of the database are licensed under the Database Contents License: http://opendatacommons.org/licenses/dbcl/1.0/
+It contains information from Wustl Indoor RGBD dataset, which is made available here under the Open Database License http://opendatacommons.org/licenses/odbl/1.0/. 
+If you use our data, please cite our paper: 
+
+@inproceedings{taira2018inloc, 
+  title={{InLoc}: Indoor Visual Localization with Dense Matching and View Synthesis}, 
+  author={Taira, Hajime and Okutomi, Masatoshi and Sattler, Torsten and Cimpoi, Mircea and Pollefeys, Marc and Sivic, Josef and Pajdla, Tomas and Torii, Akihiko}, 
+  booktitle={{CVPR}}, 
+  year={2018} 
+}
+
+and the paper presenting original Wustl Indoor RGBD dataset: 
+
+@inproceedings{wijmans17rgbd,
+  author = {Erik Wijmans and
+            Yasutaka Furukawa},
+  title = {Exploiting 2D Floorplan for Building-scale Panorama RGBD Alignment},
+  booktitle = {Computer Vision and Pattern Recognition, {CVPR}},
+  year = {2017},
+  url = {http://cvpr17.wijmans.xyz/CVPR2017-0111.pdf}
+}
+
+
+Wustl Indoor RGBD dataset is made available under the Open Database License: http://opendatacommons.org/licenses/odbl/1.0/. Any rights in individual contents of the database are licensed under the Database Contents License: http://opendatacommons.org/licenses/dbcl/1.0/
+
+If you use their data, please cite their paper:
+
+@inproceedings{wijmans17rgbd,
+  author = {Erik Wijmans and
+            Yasutaka Furukawa},
+  title = {Exploiting 2D Floorplan for Building-scale Panorama RGBD Alignment},
+  booktitle = {Computer Vision and Pattern Recognition, {CVPR}},
+  year = {2017},
+  url = {http://cvpr17.wijmans.xyz/CVPR2017-0111.pdf}
+}
diff --git a/datasets/inloc/densePE_top100_shortlist_cvpr18.mat b/datasets/inloc/densePE_top100_shortlist_cvpr18.mat
diff --git a/datasets/inloc/download.sh b/datasets/inloc/download.sh
@@ -0,0 +1,2 @@
+wget http://www.ok.sc.e.titech.ac.jp/INLOC/materials/cutouts.tar.gz
+wget http://www.ok.sc.e.titech.ac.jp/INLOC/materials/iphone7.tar.gz
diff --git a/datasets/ivd/.gitignore b/datasets/ivd/.gitignore
@@ -0,0 +1,6 @@
+be/
+cz/
+dk/
+fr/
+gb/
+nl/
diff --git a/datasets/ivd/dirs.txt b/datasets/ivd/dirs.txt
@@ -0,0 +1,89 @@
+be/Brussels/ChIJ0Yp2ZYDDw0cRhsIm9JySp4w
+be/Brussels/ChIJ4Q93mHjEw0cRw_GTsvKSJts
+be/Brussels/ChIJ4UP6DX_Ew0cRekV7szQb5cc
+be/Brussels/ChIJ56wRXYfDw0cRYxIKrhOO91M
+be/Brussels/ChIJ5ZZmOn_Ew0cRudnhY_vjyrM
+be/Brussels/ChIJ7Z2QnYjDw0cRrOzR3G4prM8
+be/Brussels/ChIJa5YBW4fEw0cRXEdEmmrHNzM
+be/Brussels/ChIJBb27mIDDw0cRLFEv29XVFKM
+be/Brussels/ChIJKf3XZ4fDw0cRat9jjYgAuUQ
+be/Brussels/ChIJob4Rz4fDw0cRxUKDwIISIT4
+be/Brussels/ChIJPYlgS4fEw0cRhva9IKvp0iQ
+be/Brussels/ChIJV1d24pnEw0cRPiHMOM4Bv5g
+be/Brussels/ChIJvTZ4_3_Dw0cRzSOE0556hYQ
+be/Brussels/ChIJw7onk4DEw0cRgfFbQNy80II
+be/Brussels/ChIJwQ0CZX_Dw0cRAgyG-NX_zKA
+be/Brussels/ChIJXSgSyXnEw0cRC5dTsT6TDNw
+be/Brussels/ChIJxVcg8nnEw0cR7HSIC7jELKM
+cz/Prague/ChIJ23fPA_CUC0cR8r0Zdb5P65M
+cz/Prague/ChIJ5xu1deqUC0cRABd7DkCu1wI
+cz/Prague/ChIJ97RdLIOUC0cRw6synCh9ywk
+cz/Prague/ChIJB2XzV_KUC0cR2XDxkOqS2IE
+cz/Prague/ChIJgXj2so-UC0cR6fYoBN3omkc
+cz/Prague/ChIJi1yKt5KUC0cRd7MVbEd2xi8
+cz/Prague/ChIJj5XMEO6UC0cRSdD6zoOKd3k
+cz/Prague/ChIJJQeVZI-UC0cREeV8jNifDCs
+cz/Prague/ChIJK9mieYGUC0cR0Ynne3yTYB4
+cz/Prague/ChIJpdPa4JSUC0cRL2SbDI32MKQ
+cz/Prague/ChIJqW2dWIiUC0cRlRmF5kfyZCE
+cz/Prague/ChIJQxyaBeGUC0cRZjpStOMuVMs
+cz/Prague/ChIJS7Lkm52UC0cRrwqLxO9gZDo
+cz/Prague/ChIJu2niIWeTC0cRHEJGY5xf_Io
+cz/Prague/ChIJua9Ogo2UC0cRGmG8vsSar_I
+cz/Prague/ChIJW08COB6VC0cRoonZ8lA4rWk
+cz/Prague/ChIJwbOybpWUC0cR0Ie0VH2PcqY
+cz/Prague/ChIJXXFEO-KUC0cRXxm7CSQYrzM
+dk/Copenhagen/ChIJ36KzuxVTUkYRDDCgC3HCTGw
+dk/Copenhagen/ChIJ6a9VvnNTUkYR70EJ8UFFlv0
+dk/Copenhagen/ChIJb1Q_gRBTUkYRBX_THfQTZqk
+dk/Copenhagen/ChIJB57HZxBTUkYRognketO_9qs
+dk/Copenhagen/ChIJCbqRvxZTUkYRGWI2eOd3tiw
+dk/Copenhagen/ChIJfXQRBBNTUkYRoec4408un1k
+dk/Copenhagen/ChIJGWMBpQ1TUkYRzYR5oWDqDhM
+dk/Copenhagen/ChIJMTzE6yFTUkYR90_sHEhzRqc
+dk/Copenhagen/ChIJoxxTLRFTUkYRJaF1B3HO14Y
+dk/Copenhagen/ChIJPQeIbRdTUkYRs36mNmYz77Q
+dk/Copenhagen/ChIJrfd4pHNTUkYRxMf6Mns9ZPk
+dk/Copenhagen/ChIJU3W3ZBFTUkYRR1sx91IE5FM
+dk/Copenhagen/ChIJvYWnuhhTUkYR3m95N7vDrmo
+dk/Copenhagen/ChIJwdrCyAVTUkYRLM0WFavcxfY
+dk/Copenhagen/ChIJxYb1Og5TUkYRWKm9JRjAiN8
+dk/Copenhagen/ChIJy84GHRFTUkYRY-FdhFM-nKI
+dk/Copenhagen/ChIJye06RjlTUkYRK7RPt-oBGME
+fr/Paris/ChIJ3_e_BiVu5kcRrhLcUCoJGnE
+fr/Paris/ChIJ4bki89px5kcRVMlK4LFJbrU
+fr/Paris/ChIJ65TvOyJw5kcRaLuvK3Qgz7E
+fr/Paris/ChIJ_7ZXHuFx5kcRs1LnTeCsOBM
+fr/Paris/ChIJ9wPDGP1t5kcRuU-2tWgnWOQ
+fr/Paris/ChIJ9XyV5B5u5kcRA2PRdiGh4uY
+fr/Paris/ChIJAVjEb9tx5kcRxWEv0cQd29s
+fr/Paris/ChIJG5Qwtitu5kcR2CNEsYy9cdA
+fr/Paris/ChIJjfAnTnFl5kcR0nn0H9Jwpz0
+fr/Paris/ChIJk0iY0udx5kcRzQmiht_HW6g
+fr/Paris/ChIJoyC4CRxu5kcRRTPcWX5srLc
+fr/Paris/ChIJqZ1K4UYN5kcR8GlGQjFFvzw
+fr/Paris/ChIJryQ0vfxx5kcRwTFL9nbYZTU
+fr/Paris/ChIJXUi-ScJv5kcRaUX9yJaj4ys
+gb/Edinburgh/ChIJ1U2TFYbHh0gRLC_ZKcPAopo
+gb/Edinburgh/ChIJ68bm6kjGh0gRkyF0XlT5Rww
+gb/Edinburgh/ChIJ7xHasgG4h0gRdMGn6H6V9e8
+gb/Edinburgh/ChIJBXwTKpHHh0gRcN4_XRiKV7I
+gb/Edinburgh/ChIJgYCnl3e4h0gRnTr0EPT_2zo
+gb/Edinburgh/ChIJIeS60YTHh0gRw7W-8gMnzIg
+gb/Edinburgh/ChIJJ21nqojHh0gRqelG4UPQbXU
+gb/Edinburgh/ChIJQRw3B5HHh0gRGokny3GM2JY
+gb/Edinburgh/ChIJsbrmBZHHh0gRzJntEk_0L50
+gb/Edinburgh/ChIJU0zUuprHh0gR7w-z7QSk85E
+gb/Edinburgh/ChIJW9Jx15HHh0gRl1m9r1vbfS8
+nl/Amsterdam/ChIJ5Ra7we4JxkcRhYVAaq5zQ9U
+nl/Amsterdam/ChIJ74XcqrgJxkcR9g0UqSG4KPU
+nl/Amsterdam/ChIJO5oH0t0JxkcRKkXlC7IYHMA
+nl/Amsterdam/ChIJS4P1Hb4JxkcRIphGLOfikzU
+nl/Amsterdam/ChIJSxklPO0JxkcRCqxBkavK008
+nl/Amsterdam/ChIJtWMWEeAJxkcR7PV7zX9aRh0
+nl/Amsterdam/ChIJU2qVoLkJxkcRB8mgtNMmmEc
+nl/Amsterdam/ChIJU9-Lo94JxkcRBzlvVAQV7n8
+nl/Amsterdam/ChIJV-jqPbkJxkcRVwprv5lxPkg
+nl/Amsterdam/ChIJX1rTlu8JxkcRGsV8-a4oKMI
+nl/Amsterdam/ChIJx_aU48UJxkcRxOEtx6d5Lng
+nl/Amsterdam/ChIJySp1EbkJxkcRc3bwx1UJxcE
diff --git a/datasets/ivd/download.sh b/datasets/ivd/download.sh
@@ -0,0 +1,2 @@
+<urls.txt xargs -n2 -P8 wget -O
+