Capsule Nets for Content Based 3D Model Retrieval
First (as far as I know) investigation of the use of Capsule Networks for content-based 3d model retrieval. (information retrieval for 3D models)
This project was accomplished in just under 4 weeks. I probably used 120 hours of total compute time split between single GPU and 8 GPU aws instances (ModelNet40 takes FOREVER to train!). The final reported accuracies are Stratified 5-Fold cross validated.
|Mean Average Precision||88.44%||4.15%||70.10%||2.22%|
- clean up code
- make replicable
*Cleaning up as we speak
this code has gotten pretty messy in the last week and I'm in the middle of cleaning it up. Please bare with me.
Download the data
cd 3d_model_retriever/ wget http://vision.princeton.edu/projects/2014/3DShapeNets/ModelNet10.zip unzip ModelNet10.zip wget http://modelnet.cs.princeton.edu/ModelNet40.zip unzip ModelNet40.zip
Install binvox binary
In the root of the project
- MACOSX install:
wget http://www.patrickmin.com/binvox/mac/binvox\?rnd\=1520896803665325 -O binvox chmod 755 binvox
- Linux install:
wget http://www.patrickmin.com/binvox/linux64/binvox?rnd=1520896952313989 -O binvox chmod 755 binvox
cd 3d_model_retriever python3 -m venv .env source .env/bin/activate pip install -r mac_requirements.txt
Convert all *.off files to *.binvox
python binvox_convert.py ModelNet10/ --remove-all-dupes python binvox_convert.py ModelNet40/ --remove-all-dupes
Processing *.off files is done by a 3rd party tool made by Patrick Min.
What the hell is an *.off file?
"To study 3D shape representation, we propose to represent a geometric 3D shape as a probability distribution of binary variables on a 3D voxel grid. Each 3D mesh is represented as a binary tensor: 1 indicates the voxel is inside the mesh surface, and 0 indicates the voxel is outside the mesh (i.e., it is empty space). The grid size in our experiments is 30 × 30 × 30."
./binvox -cb -e -c -d 30 sample.off
- -e is important, with a lot of troubleshooting it was shown that not using this led to inconsistent voxelization :headache:
** Viewing .binvox data for troubleshooting **
Load The Data in Python
from data import load_data (x_train, y_train), (x_test, y_test), target_names = load_data('./ModelNet10')