Experimenting with a newly published deep learning paper and how it can be used for content-based 3D model retrieval. (info retrieval for CAD)
Switch branches/tags
Nothing to show
Clone or download
Latest commit a743169 Apr 2, 2018
Permalink
Failed to load latest commit information.
archived_reference archive some stuff and unarchive other useful things Mar 27, 2018
latent_space_matrices typos Mar 26, 2018
results 5-fold cross validation results in Apr 1, 2018
.gitignore plots look good now Mar 28, 2018
Content Based 3d Model Retrieval.pdf the report Apr 1, 2018
Investigate Airplane Rotational Invariance? .ipynb submitted! Mar 30, 2018
Miscellaneous.ipynb grid search details Mar 31, 2018
ModelNet10_class_imbalance.png plot attempt again Mar 28, 2018
ModelNet40_class_imbalance.png plot attempt again Mar 28, 2018
README.md todo Apr 2, 2018
Refinement and Analysis of Results.ipynb 5-fold cross validation results in Apr 1, 2018
Train Labels.png plots fixed more :-) Mar 28, 2018
Training Class Balance After Upsampling.png class balance plots Mar 28, 2018
Validation Labels.png plots fixed more :-) Mar 28, 2018
binvox_converter.py OFF was in the wrong location in the file for a few classses Mar 21, 2018
binvox_rw_py.py let's call it what they called it Mar 10, 2018
capsulelayers.py WORKING ARCHITECTURE FOR MORE THAN ONE CONVCAPS LAYER! Mar 27, 2018
capsulenet.py WORKING ARCHITECTURE FOR MORE THAN ONE CONVCAPS LAYER! Mar 27, 2018
count_files.py plot attempt again Mar 28, 2018
data.py typos Mar 26, 2018
mac_requirements.txt not using this as a package Mar 17, 2018
modelnet10_arch.py oye vey, cross validation on modelnet40 takes FOREVER Apr 1, 2018
modelnet40_arch.py modelnet40_arch running with same arch as best modelnet10 Mar 30, 2018
plane1.png next to eachother Apr 2, 2018
planes.png next to eachother Apr 2, 2018
plots.py save performance against 90 rotated about z axis Mar 29, 2018
read_results.py grid search details Mar 31, 2018
results.py code is messy, but it's the 11th hour Apr 1, 2018
results_modelnet10.csv latest Mar 31, 2018
results_modelnet40.csv 5-fold cross validation results in Apr 1, 2018
utils.py more appropriate variable names Mar 26, 2018

README.md

Capsule Nets for Content Based 3D Model Retrieval

First (as far as I know) investigation of the use of Capsule Networks for content-based 3d model retrieval. (information retrieval for 3D models)

This project was accomplished in just under 4 weeks. I probably used 120 hours of total compute time split between single GPU and 8 GPU aws instances (ModelNet40 takes FOREVER to train!). The final reported accuracies are Stratified 5-Fold cross validated.

alt text

Paper

Content-Based 3D Model Retrieval

Results

ModelNet10 ModelNet40
Score 2xStderr score 2xStderr
Accuracy 93.08% 1.18% 82.73% 2.12%
Mean Average Precision 88.44% 4.15% 70.10% 2.22%

TODO:

  • clean up code
  • make replicable

*Cleaning up as we speak

this code has gotten pretty messy in the last week and I'm in the middle of cleaning it up. Please bare with me.

Setup

Download the data

cd 3d_model_retriever/
wget http://vision.princeton.edu/projects/2014/3DShapeNets/ModelNet10.zip
unzip ModelNet10.zip
wget http://modelnet.cs.princeton.edu/ModelNet40.zip
unzip ModelNet40.zip

Install binvox binary

In the root of the project

  • MACOSX install:
    wget http://www.patrickmin.com/binvox/mac/binvox\?rnd\=1520896803665325 -O binvox
    chmod 755 binvox
  • Linux install:
    wget http://www.patrickmin.com/binvox/linux64/binvox?rnd=1520896952313989 -O binvox
    chmod 755 binvox

Make env

cd 3d_model_retriever
python3 -m venv .env
source .env/bin/activate
pip install -r mac_requirements.txt

Convert all *.off files to *.binvox

python binvox_convert.py ModelNet10/ --remove-all-dupes
python binvox_convert.py ModelNet40/ --remove-all-dupes

Additional Notes

*.off files

Processing *.off files is done by a 3rd party tool made by Patrick Min.

What the hell is an *.off file?

Voxel Resolution

"To study 3D shape representation, we propose to represent a geometric 3D shape as a probability distribution of binary variables on a 3D voxel grid. Each 3D mesh is represented as a binary tensor: 1 indicates the voxel is inside the mesh surface, and 0 indicates the voxel is outside the mesh (i.e., it is empty space). The grid size in our experiments is 30 × 30 × 30."

./binvox -cb -e -c -d 30 sample.off
  • -e is important, with a lot of troubleshooting it was shown that not using this led to inconsistent voxelization :headache:

** Viewing .binvox data for troubleshooting **

./viewvox <filename>.binvox

Load The Data in Python

from data import load_data
(x_train, y_train), (x_test, y_test), target_names = load_data('./ModelNet10')