Skip to content
Go to file

Capsule Nets for Content Based 3D Model Retrieval

First (as far as I know) investigation of the use of Capsule Networks for content-based 3d model retrieval. (information retrieval for 3D models)

This project was accomplished in just under 4 weeks. I probably used 120 hours of total compute time split between single GPU and 8 GPU aws instances (ModelNet40 takes FOREVER to train!). The final reported accuracies are Stratified 5-Fold cross validated.

alt text


Content-Based 3D Model Retrieval


ModelNet10 ModelNet40
Score 2xStderr score 2xStderr
Accuracy 93.08% 1.18% 82.73% 2.12%
Mean Average Precision 88.44% 4.15% 70.10% 2.22%


  • clean up code
  • make replicable

*Cleaning up as we speak

this code has gotten pretty messy in the last week and I'm in the middle of cleaning it up. Please bare with me.


Download the data

cd 3d_model_retriever/

Install binvox binary

In the root of the project

  • MACOSX install:
    wget\?rnd\=1520896803665325 -O binvox
    chmod 755 binvox
  • Linux install:
    wget -O binvox
    chmod 755 binvox

Make env

cd 3d_model_retriever
python3 -m venv .env
source .env/bin/activate
pip install -r mac_requirements.txt

Convert all *.off files to *.binvox

python ModelNet10/ --remove-all-dupes
python ModelNet40/ --remove-all-dupes

Additional Notes

*.off files

Processing *.off files is done by a 3rd party tool made by Patrick Min.

What the hell is an *.off file?

Voxel Resolution

"To study 3D shape representation, we propose to represent a geometric 3D shape as a probability distribution of binary variables on a 3D voxel grid. Each 3D mesh is represented as a binary tensor: 1 indicates the voxel is inside the mesh surface, and 0 indicates the voxel is outside the mesh (i.e., it is empty space). The grid size in our experiments is 30 × 30 × 30."

./binvox -cb -e -c -d 30
  • -e is important, with a lot of troubleshooting it was shown that not using this led to inconsistent voxelization :headache:

** Viewing .binvox data for troubleshooting **

./viewvox <filename>.binvox

Load The Data in Python

from data import load_data
(x_train, y_train), (x_test, y_test), target_names = load_data('./ModelNet10')


Experimenting with a newly published deep learning paper and how it can be used for content-based 3D model retrieval. (info retrieval for CAD)




No releases published


No packages published