This GitHub Repository contains the code behind the paper Neural Representation of Open Surfaces. In this work we show how it is possible to encode shapes with boundary curves (open surfaces) within a simple MLP and an associated latent space by representing the shapes as the zero level set of a learned Semi-Signed Distance Field. This can be used to represent a variety of shapes, interpolate between shapes (shown as an interpolation between a pair of pants and a shirt in the GIF to the left), cluster shapes according to similiar topology and geometry (shown in the image to the right) and to do shape completion.
For more details about the project, please visit the project page for this work.
If you find our work useful, please cite the paper:
@article {10.1111:cgf.14916,
journal={Computer Graphics Forum},
title={{Neural Representation of Open Surfaces}},
author={Christiansen, Thor V. and B{\ae}rentzen, Jakob Andreas and Paulsen,
Rasmus R. and Hannemose, Morten R.},
year={2023},
publisher={The Eurographics Association and John Wiley & Sons Ltd.},
ISSN= {1467-8659},
DOI={10.1111/cgf.14916}
}
In order to use the code, different packages are required. Specifically, PyGEL and libIGL. Depending on your system, these libraries can be installed via pip by using the following commands:
pip install PyGEL3D
and
python -m pip install libigl
Alternatively, you can build the libraries yourself. For the PyGEL library a guide is provided here and for the libIGL library, a guide is here.
Moreover, PyTorch is required, which can be installed from here. Lastly, a couple of standard packages are needed, which can be installed by running the following line inside the directory of this code:
pip install -r requirements.txt
In order to use this code, you need to split your files (3D objects) into a training set and a test/inference set. Then you should place the training meshes inside the train folder under experiment -> your_experiment -> data -> train. The same thing goes for the test/inference meshes, which should be put into the test folder at the same location as the train folder namely experiment -> your_experiment -> data -> test.
Note: It is assumed that the meshes are aligned on beforehand (This can be done manually, using landmarks or simple ICP). Moreover, it is assumed that the meshes have been centered around the origin and scaled in such a way that they all fit inside the unit sphere. Lastly, the files should be triangle meshes in .obj file format.
The preprocessing step can be done using the Jupyter Notebook 'Data Preparation' (Data Preparation.ipynb). Simply provide a list with the path and the name of your meshes in the Jupyter Notebook e.g. [path/mesh1.obj, path/mesh2.obj, path/mesh3.obj, ...] for the train files and similarly for the test files. Then all the meshes will be centered around the origin and scaled to fit inside the unit sphere as well as saved in the respective folders (train) and (test). For more information see the Notebook.
Also note: It is assumed that the format of the 3D files is .obj, .ply or .off
It is possible to train two different kinds of networks. One network that learns two signals: SSDF and UDF and another network which approximates the GWN (See the paper for more details). To train the SSDF network, simply go into the folder "your_experiment" and run:
python ../../train.py "your_experiment" "ssdf"
If you want to train the GWN network, just replace "ssdf"' with "gwn"
Similarly, if you want to do inference, go into the folder "your_experiment" and run:
python ../../test.py "your_experiment" "ssdf"
Also, just replace "ssdf"' with "gwn", if you want to do inference for the gwn based network.
When you are done training, you need to reconstruct the meshes in the training samples in order to find the threshold 'k' for the gradient length. Please see the paper for more details.
Running surface reconstruction:
python ../../shape_reconstruction.py "your_experiment" "train" "ssdf"
If you want to reconstruct the surfaces of the training samples for the GWN network, just replace "ssdf"' with "gwn".
Afterwards you should just run:
python ../../find_gwn_threshold.py "your_experiment" "ssdf"
Also, just replace "ssdf"' with "gwn", if you want to find the threshold 'k' for the gwn based network.
Then you can reconstruct the open surfaces (surfaces with boundary curves) of the training samples by running (using either "ssdf"' or "gwn"):
python ../../open_shapes.py "your_experiment" "train" "ssdf"
Having completed the steps above, you can reconstruct the surfaces of the test/inference samples by running (agin using either "ssdf"' or "gwn"):
python ../../shape_reconstruction.py "your_experiment" "test" "ssdf"
And to recover the surfaces of the test/inference samples (the shapes with boundary curves) you need to run:
python ../../open_shapes.py "your_experiment" "test" "ssdf"
If you want to interpolate between two different meshes, go to "your_experiment" and run the following command:
python ../../interpolations.py "your_experiment" "ssdf"' "test" "file1" "file2" n
Here "your_experiment" is the name of the folder in which your data is. Similar to before "ssdf"' can be exchanged with "gwn", if you want to interpolate between two shapes, where the network uses the "gwn" as an implicit surface representation. "file1" is the name of the first shape (without the .obj extension), "file2" is the name of the second file you want to interpolate between (again without the .obj extension) and n is the number of interpolations including the original meshes. Notice: Before running the interpolations script, you need to first reconstruct the shapes in the training set and then find the threshold 'k'.
If you want to change the configurations for your experiment e.g. the number of epochs, the number of points sampled each for each shape etc., batch size etc., go into the "cfgs" folder in the 'your_experiment' folder, find the file 'default.yaml' and change the parameters.
If you have any questions or encounter any problems with the code, please do not hesitate to contact me on tdvc@dtu.dk.