README update

README.md

@@ -3,13 +3,14 @@
 </p>
 
 
-## Kaolin: Accelerating 3D deep learning research
+## Kaolin: A PyTorch Library for Accelerating 3D Deep Learning Research
 
 **[Documentation](http://nvidiagameworks.github.io/kaolin)** | **[Paper](https://arxiv.org/abs/1911.05063)**
 
 
 Kaolin is a PyTorch library aiming to accelerate 3D deep learning research. Kaolin provides efficient implementations of differentiable 3D modules for use in deep learning systems. With functionality to load and preprocess several popular 3D datasets, and native functions to manipulate meshes, pointclouds, signed distance functions, and voxel grids, Kaolin mitigates the need to write wasteful boilerplate code. Kaolin packages together several differentiable graphics modules including rendering, lighting, shading, and view warping. Kaolin also supports an array of loss functions and evaluation metrics for seamless evaluation and provides visualization functionality to render the 3D results. Importantly, we curate a comprehensive model zoo comprising many state-of-the-art 3D deep learning architectures, to serve as a starting point for future research endeavours.
 
+> *Fun fact*: The name _Kaolin_, it’s from Kaolinite, a form of plasticine (clay) that is sometimes used in 3D modeling.
 
 
 ## Table of Contents
@@ -34,17 +35,19 @@ Kaolin is a PyTorch library aiming to accelerate 3D deep learning research. Kaol
 Currently, the (beta) release contains several processing functions for 3D deep learning on meshes, voxels, signed distance functions, and pointclouds. Loading of several popular datasets (eg. ShapeNet, ModelNet, SHREC) are supported out-of-the-box. We also implement several 3D conversion and transformation operations (both within and across the aforementioned representations). 
 
 Kaolin supports several 3D tasks such as:
-* Differentiable renderers (Neural Mesh Renderer, Soft Rasterizer, Differentiable Interpolation-based Renderer, and a modular and extensible abstract DifferentiableRenderer specification).
-* Single-image based mesh reconstruction (Pixel2Mesh, GEOMetrics, OccupancyNets, and more...)
-* Pointcloud classification and segmentation (PointNet, PoinNet++, DGCNN, ...)
-* Mesh classification and segmentation
-* 3D superresolution on voxel grids
+* Differentiable rendering (see [Neural Mesh Renderer](https://github.com/hiroharu-kato/neural_renderer), its [PyTorch port](https://github.com/daniilidis-group/neural_renderer), [Soft Rasterizer](https://github.com/ShichenLiu/SoftRas), [Differentiable Interpolation-based Renderer](https://nv-tlabs.github.io/DIB-R/), and a modular and extensible abstract DifferentiableRenderer specification).
+* Single-image based mesh reconstruction ([Pixel2Mesh](https://github.com/nywang16/Pixel2Mesh), [GEOMetrics](https://github.com/EdwardSmith1884/GEOMetrics), [OccupancyNets](https://github.com/autonomousvision/occupancy_networks), and more...)
+* Pointcloud classification and segmentation ([PointNet](https://github.com/fxia22/pointnet.pytorch), [PoinNet++](https://github.com/erikwijmans/Pointnet2_PyTorch), [DGCNN](https://github.com/muhanzhang/pytorch_DGCNN), ...)
+* Mesh classification and segmentation ([MeshCNN](https://github.com/ranahanocka/MeshCNN), [GCN](https://github.com/tkipf/pygcn))
+* 3D superresolution on voxel grids ([ODM](https://github.com/EdwardSmith1884/Multi-View-Silhouette-and-Depth-Decomposition-for-High-Resolution-3D-Object-Representation), VoxelUNet, and more...)
 * Basic graphics utilities (lighting, shading, etc.)
 
 ## Model Zoo
 
 Kaolin curates a large _model zoo_ containing reference implementations of popular 3D DL architectures. Head over [here](kaolin/models) to check them out.
 
+> *NOTE*: For several of these models, the implementation is due to the original authors. We build a bridge to our library and wherever possible, we introduce optimization. If you use any of the models in the model zoo, or the graphics packages (eg. differentiable renderers such as [NMR](https://arxiv.org/abs/1711.07566), [SoftRas](https://arxiv.org/abs/1904.01786), [DIB-R](https://arxiv.org/abs/1908.01210)), please cite the original papers in addition to Kaolin. For convenience, BibTeX citation formats for each of the original papers are included in the documentation for each model provided.
+
 
 ## Installation and Usage
 
@@ -85,7 +88,7 @@ To verify that `kaolin` has been installed, fire up your python interpreter, and
 
 ```python
 >>> import kaolin as kal
->>> print(kal.__version)
+>>> print(kal.__version__)
 ```
 
 ### Building the Documentation
@@ -111,24 +114,29 @@ $ pytest tests/
 
 - **conversions**: Supports conversion across all popular 3D representations.
 
-- **models**: Implemented models include: 
-    - DGCNN (https://arxiv.org/abs/1801.07829v1)
-    - DIB-R (https://arxiv.org/abs/1908.01210)
-    - GEOMetrics (https://arxiv.org/abs/1901.11461)
-    - Image2Mesh (https://arxiv.org/abs/1711.10669)
-    - Occupancy Network (https://arxiv.org/abs/1812.03828)
-    - Pixel2Mesh (https://arxiv.org/abs/1804.01654)
-    - PointNet (https://arxiv.org/abs/1612.00593)
-    - PointNet++ (https://arxiv.org/abs/1706.02413)
+- **models**: Provided models include the following. For each implementation, we also provide a link to the *original implementation which it was ported from* 
+    - DGCNN [[Paper](https://arxiv.org/abs/1801.07829v1)], [[Original implementation](https://github.com/WangYueFt/dgcnn)]
+    - DIB-R [[Paper](https://arxiv.org/abs/1908.01210)], [[Original implementation](https://github.com/nv-tlabs/DIB-R/)]
+    - GEOMetrics [[Paper](https://arxiv.org/abs/1901.11461)], [[Original implementation](https://github.com/EdwardSmith1884/GEOMetrics)]
+    - Image2Mesh [[Paper](https://arxiv.org/abs/1711.10669)], [[Original implementation](https://github.com/jhonykaesemodel/image2mesh)]
+    - Occupancy Network [[Paper](https://arxiv.org/abs/1812.03828)], [[Original implementation](https://github.com/autonomousvision/occupancy_networks)]
+    - Pixel2Mesh [[Paper](https://arxiv.org/abs/1804.01654)], [[Original implementation (TensorFlow)](https://github.com/nywang16/Pixel2Mesh)], [[Re-implementation that we borrow from](https://github.com/EdwardSmith1884/GEOMetrics)]
+    - PointNet [[Paper](https://arxiv.org/abs/1612.00593)], [[Original implementation](https://github.com/charlesq34/pointnet)], [[Re-implementation (we borrow from here)](https://github.com/fxia22/pointnet.pytorch/)]
+    - PointNet++ [[Paper](https://arxiv.org/abs/1706.02413)], [[Original implementation (we borrow  from here)](https://github.com/charlesq34/pointnet2)], [[Re-implementation (we borrow from here)](https://github.com/erikwijmans/Pointnet2_PyTorch)]
     - MeshEncoder: A simple mesh encoder architecture.
     - GraphResNet: MeshEncoder with residual connections.
-    - OccupancyNetworks (https://arxiv.org/abs/1812.03828)
-    - And many more!
+    - OccupancyNetworks [[Paper](https://arxiv.org/abs/1812.03828)], [[Original implementation](https://github.com/autonomousvision/occupancy_networks)]
+    - MeshCNN: [[Paper](https://arxiv.org/abs/1809.05910)], [[Original implementation](https://github.com/ranahanocka/MeshCNN)]
+    - VoxelGAN [[Paper](http://3dgan.csail.mit.edu/papers/3dgan_nips.pdf)], [[Original implementation](https://github.com/zck119/3dgan-release)]
+    - AtlasNet [[Paper](https://arxiv.org/abs/1802.05384)], [[Original implementation](https://github.com/ThibaultGROUEIX/AtlasNet)]
+    - And many more to come!
+
+> *NOTE*: For several of these models, the implementation is due to the original authors. We build a bridge to our library and wherever possible, we introduce optimization. If you use any of the models in the model zoo, or the graphics packages (eg. differentiable renderers such as [NMR](https://arxiv.org/abs/1711.07566), [SoftRas](https://arxiv.org/abs/1904.01786), [DIB-R](https://arxiv.org/abs/1908.01210)), please cite the original papers in addition to Kaolin. For convenience, BibTeX citation formats for each of the original papers are included in the documentation for each model provided.
 
 - **graphics**: Kaolin provides a flexible and modular framework for building differentiable renderers, making it simple to replace individual components with new ones. Kaolin also provides implementations of the following differentiable renderers:
-    - DIB-R (https://arxiv.org/abs/1908.01210)
-    - SoftRas (https://arxiv.org/abs/1904.01786)
-    - Neural 3D Mesh Renderer (https://arxiv.org/abs/1711.07566)
+    - DIB-R [[Paper](https://arxiv.org/abs/1908.01210)], [[Original implementation (we borrow from here)](https://github.com/nv-tlabs/DIB-R/)]
+    - SoftRas [[Paper](https://arxiv.org/abs/1904.01786)], [[Original implementation (we borrow from here)](https://github.com/ShichenLiu/SoftRas)]
+    - Neural 3D Mesh Renderer [[Paper](https://arxiv.org/abs/1711.07566)], [[Original Chainer implementation](https://github.com/hiroharu-kato/neural_renderer)], [[PyTorch re-implementation (we borrow from here)](https://github.com/daniilidis-group/neural_renderer)]
 
 - **metrics**: Implemented metrics and loss functions:
     - Mesh: Triangle Distance, Chamfer Distance, Edge Length regularization, Laplacian regularization, Point to Surface distance, Normal consistency
@@ -176,5 +184,25 @@ If you find this library useful, consider citing the following paper:
     journal = {arXiv:1911.05063},
     year = {2019},
 }
+```
 
-```
+## Contributors for borrowed sources
+
+Here is a list of all authors on relevant research papers that Kaolin borrows code from. Without the efforts of these folks (and their willingness to release their implementations under permissable copyleft licenses), Kaolin would not have been possible.
+
+* Kornia: Edgar Riba, Dmytro Mishkin, Daniel Ponsa. [[Paper](https://arxiv.org/pdf/1910.02190.pdf)] [[Code](https://github.com/kornia/kornia)]
+* Occupancy Networks: Lars Mescheder, Michael Oechsle, Michael Niemeyer, Sebastian Nowozin, Andreas Geiger. [[Paper](https://avg.is.tuebingen.mpg.de/publications/occupancy-networks)] [[Code](https://github.com/autonomousvision/occupancy_networks)]
+* Multi-View Silhouette and Depth Decomposition for High Resolution 3D Object Representation: Edward Smith, Scott Fujimoto, David Meger. [[Paper](https://papers.nips.cc/paper/7883-multi-view-silhouette-and-depth-decomposition-for-high-resolution-3d-object-representation.pdf)] [[Code](https://github.com/EdwardSmith1884/Multi-View-Silhouette-and-Depth-Decomposition-for-High-Resolution-3D-Object-Representation)]
+* Pytorch Chamfer Distance: Christian Diller. [[Code](https://github.com/chrdiller/pyTorchChamferDistance)]
+* GEOMetrics: Edward Smith, Scott Fujimoto, Adriana Romero, David Meger. [[Paper](https://arxiv.org/abs/1901.11461)] [[Code](https://github.com/EdwardSmith1884/GEOMetrics)]
+* DeepSDF: Jeong Joon Park, Peter Florence, Julian Straub, Richard Newcombe, Steven Lovegrove. [[Paper](http://openaccess.thecvf.com/content_CVPR_2019/html/Park_DeepSDF_Learning_Continuous_Signed_Distance_Functions_for_Shape_Representation_CVPR_2019_paper.html)] [[Code](https://github.com/facebookresearch/DeepSDF)]
+* PointGAN: Fei Xia. [[Code](https://github.com/fxia22/pointGAN)]
+* AtlasNet: Thibault Groueix, Matthew Fisher, Vladimir G. Kim, Bryan C. Russell, Mathieu Aubry. [[Paper](https://arxiv.org/abs/1802.05384)] [[Code](https://github.com/ThibaultGROUEIX/AtlasNet)]
+* PointNet: Charles R. Qi, Hao Su, Kaichun Mo, Leonidas J. Guibas. Also, Fei Xia (reimplementation). [[Paper](https://arxiv.org/abs/1612.00593)] [[Code](https://github.com/fxia22/pointnet.pytorch)]
+* MeshCNN: Rana Hanocka, Amir Hertz, Noa Fish, Raja Giryes, Shachar Fleishman, Daniel Cohen-Or. [[Paper](https://arxiv.org/abs/1809.05910)] [[Code](https://github.com/ranahanocka/MeshCNN)]
+* DGCNN: Muhan Zhang, Zhicheng Cui, Marion Neumann, Yixin Chen. [[Paper](https://www.cse.wustl.edu/~muhan/papers/AAAI_2018_DGCNN.pdf)] [[Code](https://github.com/muhanzhang/pytorch_DGCNN)]
+* Neural 3D Mesh Renderer: Hiroharu Kato, Yoshitaka Ushiku, Tatsuya Harada. Also, Nikos Kolotouros (for reimplementation). [[Paper](https://arxiv.org/abs/1711.07566)] [[Code](https://github.com/daniilidis-group/neural_renderer)]
+* SoftRasterizer: Shichen Liu, Tianye Li, Weikai Chen, Hao Li. [[Paper](https://arxiv.org/abs/1904.01786)] [[Code](https://github.com/ShichenLiu/SoftRas)]
+* DIB-R: Wenzheng Chen, Jun Gao, Huan Ling, Edward J. Smith, Jaakko Lehtinen, Alec Jacobson, Sanja Fidler. [[Paper](https://arxiv.org/abs/1908.01210)] [[Code](https://github.com/nv-tlabs/DIB-R)]
+* PointNet++: Charles R. Qi, Li (Eric) Yi, Hao Su, Leonidas J. Guibas. Also, Erik Wijmans (reimplementation). [[Paper](https://arxiv.org/abs/1706.02413)] [[Code](https://github.com/erikwijmans/Pointnet2_PyTorch)]
+* Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling': Jiajun Wu, Chengkai Zhang, Tianfan Xue, William T. Freeman, Joshua B. Tenenbaum. [[Paper](http://arxiv.org/abs/1610.07584)] [[Code](https://github.com/zck119/3dgan-release)]

docs/modules/models.rst

@@ -18,3 +18,21 @@ kaolin.models
 .. autofunction:: models.Voxel3DIWGAN.Voxel3DIWDiscriminator
 .. autofunction:: models.VoxelSuperresODM.SuperresNetwork
 .. autofunction:: models.VoxelSuperresSimple.EncoderDecoder
+.. automodule:: kaolin.models.DIBREncoder
+    :members:
+.. automodule:: kaolin.models.GEOMetrics
+    :members:
+.. automodule:: kaolin.models.GraphResNet
+    :members:
+.. automodule:: kaolin.models.Image2MeshReconstructionBaseline
+    :members:
+.. automodule:: kaolin.models.MeshEncoder
+    :members:
+.. automodule:: kaolin.models.OccupancyNetwork
+    :members:
+.. automodule:: kaolin.models.Pixel2Mesh
+    :members:
+.. automodule:: kaolin.models.SimpleGCN
+    :members:
+.. automodule:: kaolin.models.VGG18
+    :members:

kaolin/graphics/softras/__init__.py

@@ -1,3 +1,27 @@
+# Soft Rasterizer (SoftRas)
+# 
+# Copyright (c) 2017 Hiroharu Kato
+# Copyright (c) 2018 Nikos Kolotouros
+# Copyright (c) 2019 Shichen Liu
+# 
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+
 from . import functional
 from .mesh import Mesh
 from .renderer import Renderer, SoftRenderer

kaolin/graphics/softras/cuda/create_texture_image_cuda.cpp

@@ -1,18 +1,3 @@
-// Copyright (c) 2019, NVIDIA CORPORATION. All rights reserved.
-// #
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-// #
-//     http://www.apache.org/licenses/LICENSE-2.0
-// #
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-
 // Soft Rasterizer (SoftRas)
 
 // Copyright (c) 2017 Hiroharu Kato

kaolin/graphics/softras/functional/__init__.py

@@ -1,3 +1,27 @@
+# Soft Rasterizer (SoftRas)
+# 
+# Copyright (c) 2017 Hiroharu Kato
+# Copyright (c) 2018 Nikos Kolotouros
+# Copyright (c) 2019 Shichen Liu
+# 
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+
 from .get_points_from_angles import get_points_from_angles
 from .ambient_lighting import ambient_lighting
 from .directional_lighting import directional_lighting

kaolin/graphics/softras/functional/ambient_lighting.py

@@ -1,18 +1,3 @@
-# Copyright (c) 2019, NVIDIA CORPORATION. All rights reserved.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-# 
-# 
 # Soft Rasterizer (SoftRas)
 # 
 # Copyright (c) 2017 Hiroharu Kato

kaolin/graphics/softras/functional/directional_lighting.py

@@ -1,18 +1,3 @@
-# Copyright (c) 2019, NVIDIA CORPORATION. All rights reserved.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-# 
-# 
 # Soft Rasterizer (SoftRas)
 # 
 # Copyright (c) 2017 Hiroharu Kato

kaolin/graphics/softras/functional/face_vertices.py

@@ -1,18 +1,3 @@
-# Copyright (c) 2019, NVIDIA CORPORATION. All rights reserved.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-# 
-# 
 # Soft Rasterizer (SoftRas)
 # 
 # Copyright (c) 2017 Hiroharu Kato

kaolin/graphics/softras/functional/get_points_from_angles.py

@@ -1,18 +1,3 @@
-# Copyright (c) 2019, NVIDIA CORPORATION. All rights reserved.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-# 
-# 
 # Soft Rasterizer (SoftRas)
 # 
 # Copyright (c) 2017 Hiroharu Kato