Small improvements to documentation; improved access to tutorials. (#534

) Streamlined installation instructions; other small fixes. Small fixes Signed-off-by: Maria Masha Shugrina <mshugrina@nvidia.com> Fixed doc build issue Signed-off-by: Maria Masha Shugrina <mshugrina@nvidia.com> Fixed PR comments; fixed rtd theme version; added link for DIB-R example to tutorial notebook. Signed-off-by: Maria Masha Shugrina <mshugrina@nvidia.com> Co-authored-by: Maria Masha Shugrina <mshugrina@nvidia.com>
NVIDIAGameWorks · Mar 24, 2022 · a6ea5bf · a6ea5bf
1 parent 26f6c93
commit a6ea5bf
Show file tree

Hide file tree

Showing 12 changed files with 140 additions and 67 deletions.
diff --git a/README.md b/README.md
@@ -7,7 +7,7 @@
 ## Overview
 NVIDIA Kaolin library provides a PyTorch API for working with a variety of 3D representations and includes a growing collection of GPU-optimized operations such as modular differentiable rendering, fast conversions between representations, data loading, 3D checkpoints and more. 
 
-Kaolin library is part of a larger suite of tools for 3D deep learning research. For example, the [Omniverse Kaolin App](https://docs.omniverse.nvidia.com/app_kaolin/app_kaolin/overview.html) will allow interactive visualization of 3D checkpoints. To find out more about the Kaolin ecosystem, visit the [NVIDIA Kaolin Dev Zone page](https://developer.nvidia.com/kaolin).
+Kaolin library is part of a larger suite of tools for 3D deep learning research. For example, the [Omniverse Kaolin App](https://docs.omniverse.nvidia.com/app_kaolin/app_kaolin/overview.html) allows interactive visualization of 3D checkpoints. To find out more about the Kaolin ecosystem, visit the [NVIDIA Kaolin Dev Zone page](https://developer.nvidia.com/kaolin).
 
 ## Installation and Getting Started
 
@@ -17,7 +17,7 @@ Visit the [Kaolin Library Documentation](https://kaolin.readthedocs.io/en/latest
 
 With the version 0.10.0 we are focusing on Volumetric rendering, adding new features for tetrahedral meshes, including DefTet volumetric renderer and losses, and Deep Marching Tetrahedrons, and adding new primitive operations for efficient volumetric rendering of Structured Point Clouds, we are also adding support for materials with USD importation.
 
-Finally we are adding two new tutorials to show how to use the latest features from Kaolin:
+Finally we are adding two new tutorials to show how to use the latest features from Kaolin. See [Tutorial Index](https://kaolin.readthedocs.io/en/latest/notes/tutorial_index.html) for more.
 * [How to use DMtet](examples/tutorial/dmtet_tutorial.ipynb) to rencontruct a mesh from point clouds generated by the Omniverse Kaolin App
 
 <p float="left">
@@ -55,21 +55,22 @@ Please review our [contribution guidelines](CONTRIBUTING.md).
 
 ## Citation
 
-If you found this codebase useful in your research, please cite
+If you are using Kaolin library for your research, please cite:
 
 ```
-@misc{Kaolin2019,
-author = {Clement Fuji Tsang and Masha Shugrina and Jean Francois Lafleche and Towaki Takikawa and Jiehan Wang and Charles Loop and Wenzheng Chen and Krishna Murthy Jatavallabhula and Edward Smith and Artem Rozantsev and Or Perel and Frank Shen and Jun Gao and Sanja Fidler and Gavriel State and Jason Gorski and Tommy Xiang and Jianing Li and Michael Li and Rev Lebaredian}
-title = {Kaolin}
-year = {2019}
+@misc{KaolinLibrary,
+      author = {Fuji Tsang, Clement and Shugrina, Maria and Lafleche, Jean Francois and Takikawa, Towaki and Wang, Jiehan and Loop, Charles and Chen, Wenzheng and Jatavallabhula, Krishna Murthy and Smith, Edward and Rozantsev, Artem and Perel, Or and Shen, Tianchang and Gao, Jun and Fidler, Sanja and State, Gavriel and Gorski, Jason and Xiang, Tommy and Li, Jianing and Li, Michael and Lebaredian, Rev},
+      title = {Kaolin: A Pytorch Library for Accelerating 3D Deep Learning Research},
+      year = {2022},
+      howpublished={\url{https://github.com/NVIDIAGameWorks/kaolin}}
 }
 ```
 ## Contributors
 
 Current Team:
 
 - Technical Lead: Clement Fuji Tsang
-- Manager: Masha Shugrina
+- Manager: Maria (Masha) Shugrina
 - Jean-Francois Lafleche
 - Charles Loop
 - Or Perel
@@ -87,7 +88,7 @@ Other Majors Contributors:
 - Michael Li
 - Krishna Murthy Jatavallabhula
 - Artem Rozantsev
-- Frank Shen
+- Tianchang (Frank) Shen
 - Edward Smith
 - Gavriel State
 - Tommy Xiang
diff --git a/docs/_templates/layout.html b/docs/_templates/layout.html
@@ -9,6 +9,7 @@
     a, a:visited, a:active {
       color: var(--nvidia-color);
     }
+
     /* Sidebar header (and topbar for mobile) */
     .wy-side-nav-search, .wy-nav-top {
       background-color: var(--nvidia-color);
@@ -25,7 +26,11 @@
     .rst-content .note .admonition-title, .rst-content .note .wy-alert-title, .rst-content .seealso .admonition-title, .rst-content .seealso .wy-alert-title, .rst-content .wy-alert-info.admonition-todo .admonition-title, .rst-content .wy-alert-info.admonition-todo .wy-alert-title, .rst-content .wy-alert-info.admonition .admonition-title, .rst-content .wy-alert-info.admonition .wy-alert-title, .rst-content .wy-alert-info.attention .admonition-title, .rst-content .wy-alert-info.attention .wy-alert-title, .rst-content .wy-alert-info.caution .admonition-title, .rst-content .wy-alert-info.caution .wy-alert-title, .rst-content .wy-alert-info.danger .admonition-title, .rst-content .wy-alert-info.danger .wy-alert-title, .rst-content .wy-alert-info.error .admonition-title, .rst-content .wy-alert-info.error .wy-alert-title, .rst-content .wy-alert-info.hint .admonition-title, .rst-content .wy-alert-info.hint .wy-alert-title, .rst-content .wy-alert-info.important .admonition-title, .rst-content .wy-alert-info.important .wy-alert-title, .rst-content .wy-alert-info.tip .admonition-title, .rst-content .wy-alert-info.tip .wy-alert-title, .rst-content .wy-alert-info.warning .admonition-title, .rst-content .wy-alert-info.warning .wy-alert-title, .rst-content .wy-alert.wy-alert-info .admonition-title, .wy-alert.wy-alert-info .rst-content .admonition-title, .wy-alert.wy-alert-info .wy-alert-title {
         background: #b8d27c;
     }
-}
+
+    .icon, .version, a.icon.icon-home {
+        color: white;
+    }
+
   </style>
 {% endblock %}
 
@@ -35,8 +40,9 @@
 </p>
 
 <ul>
+  <li class="toctree-l1"><a href="{{ pathto('index') }}">Welcome</a></li>
   <li class="toctree-l1"><a href="{{ pathto('notes/installation') }}">Installation</a></li>
-  <li class="toctree-l1"><a href="{{ pathto('notes/overview') }}">Overview</a></li>
+  <li class="toctree-l1"><a href="{{ pathto('notes/overview') }}">API Overview</a></li>
 </ul>
 
 {{super()}}

diff --git a/docs/index.rst b/docs/index.rst
@@ -1,20 +1,23 @@
-Welcome to Kaolin's documentation!
-==================================
+Welcome to Kaolin Library Documentation
+=======================================
 
 .. image:: ../assets/kaolin.png
 
 NVIDIA Kaolin library provides a PyTorch API for working with a variety of 3D representations and includes a growing collection of GPU-optimized operations such as modular differentiable rendering, fast conversions between representations, data loading, 3D checkpoints and more.
+See :ref:`Installation <installation>`, :ref:`API Overview <overview>` and :ref:`Tutorials <tutorial_index>` to get started!
 
-Kaolin library is part of a larger suite of tools for 3D deep learning research. For example, `Omniverse Kaolin app <https://docs.omniverse.nvidia.com/app_kaolin/app_kaolin/overview.html>`_ will allow interactive visualization of 3D checkpoints. To find out more about the Kaolin ecosystem, visit the `NVIDIA Kaolin Dev Zone page <https://developer.nvidia.com/kaolin>`_.
+Kaolin library is part of a larger suite of tools for 3D deep learning research. For example, `Omniverse Kaolin app <https://docs.omniverse.nvidia.com/app_kaolin/app_kaolin/overview.html>`_ allows interactive visualization of 3D checkpoints. To find out more about the Kaolin ecosystem, visit the `NVIDIA Kaolin Dev Zone page <https://developer.nvidia.com/kaolin>`_.
 
 
 .. toctree::
    :titlesonly:
    :maxdepth: 1
    :caption: Tutorials:
 
-   notes/diff_render
+   notes/tutorial_index
    notes/checkpoints
+   notes/diff_render
+   notes/spc_summary
 
 .. toctree::
    :titlesonly:

diff --git a/docs/notes/checkpoints.rst b/docs/notes/checkpoints.rst
@@ -1,5 +1,7 @@
-3D Checkpoints
-==============
+.. _3d_viz:
+
+3D Checkpoint Visualization
+===========================
 
 .. image:: ../img/koala.jpg
 

diff --git a/docs/notes/installation.rst b/docs/notes/installation.rst
@@ -5,78 +5,75 @@
 Installation
 ============
 
-Kaolin is written with Pytorch and C++ / CUDA for efficient custom ops.
+Kaolin is written with PyTorch and uses C++ / CUDA for efficient custom ops.
 
 Requirements
 ------------
 
-* Linux, macOS or Windows
+* Linux, macOS (CPU-only) or Windows
 * Python >= 3.6 (3.6 and 3.7 recommended for Windows)
 * CUDA >= 10.0 (with 'nvcc' installed)
 
 Dependencies
 ------------
 
 * torch >= 1.5, <= 1.10.2
-* cython == 0.29.20
-* scipy >= 1.2.0
-* Pillow >= 8.0.0
-* usd-core >= 20.11 (optional, required for USD related features such as visualization and importer / exporter)
+* cython == 0.29.20 (auto-installed)
+* scipy >= 1.2.0 (auto-installed)
+* Pillow >= 8.0.0 (auto-installed)
+* usd-core >= 20.11 (auto-installed; required for USD I/O and 3D checkpoints with :class:`~kaolin.visualize.Timelapse`)
 
 Installation from source
 ------------------------
 
 .. Note::
-    If you just want to try Kaolin, we recommend using a virtual environment, for instance with `Anaconda <https://www.anaconda.com/>`_:
+    We recommend installing Kaolin into a virtual environment, for instance with `Anaconda <https://www.anaconda.com/>`_:
 
     .. code-block:: bash
     
         $ conda create --name kaolin python=3.7
         $ conda activate kaolin
 
-We recommend following instructions from `https://pytorch.org <https://pytorch.org>`_ for installing PyTorch, and `https://cython.readthedocs.io <https://cython.readthedocs.io/en/latest/src/quickstart/install.html>`_ for installing cython, however Kaolin installation will attempt to automatically install the latest compatible version if none is installed (may fail on some systems).
-Kaolin may also function with some incompatible PyTorch versions; to override the PyTorch version check, set environment variable ``export IGNORE_TORCH_VER=1`` before installing.
+1. Clone Repository
+^^^^^^^^^^^^^^^^^^^
 
-To install the library. You must first clone the repository:
+Clone and optionally check out an `official release <https://github.com/NVIDIAGameWorks/kaolin/tags>`_:
 
 .. code-block:: bash
 
     $ git clone --recursive https://github.com/NVIDIAGameWorks/kaolin
     $ cd kaolin
+    $ git checkout v0.10.0
 
-If instead of the latest version you want a specific release like 0.10.0, 0.9.0, 0.9.1 or 0.1 you can then select the tag, example:
+2. Install Pytorch
+^^^^^^^^^^^^^^^^^^
+Follow `official instructions <https://pytorch.org>`_ to install PyTorch of a supported version.
+Kaolin may be able to work with other PyTorch versions. See below for overriding PyTorch version check during install.
 
-.. code-block:: bash
 
-    $ git checkout v0.10.0
+3. Optional Environment Variables
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-To enable installation of experimental features, set
-environment variable
-``export KAOLIN_INSTALL_EXPERIMENTAL=1``. To install, run:
+* If trying Kaolin with an unsupported PyTorch version, set: ``export IGNORE_TORCH_VER=1``
+* To install experimental features (like :ref:`kaolin-dash3d <dash 3d>`), set: ``export KAOLIN_INSTALL_EXPERIMENTAL=1``
+* If using heterogeneous GPU setup, set the architectures for which to compile the CUDA code, e.g.: ``export TORCH_CUDA_ARCH_LIST="7.0 7.5"``
+* In some setups, there may be a conflict between cub available with cuda install > 11 and ``third_party/cub`` that kaolin includes as a submodule. If conflict occurs or cub is not found, set ``CUB_HOME`` to the cuda one, e.g. typically on Linux: ``export CUB_HOME=/usr/local/cuda-*/include/``
 
-.. Note::
-    On CUDA >= 11.0, CUB is already available and ``CUB_HOME`` should be specified to avoid conflict with the submodule ``third_party/cub`` (typically on linux ``export CUB_HOME=/usr/local/cuda-*/include/``).
+
+4. Install Kaolin
+^^^^^^^^^^^^^^^^^
 
 .. code-block:: bash
 
     $ python setup.py develop
 
 .. Note::
-    If you are using heterogeneous GPUs setup set the architectures for which you want to compile the cuda code using the ``TORCH_CUDA_ARCH_LIST`` environment variable.
-
-    Example:
-
-    .. code-block:: bash
-    
-        $ export TORCH_CUDA_ARCH_LIST="7.0 7.5"
-
-.. Note::
-    Kaolin can be installed without GPU, however, CPU support is limited to some ops.
+    Kaolin can be installed without GPU, however, CPU support is limited and many CUDA-only functions will be missing.
 
 Testing your installation
 -------------------------
 
-A quick test is to see if you can properly import kaolin and print the current version by running the following:
+Run a quick test of your installation and version:
 
 .. code-block:: bash
 
@@ -85,19 +82,12 @@ A quick test is to see if you can properly import kaolin and print the current v
 Running tests
 ^^^^^^^^^^^^^
 
-A more exhaustive test is to execute all the official tests.
-
-First, pytest dependencies are necessary to run those tests, to install those run:
+For an exhaustive check, install testing dependencies and run tests as follows:
 
 .. code-block:: bash
 
     $ pip install -r tools/ci_requirements.txt
- 
-Then run the tests as following:
-
-.. code-block:: bash
-
     $ pytest tests/python/
 
 .. Note::
-    These tests rely on cuda operations and will fail if you installed on CPU only, where not all functionality is available.
+    These tests rely on CUDA operations and will fail if you installed on CPU only, where not all functionality is available.
diff --git a/docs/notes/overview.rst b/docs/notes/overview.rst
@@ -2,15 +2,11 @@
 
 .. _overview:
 
-API Overview and Reference
-==========================
+API Overview
+============
 
-Kaolin library
---------------
-
-NVIDIA Kaolin library provides a PyTorch API for working with a variety of 3D representations and includes a growing collection of GPU-optimized operations such as modular differentiable rendering, fast conversions between representations, data loading, 3D checkpoints and more.
-
-It is composed of multiple modules:
+Below is a summary of Kaolin functionality. Refer to :ref:`tutorial_index` for specific use cases, examples
+and recipes that use these building blocks.
 
 Operators for 3D Data:
 ^^^^^^^^^^^^^^^^^^^^^^

diff --git a/docs/notes/spc_summary.rst b/docs/notes/spc_summary.rst
@@ -0,0 +1,23 @@
+Structured Point Clouds (SPCs)
+==============================
+
+Structured Point Cloud is a versatile octree data structure useful for a wide range of tasks.
+
+.. image:: ../img/mesh_to_spc.png
+
+Understanding SPCs Tutorial:
+----------------------------
+
+See our Jupyter notebook for an walk-through of SPC features:
+
+`examples/tutorial/understanding_spcs_tutorial.ipynb <https://github.com/NVIDIAGameWorks/kaolin/blob/master/examples/tutorial/understanding_spcs_tutorial.ipynb>`_
+
+
+SPC Documentation:
+------------------
+
+Functions useful for working with SPCs are available in the following modules:
+
+* :ref:`kaolin.ops.spc<kaolin.ops.spc>` - general explanation and operations
+* :ref:`kaolin.render.spc<kaolin.render.spc>` - rendering utilities
+* :class:`kaolin.rep.Spc` - high-level wrapper
diff --git a/docs/notes/tutorial_index.rst b/docs/notes/tutorial_index.rst
@@ -0,0 +1,52 @@
+.. _tutorial_index:
+
+Tutorial Index
+==============
+
+Kaolin provides tutorials as ipython notebooks, docs pages and simple scripts. Note that the links
+point to master.
+
+
+Detailed Tutorials
+------------------
+
+* `Deep Marching Tetrahedra <https://github.com/NVIDIAGameWorks/kaolin/blob/master/examples/tutorial/dmtet_tutorial.ipynb>`_: reconstructs a tetrahedral mesh from point clouds with `DMTet <https://nv-tlabs.github.io/DMTet/>`_, covering:
+    * generating data with Omniverse Kaolin App
+    * loading point clouds from a ``.usd`` file
+    * chamfer distance as a loss function
+    * differentiable marching tetrahedra
+    * using Timelapse API for 3D checkpoints
+    * visualizing 3D results of training
+* `Understanding Structured Point Clouds (SPCs) <https://github.com/NVIDIAGameWorks/kaolin/blob/master/examples/tutorial/understanding_spcs_tutorial.ipynb>`_: walks through SPC features, covering:
+    * under-the-hood explanation of SPC, why it's useful and key ops
+    * loading a mesh
+    * sampling a point cloud
+    * converting a point cloud to SPC
+    * setting up camera
+    * rendering SPC with ray tracing
+    * storing features in an SPC
+* `Differentiable Rendering <https://github.com/NVIDIAGameWorks/kaolin/blob/master/examples/tutorial/dibr_tutorial.ipynb>`_: optimizes a triangular mesh from images using `DIB-R <https://github.com/nv-tlabs/DIB-R-Single-Image-3D-Reconstruction>`_ renderer, covering:
+    * generating data with Omniverse Kaolin App, and loading this synthetic data
+    * loading a mesh
+    * computing mesh laplacian
+    * DIB-R rasterization
+    * differentiable texture mapping
+    * computing mask intersection-over-union loss (IOU)
+    * using Timelapse API for 3D checkpoints
+    * visualizing 3D results of training
+* :ref:`3d_viz`: explains saving 3D checkpoints and visualizing them, covering:
+    * using Timelapse API for writing 3D checkpoints
+    * understanding output file format
+    * visualizing 3D checkpoints using Omniverse Kaolin App
+    * visualizing 3D checkpoints using bundled ``kaolin-dash3d`` commandline utility
+
+
+Simple Recipes
+--------------
+
+* I/O and Data Processing:
+    * `usd_kitchenset.py <https://github.com/NVIDIAGameWorks/kaolin/blob/master/examples/tutorial/usd_kitchenset.py>`_: loading multiple meshes from a ``.usd`` file and saving
+* Visualization:
+    * `visualize_main.py <https://github.com/NVIDIAGameWorks/kaolin/blob/master/examples/tutorial/visualize_main.py>`_: using Timelapse API to write mock 3D checkpoints
+
+
diff --git a/examples/tutorial/dibr_tutorial.ipynb b/examples/tutorial/dibr_tutorial.ipynb
@@ -7,7 +7,7 @@
    "source": [
     "# Optimizing a mesh using a Differentiable Renderer\n",
     "\n",
-    "Differentiable rendering can be used to optimize the underlying 3D properties, like geometry and lighting, by backpropagating gradients from the loss in the image space. In this tutorial, we optimize geometry and texture of a single object based on a dataset of rendered ground truth views. This tutorial demonstrates functionality in `kaolin.render.mesh`, including the key `dibr_rasterization` function. See detailed [API documentation](https://kaolin.readthedocs.io/en/latest/modules/kaolin.render.mesh.html).\n",
+    "Differentiable rendering can be used to optimize the underlying 3D properties, like geometry and lighting, by backpropagating gradients from the loss in the image space. In this tutorial, we optimize geometry and texture of a single object based on a dataset of rendered ground truth views. This tutorial demonstrates functionality in `kaolin.render.mesh`, including the key `dibr_rasterization` function.  See detailed [API documentation](https://kaolin.readthedocs.io/en/latest/modules/kaolin.render.mesh.html). Note that this script is didactic and is not meant as a production end-to-end example; for more examples using DIB-R differentiable renderer, see [this repository](https://github.com/nv-tlabs/DIB-R-Single-Image-3D-Reconstruction).\n",
     "\n",
     "In addition, we demonstrate the use of [Kaolin's 3D checkpoints and training visualization](https://kaolin.readthedocs.io/en/latest/modules/kaolin.visualize.html) with the [Omniverse Kaolin App](https://docs.omniverse.nvidia.com/app_kaolin/app_kaolin/user_manual.html).\n",
     "\n",
@@ -496,7 +496,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.7.10"
+   "version": "3.7.11"
   }
  },
  "nbformat": 4,

diff --git a/examples/tutorial/network.py → examples/tutorial/dmtet_network.py b/examples/tutorial/network.py → examples/tutorial/dmtet_network.py
diff --git a/examples/tutorial/dmtet_tutorial.ipynb b/examples/tutorial/dmtet_tutorial.ipynb
@@ -24,7 +24,7 @@
     "import torch\n",
     "import kaolin\n",
     "import numpy as np\n",
-    "from network import Decoder\n",
+    "from dmtet_network import Decoder\n",
     "\n",
     "# path to the point cloud to be reconstructed\n",
     "pcd_path = \"../samples/bear_pointcloud.usd\"\n",

diff --git a/tools/doc_requirements.txt b/tools/doc_requirements.txt
@@ -1,3 +1,3 @@
 setuptools>=50.3
 sphinx>=3.5.4
-sphinx_rtd_theme
+sphinx_rtd_theme==1.0.0