Skip to content

Commit

Permalink
Fix some documentation issues of camera API (#601)
Browse files Browse the repository at this point in the history
* separate classes in dedicated web pages

Signed-off-by: Clement Fuji Tsang <cfujitsang@nvidia.com>

* fine-tune camera doc

Signed-off-by: Clement Fuji Tsang <cfujitsang@nvidia.com>
  • Loading branch information
Caenorst committed Jul 31, 2022
1 parent decc467 commit 1c48e9f
Show file tree
Hide file tree
Showing 12 changed files with 247 additions and 121 deletions.
15 changes: 15 additions & 0 deletions docs/modules/kaolin.render.camera.camera.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
:orphan:

.. _kaolin.render.camera.Camera:

kaolin.render.camera.Camera
===========================

API
---

.. autoclass:: kaolin.render.camera.Camera
:members:
:undoc-members:
:show-inheritance:

14 changes: 14 additions & 0 deletions docs/modules/kaolin.render.camera.camera_extrinsics.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
:orphan:

.. _kaolin.render.camera.CameraExtrinsics:

kaolin.render.camera.CameraExtrinsics
=====================================

API
---

.. autoclass:: kaolin.render.camera.CameraExtrinsics
:members:
:undoc-members:
:show-inheritance:
14 changes: 14 additions & 0 deletions docs/modules/kaolin.render.camera.camera_intrinsics.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
:orphan:

.. _kaolin.render.camera.CameraIntrinsics:

kaolin.render.camera.CameraIntrinsics
=====================================

API
---

.. autoclass:: kaolin.render.camera.CameraIntrinsics
:members:
:undoc-members:
:show-inheritance:
14 changes: 14 additions & 0 deletions docs/modules/kaolin.render.camera.extrinsics_rep.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
:orphan:

.. _kaolin.render.camera.ExtrinsicsRep:

kaolin.render.camera.ExtrinsicsRep
==================================

API
---

.. autoclass:: kaolin.render.camera.ExtrinsicsRep
:members:
:undoc-members:
:show-inheritance:
14 changes: 14 additions & 0 deletions docs/modules/kaolin.render.camera.orthographic_intrinsics.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
:orphan:

.. _kaolin.render.camera.OrthographicIntrinsics:

kaolin.render.camera.OrthographicIntrinsics
===========================================

API
---

.. autoclass:: kaolin.render.camera.OrthographicIntrinsics
:members:
:undoc-members:
:show-inheritance:
14 changes: 14 additions & 0 deletions docs/modules/kaolin.render.camera.pinhole_intrinsics.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
:orphan:

.. _kaolin.render.camera.PinholeIntrinsics:

kaolin.render.camera.PinholeIntrinsics
======================================

API
---

.. autoclass:: kaolin.render.camera.PinholeIntrinsics
:members:
:undoc-members:
:show-inheritance:
34 changes: 34 additions & 0 deletions docs/modules/kaolin.render.camera.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
.. _kaolin.render.camera:

kaolin.render.camera
====================

API
---

Classes
^^^^^^^

* :ref:`Camera <kaolin.render.camera.Camera>`
* :ref:`CameraExtrinsics <kaolin.render.camera.CameraExtrinsics>`
* :ref:`CameraIntrinsics <kaolin.render.camera.CameraIntrinsics>`
* :ref:`PinholeIntrinsics <kaolin.render.camera.PinholeIntrinsics>`
* :ref:`OrthographicIntrinsics <kaolin.render.camera.OrthographicIntrinsics>`
* :ref:`ExtrinsicsRep <kaolin.render.camera.ExtrinsicsRep>`

Functions
^^^^^^^^^

.. automodule:: kaolin.render.camera
:members:
:exclude-members:
Camera,
CameraExtrinsics,
CameraIntrinsics,
PinholeIntrinsics,
OrthographicIntrinsics,
ExtrinsicsRep
:undoc-members:
:show-inheritance:


79 changes: 46 additions & 33 deletions kaolin/render/camera/camera.py
Original file line number Diff line number Diff line change
Expand Up @@ -106,9 +106,9 @@ def _ctor_funcs_to_args(func):


class Camera:
r"""kaolin Cameras are a one-stop class for all camera related differentiable / non-differentiable transformations.
r"""Camera is a one-stop class for all camera related differentiable / non-differentiable transformations.
Cameras are represented by *batched* instances of 2 submodules:
Camera objects are represented by *batched* instances of 2 submodules:
- :class:`CameraExtrinsics`: The extrinsics properties of the camera (position, orientation).
These are usually embedded in the view matrix, used to transform vertices from world space to camera space.
Expand Down Expand Up @@ -145,39 +145,50 @@ class Camera:
How to apply transformations with kaolin's Camera:
1. Linear camera types, such as the commonly used pinhole camera,
support the view_projection_matrix() method.
support the :func:`view_projection_matrix()` method.
The returned matrix can be used to transform vertices through pytorch's matrix multiplication, or even be
passed to shaders as a uniform.
2. All Cameras are guaranteed to support a general transform() function
2. All Cameras are guaranteed to support a general :func:`transform()` function
which maps coordinates from world space to Normalized Device Coordinates space.
For some lens types which perform non linear transformations,
the view_projection_matrix() is non-defined. Therefore the camera transformation must be applied through
a dedicated function. For linear cameras, transform() may use matrices under the hood.
3. Camera parameters may also be queried directly. This is useful when implementing camera params aware code
such as ray tracers.
the :func:`view_projection_matrix()` is non-defined.
Therefore the camera transformation must be applied through
a dedicated function. For linear cameras,
:func:`transform()` may use matrices under the hood.
3. Camera parameters may also be queried directly.
This is useful when implementing camera params aware code such as ray tracers.
How to control kaolin's Camera:
- :class:`CameraExtrinsics`: is packed with useful methods for controlling the camera position and orientation:
:func:`translate()`, :func:`rotate()`, :func:`move_forward()`, :func:`move_up()`, :func:`move_right()`,
:func:`cam_pos()`, :func:`cam_up()`, :func:`cam_forward()`, :func:`cam_up()`.
- :class:`CameraIntrinsics`: exposes a lens :func:`zoom()` operation.
The exact functionality depends on the camera type.
:func:`translate() <CameraExtrinsics.translate()>`,
:func:`rotate() <CameraExtrinsics.rotate()>`,
:func:`move_forward() <CameraExtrinsics.move_forward()>`,
:func:`move_up() <CameraExtrinsics.move_up()>`,
:func:`move_right() <CameraExtrinsics.move_right()>`,
:func:`cam_pos() <CameraExtrinsics.cam_pos()>`,
:func:`cam_up() <CameraExtrinsics.cam_up()>`,
:func:`cam_forward() <CameraExtrinsics.cam_forward()>`,
:func:`cam_up() <CameraExtrinsics.cam_up()>`.
- :class:`CameraIntrinsics`: exposes a lens :func:`zoom() <CameraIntrinsics.zoom()>`
operation. The exact functionality depends on the camera type.
How to optimize the Camera parameters:
- Both :class:`CameraExtrinsics`: and :class:`CameraIntrinsics` maintain ``torch.Tensor`` buffers of parameters
which support pytorch differentiable operations.
- Both :class:`CameraExtrinsics`: and :class:`CameraIntrinsics` maintain
:class:`torch.Tensor` buffers of parameters which support pytorch differentiable operations.
- Setting ``camera.requires_grad_(True)`` will turn on the optimization mode.
- The ``gradient_mask`` function can be used to mask out gradients of specific Camera parameters.
- The :func:`gradient_mask` function can be used to mask out gradients of specific Camera parameters.
.. note::
:class:`CameraExtrinsics`: supports multiple representions of camera parameters
(see: :func:`switch_backend`).
Specific representations are better fit for optimization (e.g.: they maintain an orthogonal view matrix).
(see: :func:`switch_backend <CameraExtrinsics.switch_backend()>`).
Specific representations are better fit for optimization
(e.g.: they maintain an orthogonal view matrix).
Kaolin will automatically switch to using those representations when gradient flow is enabled
For non-differentiable uses, the default representation may provide better speed and numerical accuracy.
For non-differentiable uses, the default representation may provide better
speed and numerical accuracy.
Other useful camera properties:
Expand All @@ -187,14 +198,13 @@ class Camera:
- :class:`CameraExtrinsics`: and :class:`CameraIntrinsics`: individually support the :func:`requires_grad`
property.
- Cameras implement :func:`torch.allclose` for comparing camera parameters under controlled numerical accuracy.
The :func:`__eq__()`` function is reserved for comparison by ref.
The operator ``==`` is reserved for comparison by ref.
- Cameras support batching, either through construction, or through the :func:`cat()` method.
.. note::
Since kaolin's cameras are batched, the view/projection matrices are of shape ``(C, 4, 4)``,
and some operations, such as `:func:transform()` may return values as shapes of ``(C, B, 3)``.
Here ``C`` is the number of cameras and ``B`` is the number of transformed vertices.
Since kaolin's cameras are batched, the view/projection matrices are of shapes :math:`(\text{num_cameras}, 4, 4)`,
and some operations, such as :func:`transform()` may return values as shapes of :math:`(\text{num_cameras}, \text{num_vectors}, 3)`.
Concluding remarks on coordinate systems and other confusing conventions:
Expand Down Expand Up @@ -223,7 +233,8 @@ class Camera:
- kaolin's cameras do not assume any specific coordinate system for the camera axes. By default, the
right handed cartesian coordinate system is used. Other coordinate systems are supported through
:func:`change_coordinate_system()` and the ``coordinates.py`` module::
:func:`change_coordinate_system() <CameraExtrinsics.change_coordinate_system()>`
and the ``coordinates.py`` module::
Y
^
Expand All @@ -232,8 +243,8 @@ class Camera:
/
Z
- kaolin's NDC space is assumed to be left handed (depth goes inwards to the screen). The default range
of values is [-1, 1].
- kaolin's NDC space is assumed to be left handed (depth goes inwards to the screen).
The default range of values is [-1, 1].
"""

_extrinsics_constructors = _gather_constructors(*_EXTRINSICS_MODULES)
Expand All @@ -247,7 +258,7 @@ class Camera:
"""

def __init__(self, extrinsics: CameraExtrinsics, intrinsics: CameraIntrinsics):
r""" Constructs a new camera module from the pre-constructed extrinsics and intrinsics components.
r"""Constructs a new camera module from the pre-constructed extrinsics and intrinsics components.
.. seealso::
Expand Down Expand Up @@ -389,17 +400,18 @@ def gradient_mask(self, *args: Union[str, ExtrinsicsParamsDefEnum, IntrinsicsPar
"""Creates a gradient mask, which allows to backpropagate only through
params designated as trainable.
This function does not consider the requires_grad field when creating this mask.
This function does not consider the :attr:`requires_grad` field when creating this mask.
.. note::
The 3 extrinsics camera axes are always masked as trainable together.
This design choice ensures that these axes, as well as the view matrix, remain orthogonal.
Args:
*args : A vararg list of the extrinsic and intrinsic params that should allow gradient flow.
*args :
A vararg list of the extrinsic and intrinsic params that should allow gradient flow.
This function also supports conversion of params from their string names.
(i.e: 't' will convert to PinholeParamsDefEnum.t)
(i.e: 't' will convert to ``PinholeParamsDefEnum.t``).
Returns:
(torch.BoolTensor, torch.BoolTensor):
Expand All @@ -426,18 +438,19 @@ def gradient_mask(self, *args: Union[str, ExtrinsicsParamsDefEnum, IntrinsicsPar
return self.extrinsics.gradient_mask(*extrinsic_args), self.intrinsics.gradient_mask(*intrinsic_args)

@property
def width(self):
def width(self) -> int:
"""Camera image plane width (pixel resolution)"""
return self.intrinsics.width

@property
def height(self):
def height(self) -> int:
"""Camera image plane height (pixel resolution)"""
return self.intrinsics.height

@property
def lens_type(self) -> str:
""" A textual description of the camera lens type (i.e: 'pinhole', 'ortho') """
r"""A textual description of the camera lens type. (i.e 'pinhole', 'ortho')
"""
return self.intrinsics.lens_type

@property
Expand Down Expand Up @@ -514,7 +527,7 @@ def view_projection_matrix(self):
Works only for cameras with linear projection transformations.
Returns:
(torch.Tensor): The view projection matrix, of shape ``(C, 4, 4)`` where C is the number of cameras.
(torch.Tensor): The view projection matrix, of shape :math:`(\text{num_cameras}, 4, 4)`
"""
view = self.extrinsics.view_matrix()
projection = self.intrinsics.projection_matrix()
Expand Down

0 comments on commit 1c48e9f

Please sign in to comment.