Skip to content

Commit

Permalink
Update offscreen rendering guide
Browse files Browse the repository at this point in the history
  • Loading branch information
@mmatl
mmatl committed Feb 25, 2019
1 parent 03b4c51 commit 9c25308
Show file tree
Hide file tree
Showing 4 changed files with 106 additions and 212 deletions.
191 changes: 24 additions & 167 deletions README.md
View file
Expand Up @@ -5,17 +5,21 @@
[![Coverage Status](https://coveralls.io/repos/github/mmatl/pyrender/badge.svg?branch=master)](https://coveralls.io/github/mmatl/pyrender?branch=master)
[![PyPI version](https://badge.fury.io/py/pyrender.svg)](https://badge.fury.io/py/pyrender)

Pyrender is a pure Python (2.7, 3.4, 3.5, 3.6) library for visualizing and rendering
3D scenes using OpenGL and physically-based rendering (PBR).
It is designed around the [glTF 2.0 specification from
Khronos](https://www.khronos.org/gltf/) and is mostly compliant with that
specification.
Pyrender is designed to lightweight, easy to install, and simple to use so that
you can get started visualizing 3D data instantly.
Pyrender is a pure Python (2.7, 3.4, 3.5, 3.6) library for physically-based
rendering and visualization.
It is designed to meet the [glTF 2.0 specification from Khronos](https://www.khronos.org/gltf/).

Pyrender is lightweight, easy to install, and simple to use.
It comes packaged with both an intuitive scene viewer and a headache-free
offscreen renderer with support for GPU-accelerated rendering on headless
servers, which makes it perfect for machine learning applications.

Extensive documentation, including a quickstart guide, is provided [here](https://pyrender.readthedocs.io/en/latest/).

Check out a minimial working example of offscreen rendering in a Google CoLab Notebook [here](https://colab.research.google.com/drive/1Z71mHIc-Sqval92nK290vAsHZRUkCjUx#scrollTo=02aJu2LKtoE6).
For a minimal working example of offscreen rendering using OSMesa,
check out the [OSMesa Google CoLab Notebook](https://colab.research.google.com/drive/1Z71mHIc-Sqval92nK290vAsHZRUkCjUx).
For a minimal working example of GPU-accelerated offscreen rendering using EGL,
check out the [EGL Google CoLab Notebook](https://colab.research.google.com/drive/1rTLHk0qxh4dn8KNe-mCnN8HAWdd2_BEh).


<p align="center">
Expand All @@ -30,177 +34,30 @@ You can install pyrender directly from pip.
pip install pyrender
```

If you plan to use pyrender for long runs of generating data, I'd recommend
installing my fork of pyglet first, as the existing version has a memory leak:

```bash
git clone https://github.com/mmatl/pyglet.git
cd pyglet
python setup.py install
```

## Features

Despite being lightweight, pyrender has lots of features, including:

* Simple interoperation with the amazing [trimesh](https://github.com/mikedh/trimesh) project.
* Simple interoperation with the amazing [trimesh](https://github.com/mikedh/trimesh) project,
which enables out-of-the-box support for dozens of mesh types, including OBJ,
STL, DAE, OFF, PLY, and GLB.
* An easy-to-use scene viewer with support for animation, showing face and vertex
normals, toggling lighting conditions, and saving images and GIFs.
* An offscreen rendering module for generating images of 3D scenes.
* Support for several types of lighting, including point, spot, and directional
lights.
* Shadows (for directional and spot lights only, for now).
* An offscreen rendering module that supports OSMesa and EGL backends.
* Shadow mapping for directional and spot lights.
* Metallic-roughness materials for physically-based rendering, including several
types of texture and normal mapping.
* Transparency.
* Depth and color image generation.

## Sample Usage
Sample usage of the library is shown below:

```python
import os
import numpy as np
import trimesh

from pyrender import PerspectiveCamera,\
DirectionalLight, SpotLight, PointLight,\
MetallicRoughnessMaterial,\
Primitive, Mesh, Node, Scene,\
Viewer, OffscreenRenderer

#==============================================================================
# Mesh creation
#==============================================================================

#------------------------------------------------------------------------------
# Creating textured meshes from trimeshes
#------------------------------------------------------------------------------

# Fuze trimesh
fuze_trimesh = trimesh.load('./models/fuze.obj', process=False)
fuze_mesh = Mesh.from_trimesh(fuze_trimesh)

# Drill trimesh
drill_trimesh = trimesh.load('./models/drill.obj', process=False)
drill_mesh = Mesh.from_trimesh(drill_trimesh)
drill_pose = np.eye(4)
drill_pose[0,3] = 0.1
drill_pose[2,3] = -np.min(drill_trimesh.vertices[:,2])

# Wood trimesh
wood_trimesh = trimesh.load('./models/wood.obj', process=False)
wood_mesh = Mesh.from_trimesh(wood_trimesh)

# Water bottle trimesh
bottle_gltf = trimesh.load('./models/WaterBottle.glb', process=False)
bottle_trimesh = bottle_gltf.geometry[list(bottle_gltf.geometry.keys())[0]]
bottle_mesh = Mesh.from_trimesh(bottle_trimesh)
bottle_pose = np.array([
[1.0, 0.0, 0.0, 0.1],
[0.0, 0.0, -1.0, -0.16],
[0.0, 1.0, 0.0, 0.13],
[0.0, 0.0, 0.0, 1.0],
])

#------------------------------------------------------------------------------
# Creating meshes with per-vertex colors
#------------------------------------------------------------------------------
boxv_trimesh = trimesh.creation.box(extents=0.1*np.ones(3))
boxv_vertex_colors = np.random.uniform(size=(boxv_trimesh.vertices.shape))
boxv_trimesh.visual.vertex_colors = boxv_vertex_colors
boxv_mesh = Mesh.from_trimesh(boxv_trimesh, smooth=False)

#------------------------------------------------------------------------------
# Creating meshes with per-face colors
#------------------------------------------------------------------------------
boxf_trimesh = trimesh.creation.box(extents=0.1*np.ones(3))
boxf_face_colors = np.random.uniform(size=boxf_trimesh.faces.shape)
boxf_trimesh.visual.face_colors = boxf_face_colors
boxf_mesh = Mesh.from_trimesh(boxf_trimesh, smooth=False)

#------------------------------------------------------------------------------
# Creating meshes from point clouds
#------------------------------------------------------------------------------
points = trimesh.creation.icosphere(radius=0.05).vertices
point_colors = np.random.uniform(size=points.shape)
points_mesh = Mesh.from_points(points, colors=point_colors)

#==============================================================================
# Light creation
#==============================================================================

direc_l = DirectionalLight(color=np.ones(3), intensity=1.0)
spot_l = SpotLight(color=np.ones(3), intensity=10.0,
innerConeAngle=np.pi/16, outerConeAngle=np.pi/6)
point_l = PointLight(color=np.ones(3), intensity=10.0)

#==============================================================================
# Camera creation
#==============================================================================

cam = PerspectiveCamera(yfov=(np.pi / 3.0))
cam_pose = np.array([
[0.0, -np.sqrt(2)/2, np.sqrt(2)/2, 0.5],
[1.0, 0.0, 0.0, 0.0],
[0.0, np.sqrt(2)/2, np.sqrt(2)/2, 0.4],
[0.0, 0.0, 0.0, 1.0]
])

#==============================================================================
# Scene creation
#==============================================================================

scene = Scene(ambient_light=np.array([0.02, 0.02, 0.02]))

#==============================================================================
# Adding objects to the scene
#==============================================================================

#------------------------------------------------------------------------------
# By manually creating nodes
#------------------------------------------------------------------------------
fuze_node = Node(mesh=fuze_mesh, translation=np.array([0.1, 0.15, -np.min(fuze_trimesh.vertices[:,2])]))
scene.add_node(fuze_node)
boxv_node = Node(mesh=boxv_mesh, translation=np.array([-0.1, 0.10, 0.05]))
scene.add_node(boxv_node)
boxf_node = Node(mesh=boxf_mesh, translation=np.array([-0.1, -0.10, 0.05]))
scene.add_node(boxf_node)

#------------------------------------------------------------------------------
# By using the add() utility function
#------------------------------------------------------------------------------
drill_node = scene.add(drill_mesh, pose=drill_pose)
bottle_node = scene.add(bottle_mesh, pose=bottle_pose)
wood_node = scene.add(wood_mesh)
direc_l_node = scene.add(direc_l, pose=cam_pose)
spot_l_node = scene.add(spot_l, pose=cam_pose)

#==============================================================================
# Using the viewer with a default camera
#==============================================================================

v = Viewer(scene, shadows=True)

#==============================================================================
# Using the viewer with a pre-specified camera
#==============================================================================
cam_node = scene.add(cam, pose=cam_pose)
v = Viewer(scene, central_node=drill_node)

#==============================================================================
# Rendering offscreen from that camera
#==============================================================================

r = OffscreenRenderer(viewport_width=640*2, viewport_height=480*2)
color, depth = r.render(scene)
r.delete()

import matplotlib.pyplot as plt
plt.figure()
plt.imshow(color)
plt.show()
```

For sample usage, check out the [quickstart
guide](https://pyrender.readthedocs.io/en/latest/examples/index.html) or one of
the Google CoLab Notebooks:

* [OSMesa Google CoLab Notebook](https://colab.research.google.com/drive/1Z71mHIc-Sqval92nK290vAsHZRUkCjUx)
* [EGL Google CoLab Notebook](https://colab.research.google.com/drive/1rTLHk0qxh4dn8KNe-mCnN8HAWdd2_BEh)

## Viewer Keyboard and Mouse Controls

Expand Down
59 changes: 53 additions & 6 deletions docs/source/examples/offscreen.rst
View file
Expand Up @@ -4,14 +4,48 @@ Offscreen Rendering
===================

.. note::
If you're using a headless server, make sure that you followed the guide
for installing OSMesa. See :ref:`osmesa`.
If you're using a headless server, you'll need to use either EGL (for
GPU-accelerated rendering) or OSMesa (for CPU-only software rendering).
If you're using OSMesa, be sure that you've installed it properly. See
:ref:`osmesa` for details.

Choosing a Backend
------------------

Once you have a scene set up with its geometry, cameras, and lights,
you can render it using the :class:`.OffscreenRenderer`.
you can render it using the :class:`.OffscreenRenderer`. Pyrender supports
three backends for offscreen rendering:

- Pyglet, the same engine that runs the viewer. This requires an active
display manager, so you can't run it on a headless server. This is the
default option.
- OSMesa, a software renderer.
- EGL, which allows for GPU-accelerated rendering without a display manager.

If you want to use OSMesa or EGL, you need to set the ``PYOPENGL_PLATFORM``
environment variable before importing pyrender or any other OpenGL library.
You can do this at the command line:

.. code-block:: bash
PYOPENGL_PLATFORM=osmesa python render.py
or at the top of your Python script:

.. code-block:: bash
# Top of main python script
import os
os.environ['PYOPENGL_PLATFORM'] = 'egl'
Configure the renderer with a window width, a window height, and a size for
point-cloud points:
The handle for EGL is ``egl``, and the handle for OSMesa is ``osmesa``.

Running the Renderer
--------------------

Once you've set your environment variable appropriately, create your scene and
then configure the :class:`.OffscreenRenderer` object with a window width,
a window height, and a size for point-cloud points:

>>> r = pyrender.OffscreenRenderer(viewport_width=640,
... viewport_height=480,
Expand All @@ -35,6 +69,19 @@ and enables shadow mapping for all directional lights:
>>> color, depth = r.render(scene, flags=flags)

Once you're done with the offscreen renderer, you need to close it before you
can run a different renderer or open the viewer:
can run a different renderer or open the viewer for the same scene:

>>> r.delete()

Google CoLab Examples
---------------------

For a minimal working example of offscreen rendering using OSMesa,
see the `OSMesa Google CoLab notebook`_.

.. _OSMesa Google CoLab notebook: https://colab.research.google.com/drive/1Z71mHIc-Sqval92nK290vAsHZRUkCjUx

For a minimal working example of offscreen rendering using EGL,
see the `EGL Google CoLab notebook`_.

.. _EGL Google CoLab notebook: https://colab.research.google.com/drive/1rTLHk0qxh4dn8KNe-mCnN8HAWdd2_BEh
22 changes: 14 additions & 8 deletions docs/source/index.rst
View file
Expand Up @@ -5,16 +5,22 @@
Pyrender Documentation
========================
Pyrender is a Python 2/3 implementation of Physically-Based Rendering (PBR).
If is mostly compliant with the glTF 2.0 specification, and it makes it easy
to render 3D scenes in pure Python. Dependencies are light and all
pip-installable.
Pyrender is a pure Python (2.7, 3.4, 3.5, 3.6) library for physically-based
rendering and visualization.
It is designed to meet the glTF 2.0 specification_ from Khronos

.. image:: _static/rotation.gif
.. _specification: https://www.khronos.org/gltf/

Pyrender is lightweight, easy to install, and simple to use.
It comes packaged with both an intuitive scene viewer and a headache-free
offscreen renderer with support for GPU-accelerated rendering on headless
servers, which makes it perfect for machine learning applications.
Check out the :ref:`guide` for a full tutorial, or fork me on
Github_.

Pyrender supports rendering objects with metallic-roughness textures,
normal maps, ambient occlusion textures, emission textures, and shadows.
It also includes an easy-to-use viewer for visualizing and debugging 3D data.
.. _Github: https://github.com/mmatl/pyrender

.. image:: _static/rotation.gif

.. image:: _static/damaged_helmet.png

Expand Down
46 changes: 15 additions & 31 deletions docs/source/install/index.rst
View file
Expand Up @@ -26,27 +26,34 @@ Getting Pyrender Working with OSMesa
------------------------------------
If you want to render scenes offscreen but don't want to have to
install a display manager or deal with the pains of trying to get
OpenGL to work over SSH, you may consider using OSMesa,
OpenGL to work over SSH, you have two options.

The first (and preferred) option is using EGL, which enables you to perform
GPU-accelerated rendering on headless servers.
However, you'll need EGL 1.5 to get modern OpenGL contexts.
This comes packaged with NVIDIA's current drivers, but if you are having issues
getting EGL to work with your hardware, you can try using OSMesa,
a software-based offscreen renderer that is included with any Mesa
install.

If you want to do this, you'll need to complete three steps:
If you want to use OSMesa with pyrender, you'll have to perform two additional
installation steps:

- :ref:`installmesa`
- :ref:`installpyopengl`
- :ref:`configurescripts`

Pyrender supports using OSMesa for creating OpenGL contexts without
a screen, but you'll need to rebuild and re-install Mesa with support
for fast offscreen rendering and OpenGL 3+ contexts.
I'd recommend installing from source, but you can also try my ``.deb``
for Ubuntu 16.04 and up.
Then, read the offscreen rendering tutorial. See :ref:`offscreen_guide`.

.. _installmesa:

Installing OSMesa
*****************

As a first step, you'll need to rebuild and re-install Mesa with support
for fast offscreen rendering and OpenGL 3+ contexts.
I'd recommend installing from source, but you can also try my ``.deb``
for Ubuntu 16.04 and up.

Installing from a Debian Package
********************************

Expand Down Expand Up @@ -137,29 +144,6 @@ on PyPI.
git clone git@github.com:mmatl/pyopengl.git
pip install ./pyopengl
.. _configurescripts:

Configuring Offscreen Rendering Scripts
***************************************
Before running any script using the :class:`OffscreenRenderer` object,
make sure to set the ``PYOPENGL_PLATFORM`` environment variable to ``osmesa``.
For example:

.. code-block:: bash
PYOPENGL_PLATFORM=osmesa python run_rendering_script.py
Alternatively, you can just add ese two lines of code to the top of any script
that you want to do offscreen rendering in. Be sure to add them before
importing any other packages.

.. code-block:: python
import os
os.environ['PYOPENGL_PLATFORM'] = 'osmesa'
If you do this, you won't be able to use the :class:`Viewer`,
but you will be able do do offscreen rendering without a display.
Building Documentation
----------------------
Expand Down

0 comments on commit 9c25308

Please sign in to comment.