Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add sphinx gallery #384

Merged
merged 59 commits into from Dec 12, 2022
Merged

Add sphinx gallery #384

merged 59 commits into from Dec 12, 2022

Conversation

FirefoxMetzger
Copy link
Contributor

@FirefoxMetzger FirefoxMetzger commented Nov 12, 2022

Closes: #345 #145

This PR contributes a sphinx gallery that converts examples into a nice RST file including output renders. It also reorganizes the example folder into introductory, feature_demo, and validation examples to track them better and to display examples by category in the gallery.

Note that this is a large PR. Most of the changes are fairly light-weight, though, so a lot of the touching lines are things like adding a title in the docstring or renaming an example. The critical pieces are in conf.py (the sphinx config) and in utils/gallery_scraper.py.

@FirefoxMetzger
Copy link
Contributor Author

One current issue is that some examples (e.g. the PBR render of the helmet) use models from examples/models. Currently these are loaded by getting a path to the model via __file__; however, __file__ is not defined when the example is run by sphinx, so those examples break. I'm not sure how we want to resolve this though. My intuitive approach would have been to think of a gfx.sample_models.[helmet/teapot] approach, but this means that we have to include this data in the package, which I am not a fan of. Then again, it might be useful because users that copy down the example from the docs will face the same pathing issues on their machines since a path based on __file__ (pointing to the target of the copy/paste) will break too.

@FirefoxMetzger
Copy link
Contributor Author

Here is also a benchmark on how long the examples take to build on my machine. It doesn't look too bad:

computation time summary:
    - ..\examples\feature_demo\plot_collection_line.py:                    20.45 sec   0.0 MB
    - ..\examples\feature_demo\plot_pbr2.py:                               16.97 sec   0.0 MB
    - ..\examples\feature_demo\plot_multi_select.py:                        3.65 sec   0.0 MB
    - ..\examples\introductory\plot_hello_triangle.py:                      3.15 sec   0.0 MB
    - ..\examples\validation\plot_validate_color.py:                        2.03 sec   0.0 MB
    - ..\examples\validation\plot_validate_light_shadow.py:                 1.95 sec   0.0 MB
    - ..\examples\feature_demo\plot_scene_subplots_video.py:                1.92 sec   0.0 MB
    - ..\examples\validation\plot_validate_mesh_colormap.py:                1.87 sec   0.0 MB
    - ..\examples\feature_demo\plot_multi_slice2.py:                        1.83 sec   0.0 MB
    - ..\examples\feature_demo\plot_light_spotlight_shadows.py:             1.82 sec   0.0 MB
    - ..\examples\validation\plot_validate_volume.py:                       1.76 sec   0.0 MB
    - ..\examples\validation\plot_validate_image1.py:                       1.75 sec   0.0 MB
    - ..\examples\validation\plot_validate_depth_clipping.py:               1.72 sec   0.0 MB
    - ..\examples\feature_demo\plot_multi_slice1.py:                        1.70 sec   0.0 MB
    - ..\examples\validation\plot_validate_helpers2.py:                     1.70 sec   0.0 MB
    - ..\examples\feature_demo\plot_light_spotlights.py:                    1.65 sec   0.0 MB
    - ..\examples\validation\plot_validate_helpers1.py:                     1.63 sec   0.0 MB
    - ..\examples\validation\plot_validate_culling.py:                      1.63 sec   0.0 MB
    - ..\examples\feature_demo\plot_colormap_mesh.py:                       1.61 sec   0.0 MB
    - ..\examples\feature_demo\plot_wireframe_material.py:                  1.59 sec   0.0 MB
    - ..\examples\feature_demo\plot_mesh_depth_material.py:                 1.58 sec   0.0 MB
    - ..\examples\feature_demo\plot_cylinder.py:                            1.58 sec   0.0 MB
    - ..\examples\validation\plot_validate_image2.py:                       1.55 sec   0.0 MB
    - ..\examples\validation\plot_validate_box.py:                          1.53 sec   0.0 MB
    - ..\examples\validation\plot_validate_image_colormap.py:               1.52 sec   0.0 MB
    - ..\examples\feature_demo\plot_image_click_events.py:                  1.52 sec   0.0 MB
    - ..\examples\validation\plot_validate_ndc.py:                          1.51 sec   0.0 MB
    - ..\examples\feature_demo\plot_skybox.py:                              1.49 sec   0.0 MB
    - ..\examples\feature_demo\plot_light_shadow.py:                        1.48 sec   0.0 MB
    - ..\examples\feature_demo\plot_scene_in_a_scene.py:                    1.48 sec   0.0 MB
    - ..\examples\feature_demo\plot_scene_subplots2.py:                     1.47 sec   0.0 MB
    - ..\examples\feature_demo\plot_colormap_image.py:                      1.45 sec   0.0 MB
    - ..\examples\feature_demo\plot_custom_object3.py:                      1.44 sec   0.0 MB
    - ..\examples\feature_demo\plot_custom_object2.py:                      1.40 sec   0.0 MB
    - ..\examples\feature_demo\plot_colormap_channels.py:                   1.33 sec   0.0 MB
    - ..\examples\feature_demo\plot_picking_mesh.py:                        1.31 sec   0.0 MB
    - ..\examples\feature_demo\plot_custom_object1.py:                      1.21 sec   0.0 MB
    - ..\examples\feature_demo\plot_transparency2.py:                       1.19 sec   0.0 MB
    - ..\examples\feature_demo\plot_manual_matrix_update.py:                0.98 sec   0.0 MB
    - ..\examples\feature_demo\plot_panzoom_camera.py:                      0.97 sec   0.0 MB
    - ..\examples\feature_demo\plot_helpers_gizmo.py:                       0.96 sec   0.0 MB
    - ..\examples\feature_demo\plot_spheres.py:                             0.91 sec   0.0 MB
    - ..\examples\feature_demo\plot_light_directional_shadow.py:            0.88 sec   0.0 MB
    - ..\examples\introductory\plot_orbit_camera.py:                        0.85 sec   0.0 MB
    - ..\examples\feature_demo\plot_pbr.py:                                 0.82 sec   0.0 MB
    - ..\examples\introductory\plot_clipping_planes.py:                     0.82 sec   0.0 MB
    - ..\examples\feature_demo\plot_mesh_slice.py:                          0.81 sec   0.0 MB
    - ..\examples\feature_demo\plot_geometry_cubes.py:                      0.78 sec   0.0 MB
    - ..\examples\feature_demo\plot_scene_subplots1.py:                     0.73 sec   0.0 MB
    - ..\examples\introductory\plot_object_bounding_box.py:                 0.70 sec   0.0 MB
    - ..\examples\feature_demo\plot_volume_render1.py:                      0.67 sec   0.0 MB
    - ..\examples\feature_demo\plot_two_canvases.py:                        0.63 sec   0.0 MB
    - ..\examples\feature_demo\plot_instancing_mesh.py:                     0.61 sec   0.0 MB
    - ..\examples\feature_demo\plot_show_scene.py:                          0.58 sec   0.0 MB
    - ..\examples\feature_demo\plot_geometry_polyhedron.py:                 0.57 sec   0.0 MB
    - ..\examples\feature_demo\plot_scene_overlay.py:                       0.57 sec   0.0 MB
    - ..\examples\feature_demo\plot_physical_color.py:                      0.57 sec   0.0 MB
    - ..\examples\feature_demo\plot_image_plus_points.py:                   0.54 sec   0.0 MB
    - ..\examples\feature_demo\plot_transparency1.py:                       0.51 sec   0.0 MB
    - ..\examples\feature_demo\plot_synced_video.py:                        0.51 sec   0.0 MB
    - ..\examples\feature_demo\plot_show_image.py:                          0.50 sec   0.0 MB
    - ..\examples\feature_demo\plot_volume_render2.py:                      0.50 sec   0.0 MB
    - ..\examples\feature_demo\plot_world_bounding_box.py:                  0.49 sec   0.0 MB
    - ..\examples\feature_demo\plot_geometry_plane.py:                      0.44 sec   0.0 MB
    - ..\examples\feature_demo\plot_picking_color.py:                       0.44 sec   0.0 MB
    - ..\examples\feature_demo\plot_volume_slice2.py:                       0.43 sec   0.0 MB
    - ..\examples\feature_demo\plot_volume_slice3.py:                       0.42 sec   0.0 MB
    - ..\examples\feature_demo\plot_wireframe2.py:                          0.37 sec   0.0 MB
    - ..\examples\feature_demo\plot_volume_slice4.py:                       0.36 sec   0.0 MB
    - ..\examples\feature_demo\plot_wireframe1.py:                          0.35 sec   0.0 MB
    - ..\examples\feature_demo\plot_line_basic.py:                          0.34 sec   0.0 MB
    - ..\examples\feature_demo\plot_geometry_torus_knot.py:                 0.33 sec   0.0 MB
    - ..\examples\feature_demo\plot_line_thick.py:                          0.32 sec   0.0 MB
    - ..\examples\feature_demo\plot_volume_slice1.py:                       0.31 sec   0.0 MB
    - ..\examples\feature_demo\plot_geometry_polyhedron_subdivisions.py:    0.29 sec   0.0 MB
    - ..\examples\feature_demo\plot_geometry_klein_bottle.py:               0.29 sec   0.0 MB
    - ..\examples\feature_demo\plot_geometry_image.py:                      0.27 sec   0.0 MB
    - ..\examples\feature_demo\plot_picking_points.py:                      0.25 sec   0.0 MB
    - ..\examples\introductory\plot_cube.py:                                0.24 sec   0.0 MB
    - ..\examples\introductory\plot_light_basic.py:                         0.24 sec   0.0 MB
    - ..\examples\feature_demo\plot_line_segments.py:                       0.22 sec   0.0 MB
    - ..\examples\feature_demo\plot_flat_shaded_torus.py:                   0.22 sec   0.0 MB
    - ..\examples\feature_demo\plot_show_util.py:                           0.20 sec   0.0 MB
    - ..\examples\introductory\plot_offscreen.py:                           0.19 sec   0.0 MB
    - ..\examples\feature_demo\plot_line_performance.py:                    0.18 sec   0.0 MB
    - ..\examples\feature_demo\plot_post_processing2.py:                    0.00 sec   0.0 MB
    - ..\examples\feature_demo\plot_sponza_scene.py:                        0.00 sec   0.0 MB
    - ..\examples\feature_demo\plot_post_processing1.py:                    0.00 sec   0.0 MB
    - ..\examples\feature_demo\plot_show_stl.py:                            0.00 sec   0.0 MB
    - ..\examples\cube_qt.py:                                               0.00 sec   0.0 MB
    - ..\examples\cube_wx.py:                                               0.00 sec   0.0 MB
    - ..\examples\integration_qt.py:                                        0.00 sec   0.0 MB
    - ..\examples\introductory\points_basic.py:                             0.00 sec   0.0 MB
    - ..\examples\light_view.py:                                            0.00 sec   0.0 MB
    - ..\examples\text_with_qt.py:                                          0.00 sec   0.0 MB

@FirefoxMetzger
Copy link
Contributor Author

Adding this link here for future reference.

It is possible to specify a custom static thumbnail image for a gallery example. This can be useful for gui-specific examples that we can't run on CI. For these, we can either run it offline and insert the thumbnail, or we could display a static image (eg the logo of the GUI backend used) for those examples.

Here is the link to the relevant docs: https://sphinx-gallery.github.io/stable/configuration.html#providing-an-image-for-the-thumbnail-image

@FirefoxMetzger
Copy link
Contributor Author

FirefoxMetzger commented Nov 27, 2022

I've reorganized this PR to make use of Display and introduced the ability to configure how gallery examples are built using comments of the form

# sphinx_gallery_pygfx_<name> = <value>

This builds on top of sphinx-gallery's mechanism, which takes the form of # sphinx_gallery_<name> = <value> (which I have discovered today). This is nice because we can configure sphinx-gallery to remove these comments from the produced RST and IPYNB files, making the result look clean. To this, I've added a pygfx namespace to isolate the variables we care about.

With this, we can easily control when to render. To create a render below a given example's code block one has to specify

# sphinx_gallery_pygfx_render = True

This will trigger our scraper to check the global scope for a variable named "renderer". The variable to check can be controlled by specifying

# sphinx_gallery_pygfx_target_name = "<var_name>"
# eg.
# sphinx_gallery_pygfx_target_name = "disp"

inside the block. The resulting variable is assumed to be a Display, Renderer, or Canvas, and is used to render the image to display.

I've also added plumbing to procude animations/GIFs by (via # sphinx_gallery_pygfx_animage = True); however, this is not fully implemented yet, because I first need to figure out how to generate video using pygfx (#381).

I've also fixed the __file__ issue mentioned earlier.

What is missing now is to set up lavapipe on the CI runners that build the docs (GH Actions and RTD). However, it should already be possible to build the gallery locally as part of the normal doc-building procedure.

@FirefoxMetzger FirefoxMetzger linked an issue Nov 27, 2022 that may be closed by this pull request
@FirefoxMetzger
Copy link
Contributor Author

Yes, because WEBP uses YUV420. I will have a look if we can control the compression quality 👍

That said, we already improve over the default quality that comes out of GIF (and its 256 colors) 🥲

image

@FirefoxMetzger
Copy link
Contributor Author

FirefoxMetzger commented Dec 7, 2022

@Korijn Now we are using lossless compression, so any compression artifacts should disappear. This comes at the cost of additional runtime though, so we may need to revisit this once our examples or animation needs have grown to the limits of RTD build time.

Korijn
Korijn previously approved these changes Dec 7, 2022
Copy link
Collaborator

@Korijn Korijn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great!

almarklein
almarklein previously approved these changes Dec 7, 2022
docs/conf.py Show resolved Hide resolved
@FirefoxMetzger
Copy link
Contributor Author

@Korijn I didn't get a notification that you resolved the conversation (GH doesn't seem to do those), so I missed that we can merge this until now.

Master/Main has moved on, so I have re-synced the branch and resolved a merge conflict manually. Take a look and see if it still looks good :) From my side, we are still LGTM.

almarklein
almarklein previously approved these changes Dec 12, 2022
Korijn
Korijn previously approved these changes Dec 12, 2022
@FirefoxMetzger
Copy link
Contributor Author

@Korijn the validation tests broke because of a pathing issue after the ids update. That's now fixed :)

@Korijn Korijn merged commit 79b4c66 into pygfx:main Dec 12, 2022
@Korijn
Copy link
Collaborator

Korijn commented Dec 12, 2022

Awesome work!

@FirefoxMetzger FirefoxMetzger deleted the sphinx-docs branch December 12, 2022 15:56
@FirefoxMetzger
Copy link
Contributor Author

And with that the gallery is live! Nice!

@almarklein
Copy link
Collaborator

Love the LSD backgrounds! 😆

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants