Handling the examples

[maximlt]

The HoloViz projects are built with an /examples folder that contain Jupyter Notebooks. It serves multiple purposes:

• contains notebooks that are ultimately part of the documentation (originally they actually contained most of the documentation, supplemented by RST files in the /doc folder)
• contains notebooks that are tested with nbsmoke (originally, now nbval is also used in places)
• contains notebooks that are distributed in the package, and together with the examples command added by pyct it makes it easy for users to "install" the notebooks, running e.g. panel examples --path . to get a local copy of the examples folder

While this concept has pros, it also has some cons:

• writing documentation in a notebook file is nice, but moving documentation around between notebook files is awkward. I think that contributes to making the documentation difficult to refactor.
• it makes linking difficult to apprehend, e.g. linking between a Notebook in the examples folder and an RST/MD file in the doc folder isn't intuitive. I think that contributes to HoloViz sites having generally poor linking.
• it requires an additional and somewhat awkward build step, to generate RST files in the doc folder that include the NotebookDirective
• distributing the /examples folder contributes to increasing the package size, in particular because of the datasets it may contain (e.g. about 3MB for HoloViews, compressed). This is becoming a problem now we're installing code in the browser where you usually want to have your packages as light as possible.

We should find a way to better handle that situation.

Some things worth considering:

• users are going to be more and more able to execute the notebooks online without downloading the examples folder
  • on binder (some HoloViz projects have it set up but it's more of an after thought and I believe is not often checked/updated by the maintainers)
  • the pyodide directive will make the sites more and more executable and we could build JupyterLite-based websites for each HoloViz project. Note that currently Datashader doesn't work on Pyodide.
  • users can download particular notebooks they are interested in directly from the websites

My suggestions:

• we should no longer distribute the /examples folder in our packages.
• we actually should no longer have an /examples folder in our repos, everything should be in the /doc folder
• we should test whatever contains executable code in the /doc folder (Testing the Markdown files that use the pyodide directive #242)
• we should build JupyterLite sites for each project, containing the full website (including only Markdown and Jupyter Notebook files)
• we should have an extra doc build step that makes the examples available somewhere. That step would convert the example files if need be (e.g. MyST test-based notebooks to Jupyter Notebooks using jupytext). It would make them available in a compressed format (zip I guess). In their getting started section (or wherever they see fit) the HoloViz websites could link this archive to let users know how to get the examples locally. Interestingly it means that having that as an extra doc build step means that we could add e.g. a conda lock file for users to be able to also re-create the environment. If the examples are all hosted somewhere in a predictable place (e.g. on S3), we could create a holoviz-examples package that would collect them all part of its build step.