Meta-ticket: Improve live documentation in the Jupyter notebook

[videlec]

Since #20690 we have access to documentation (with live execution)
within the Jupyter notebook. It was decided to keep the code simple.
There are two ongoing projects within Jupyter for having this working.
It might be wiser to wait for them...

Some of the improvements mentioned in #20690 were:

• [ ] Configure Sphinx to add a small header to our html page with
  Thebe configuration: use the Jupyter instance serving the page.
  We currently use window.location.origin; is this the right thing?

• Currently it takes 10s for 100 prompts while some sage files
  contain up to 1000 prompts. Profile Thebe and optimize it or use
  a separate thread to properly support large files.

• Expand the activate button with a menu or other widgets for user customization
  of the Jupyter server. This typically would let the user choose between:

  • tmpnb (will only be useful for Sage
    when tmpnb will include a Sage kernel; there are ongoing discussions for providing an alternative sage-enabled tmpnb)
  • a local Jupyter server,
  • whichever Jupyter server the browser is currently connected to?
  • a user-specified server.

• Check whether Jupyter could be configured to dynamically
  negotiate incoming connections that don't fall within the
  -NotebookApp.allow_origin pattern, by opening a user dialog such
  as "Page xxx requests starting a new kernel on this server; do
  you accept? yes/no/always for this site".

• Add support in Thebe for customizable (continuation) prompts, with striping and splitting as above, and automatic setting of the kernel. The customization option could look like:

  prompts = {
      "sage: ": {continuations=["....:", "...  "], kernel="sagemath"},
      ">>>> ": {continuation="...  ", kernel="python"}
  }
  

• Have Thebe provide good TAB-completion.

• Let the cursor switch to the next cell after evaluation (SHIFT+ENTER).

• Extract support for Thebe as a Sphinx extension with (see the related sage-cell extension):

  • Explicit markup to specify which code blocks are editable and with
    which kernel, or setup kernel auto-detection from the prompt.
  • The possibility for setting a default value for the above.

• Refactor the Sage Sphinx configuration to use the above.

• Configure sage-sample / sage-package to use it, to enable live documentation for Sage packages.

• Add support in Thebe for basic export to Jupyter notebooks. A
  quality loss (in particular in terms of the hierarchical
  structure) is acceptable.

• Allow to save the current state of the webpage as a .ipynb worksheet, to be able to continue working on it later, see [[/ticket/20690#comment:53]].

Tickets:

• Make Sage documentation functional #33309 Make Sage documentation functional (disables use of old thebe)
• Remove old thebe package #33529 Remove old thebe package
• Provide live documentation support in Jupyter using Thebe #24593 Replace Thebe by ThebeLab for live documentation support
• sagemath_doc_html: Use jupyter-sphinx for 3D graphics #33507 sagemath_doc_html: Use jupyter-sphinx for 3D graphics
• Rework on live sage documentation #33320 Rework on Thebe for live sage documentation

See also:

• Meta-ticket Meta-ticket: Sage docbuild #20080: Sage docbuild

Depends on #20690

CC: @vbraun @rbeezer @nthiery @hivert @slel @sagetrac-fcayre @jdemeyer @dimpase @kwankyu

Component: documentation

Keywords: thebe, thebelab

Issue created by migration from https://trac.sagemath.org/ticket/20893

[slel]

comment:3

Fix formatting in ticket description.

Regarding "There are two ongoing projects within Jupyter for having this working",
would someone add links?

[slel]

comment:4

Editing ticket description to add link to
tmpnb issue 203: "Octave kernel and SageMath kernel".

[nthiery]

comment:11

Update: Min RK reported to me that he had started reimplementing Thebe on top of JupyterLab. At this stage, this seemed to confirm our hope that Thebe will become just a thin layer on top of plain JupyterLab, making it sustainable.

[nthiery]

comment:14

A source of inspiration: sympylive.

[sagetrac-tmonteil]

comment:15

Replying to @nthiery:

Update: Min RK reported to me that he had started reimplementing Thebe on top of JupyterLab. At this stage, this seemed to confirm our hope that Thebe will become just a thin layer on top of plain JupyterLab, making it sustainable.

Is an export to a standalone .ipynb file part of the plan (so that people can save their work on some live doc) ?

[nthiery]

comment:16

Replying to @sagetrac-tmonteil:

Is an export to a standalone .ipynb file part of the plan (so that people can save their work on some live doc) ?

Ah, shoot, I forgot to discuss this specific point.

That being said, Thebe is meant to be embedded in any web page. So, at the level of Thebe itself, and without a minimum of structure specified, I believe it would be hard to define a meaningful export to .ipynb.

Plausibly we would instead be shooting for a link "download document as ipynb" pointing to the notebook generated from the original sources, either statically or on the fly. The inconvenient being of course that users won't keep their history in this scenario if they play with Thebe before switching to the notebook.

[nthiery]

Changed keywords from none to thebe, thebelab

[mkoeppe]

comment:18

This ticket needs clarification / update. Is it a meta ticket?

[mkoeppe]

comment:21

Sage development has entered the release candidate phase for 9.3. Setting a new milestone for this ticket based on a cursory review of ticket status, priority, and last modification date.