Support Python server extension

[fcollonval]

Problem

Some existing Jupyter extensions have a Python server part. It will be nice to have a story to integrate them in JupyterLite.

From technical point of view, here are some challenges

• JupyterLite has a server side in JavaScript that can not execute Python code by default
• As all code must be executed in the client, the HTTP requests define by the Jupyter extensions must be converted to regular JavaScript call between the client code and the server code.
• For performance reason, the server code must not block the main web browser thread
• The server code should be asynchronous

Proposed Solution

The proposed solution (WIP) would be to have a Python interpreter using Pyodide running as a service worker to execute the Python server code.

The final goal would be for that service worker to be a singleton shareable by multiple extensions. This will mimic the current server app behavior and avoid multiplying Pyodide instances.

flowchart TB
    C[Client] --- S[JupyterLiteServer]
    S ---|Proxy request| W 
    subgraph Mocked ServerApp
    W[Pyodide Service worker] --- E[Jupyter Server Extension]
    end

Loading

One idea to ease the maintenance of the two interfaces (for JupyterServerApp and JupyterLiteServer), would be to provide tooling base on an OpenAPI spec.

Additional context

• Code generator from OpenAPI spec: https://openapi-generator.tech

[jtpio]

must be converted to regular JavaScript call between the client code and the server code.

Would it have to? Currently fetch is already overriden in JupyterLite which allows for the frontend to not be modified, but still be able to "talk" to the /api/sessions, /api/settings... endpoints. So we could imagine having the same for this Jupyter Server extension compat layer.

Also linking to relevant discussions about using Jupyverse as a server (although different in scope):

• `jupyverse` as a backend #372
• Use jupyverse as a "real" server #779

[jtpio]

Also this could be implemented and experimented with in a separate extension first (for example under the jupyterlite organization), instead of having it in core lite.

[fcollonval]

Currently fetch is already overriden in JupyterLite which allows for the frontend to not be modified, but still be able to "talk" to the /api/sessions, /api/settings... endpoints. So we could imagine having the same for this Jupyter Server extension compat layer.

That makes a lot of sense

Also this could be implemented and experimented with in a separate extension first (for example under the jupyterlite organization), instead of having it in core lite.

Yes - I'm even guessing that on the long run this should stay as an separate official extension to allow for light version and only bring it if some packages are needing it.