Supporting text-based diagrams in Jupyter clients

[bollwyvl]

What are you trying to do?

Make Jupyter clients aware of a broadly-implemented, text-based diagram format for visually describing complex concepts.

How is it done today, and what are the limits of current practice?

MermaidJS supports ~14 common diagrams, loosely inspired by graphviz, markdown, and other sources.

GitLab, GitHub, Azure DevOps, HackMD/HedgeDoc all support integrating these in Markdown with the following syntax

```mermaid
flowchart LR
    chicken --> egg --> chicken
```

if reading this on any compatible client, the above would render like:

flowchart LR
  chicken --> egg --> chicken

Additional open source and proprietery tools offer MermaidJS as a default plugins, such as drawio.

Previous Jupyter extensions exist to integrate with this syntax for MermaidJS such as:

• jupyterlab-markup

While others required using fallback HTML pattern, <div class="mermaid">

• nb-mermaid

Other proprietery and open source tools offer MermaidJS Markdown integration under plugins, but with bespoke Markdown syntaxes.

Beyond MermaidJS, Jupyter client extensions provide similar capabilities for different grammars:

• pikchr
  • jupikchr
• graphviz/dot
  • jupyterlab-graphviz

While these other grammars (for MermaidJS or otherwise) have their merits, the portability of a the common fenced code block syntax for MermaidJS, the long history of active support for MermaidJS, and the breadth of the diagrams it offers, make it a most pragmatic syntax to adopt.

What is new in your approach and why do you think it will be successful?

Bringing native support to Jupyter Markdown continues the existing approach of implementing widely-adopted, simple syntax that give Jupyter users more power in their text-based diagrams.

Who cares? If you are successful, what difference will it make?

Users of non-Jupyter tools will be able to use more of their knowledge of Markdown to create rich interactive diagrams when working with Jupyter tools.

Jupyter users will be able to quickly generate portable diagrams from computational concepts with simple string formatting/templating. This would not incur some of the challenges of existing existing tools, such as graphviz, still
present a challenge to use on non-traditional platforms (Windows, WASM), due the the challenges of binary
compatibility.

What are the risks?

Introducing new features outside of the original Markdown "spec," bugs and all, always presents a risk that other Jupyter tools will not be able to consume authored by other tools.

In the meantime, non-MermaidJS-supporting renderers (e.g. GitHub's notebook preview) would have mermaid code blocks fall back to a fairly human-interpretable syntax, where an arrow is -->, etc.

The MermaidJS "grammar" is actually many different grammars. Creating syntax highlighting, static analysis support, etc. for this format will be non-trivial, but eventually rewarding.

MermaidJS allows for a fair amount of custom JS interactivity. Applying the highest-possible level of strict encoding reduces the chance for this to create unwanted side-effects for users, at the cost of sacrificing links/hover effects, etc.

How much will it cost?

Licensing Costs

MermaidJS is MIT-licensed,
and would incur no additional costs to developers or users for proprietary licenses.

Development Costs

Two PRs already demonstrate this approach, with some initial review feedback, suggesting
a limited additional development effort to move forward:

• JupyterLab #14102
• nbconvert #1957

Runtime Costs

At runtime, rendering a single diagram on a page, served from CDN with brotli compression a MermaidJS flowchart costs 132kb.
This is usually lower than the wire cost of an inline, base64-encoded raster image, a verbose tool-generated vector.
A hand-tuned, externally-hosted and compressed vector diagram would likely cost the least to serve.

The second diagram of the same type would incur no additional assets.

Additional diagram types on the same page will some additional overhead, as some of the features bring in more capable automatic layout engines such as ElkJS.

Maintenance Costs

On an on-going basis, Jupyter tools would need to keep these versions up-to-date to account for any upstream security vulnerabilities, etc. This is required for other widely-used JS libraries, such as MathJax, jQuery, and RequireJS,
as well as the JupyterLab stack.

How long will it take?

The above PRs would need additional review, effort and discussion.

As a purely additive feature, these needn't be added at major versions, but would become part of the "unwritten dependencies," of a given notebook, but joining all manner of other things (e.g. client plugins, kernel, and kernel dependencies).

What are the mid-term and final “exams” to check for success?

Getting rich, text-based diagrams into a pre-release of one official Jupyter tool would allow this to be validated against other Jupyter tool implementations.

Full production releases would make this broadly available for Jupyter users.

[bollwyvl]

I got a couple emoji, and would do some bumps of the PRs if there is any further interest on this.

Some other paths:

• an ipymermaid package which would hoist this to the magic/widget/mime/document for lab, coupled with an nbconvert postprocessor
  • this could use the dirty vega trick to update the mimebundle with image/svg+xml
  • or use the heavyweight mermaid cli (ick, nodejs)
• and/or rip out the special casing of mermaid on the lab PR, making the marked layer extensible (a la jupyterlab-markup so mermaid could be "just" an extension
  • but this would partially defeat the purpose of having some principle-driven consensus of a de facto text diagram format
    • and open the door for completely non-portable markdown

[fcollonval]

@bollwyvl we discuss this at the 🥇 SSC meeting, the conclusion was to open a formal JEP. Do you have time to do so?

[bollwyvl]

Great, coulda done that two weeks ago 😜

I guess making the purpose of this repo clearer (e.g. with issue templates, forms, etc) would help avoid that in the future.

[JohanMabille]

Closing this issue as the JEP has been submitted.