MyST in JS

[rowanc1]

I was encouraged to create an overview issue to show where we are spending time on the MyST in JS project (see #441 and #442). I will keep the main issue up to date for the next few weeks. I have probably missed/overlooked a few goals - happy to adjust/include based on feedback!

Overview / Goals

• Getting to a single document mode that supports MyST markdown parsing in Javascript
• Directives: admonitions, code, tables, figure, images
• Math: Dollar, AMS, directive/role math
• Roles: Basic HTML (abbr, sub, sup), references (eq, numref, ref)
• <img> tag reading/parsing
• Numbering and cross-references (single page only)
• Frontmatter
• blocks (comment, target, break) Block elements (target, comment, block-break) markdown-it-docutils#16
• Can export functions for other packages to create Directives/Roles
• Various parts of the plugin can be used independently (e.g. just the directives md.use(directivesPlugin, opts))
• Can override things like mathRenderers, etc.
• There is thought into the structure of the HTML (e.g. semantic html where we can)
• Thinking through the token stream and keeping the necessity of custom renderers to a minimum
• There is package usage and developer documentation
• There is an online demo!
• Thinking about style exports more seriously - possibly starting to document some of the html/class names

Most of these things are in some form of complete - goal over the next month is to tighten the basics, while also ensuring appropriate overlap with the existing python ecosystem where there are direct analogues (e.g. amsmath, dollarmath, etc.).

Tactical

Good to go:

• 👌 IMPROVE: Defaut HTML tags markdown-it-amsmath#4
• 👌 IMPROVE: AMSmath defaut HTML tags mdit-py-plugins#27
• 👌 IMPROVE: HTML default tags markdown-it-dollarmath#2
• ♻️ REFACTOR: Simplify render interface markdown-it-amsmath#6
• ✨ NEW: Add semantic HTML roles markdown-it-docutils#15
• 👌 IMPROVE: Admonition structure markdown-it-docutils#12
• 🔧 MAINTAIN: Create module exports for internal classes markdown-it-docutils#11
• ✨ NEW: Add math directive markdown-it-docutils#13

The docutils PRs are mostly independent but will have a few conflicts depending on the order they are merged in. Should be easy to keep up-to-date.

Works, but needs more design / architecture review:

• ✨ NEW: Initial handling of numbering and references markdown-it-docutils#14

Testing / removing from markdown-it-myst

• Test out and move to new packages jupyter-book/mystmd#33

Ongoing Discussion

• The scope of markdown-it-myst - probably a light-weight wrapper on docutils, dollarmath, etc. and packaging. Opinionated includes of "MyST", following basically what is in the jupyterbook docs.
• How far to go in implementing all of the legacy of sphinx/docutils in the JS version - I think it will be a while before we implement cpp:namespace-pop (requires reading cpp code! and .. out of scope of executablebooks likely ..?) e.g. see Create a myst-markdown repository as a ref implementation for myst #305
• Finding the right place to add various related roles and directives (I would like to see docutils stay pretty tight and be a small bundle). Maybe this is another monorepo similar to mdit-py-plugins?

[mmcky]

thanks heaps for putting this together @rowanc1 -- this is a perfect example for the Activity Board too. Love your work.

[mmcky]

How far to go in implementing all of the legacy of sphinx/docutils in the JS version - I think it will be a while before we implement cpp:namespace-pop

In my view we won't need a lot of the code documentation features of sphinx and we shouldn't focus on those elements in the first instance and focus on writing authored documents. Documentation projects can also always build via jupyter-book to get access to these elements.

[rowanc1]

Had a talk with @chrisjsewell to get the final good-to-go PRs (4 left, see above) reviewed and merged in. The final comments on all of them have been addressed, in talking to @chrisjsewell he said he would take a look tomorrow and merge them in.

We had a talk about the state management (numref, ref, eq roles & rendering) and agreed that we would go after a single document mode first. Before moving this PR forward, the other changes should come in (e.g. exports and types have changed slightly).

[mmcky]

thanks for the update @rowanc1 it is exciting to see this coming along.

We had a talk about the state management (numref, ref, eq roles & rendering) and agreed that we would go after a single document mode first.

+1 This makes sense.

[rowanc1]

I am working on a document that helps me organize all of the components of MyST/Sphinx/RST features, mostly for high-grading the pieces of MyST for JS, but this could be a starting place for "myst as a standard" likely.

https://docs.google.com/spreadsheets/d/1ji2vW3XhcTwhcphsVwWwxTcKKXSkwqC1F04RAZK3ASA/edit#gid=0

I have been labelling then with:

• 0 - included in myst in js now (or in a PR coming in tomorrow!)
• 1 - next up, and
• 2 - after single document is complete.

There are also a number of them in red that likely won't make the cut at all for a non-sphinx implementation where program documentation is not the main point. I would love any comments in there or notes on what to prioritize for this push!

[choldgraf]

@rowanc1 what is the status on this issue? It is a bit hard to figure out what are the next steps needed to close it. If this is more like a large and ongoing/evolving effort, maybe we could designate a place for separate backlog items (e.g., issues in a repo, or a github project board), and then put those individual issues on the activity board for review?

[rowanc1]

Yep - that sounds good. Will close this one now that things are merged in!