feat(myst plugin): Implement working MySt Plugin

[vijay-prema]

No description provided.

[nokome]

Thanks for this @vijay-prema.

A little bug I noticed: when encoding, there is no blank line after admonition, table, figure etc. These need to be encoded with a \n\n at the end like paragraph does.

There are a few Stencila node types for which encoding is not implemented but which can be. We should ensure these types are also handled in decoding also (i.e. ensure we have full roundtrip encoding/decoding).

• Block node types:

  • Claim (analogous to a MyST "proof" see https://mystmd.org/guide/proofs-and-theorems, note there are different claim types)
  • CodeChunk (equivalent to a MyST code-cell see https://mystmd.org/guide/notebooks-with-markdown)
  • IncludeBlock (equivalent to an embed directive see https://mystmd.org/guide/embed)
  • QuoteBlock (supported by MyST see https://mystmd.org/guide/typography#quotations)
  • Section (equivalent to MyST block see https://mystmd.org/guide/blocks)

• Inline node types:

  • AudioObject & VideoObject (encode like an image, this is what we do for Stencila Markdown also, see https://mystmd.org/guide/figures#videos)
  • Cite & CiteGroup (we don't have support in Stencila Markdown yet, but may as well implement encoding since MyST does https://mystmd.org/guide/citations#markdown-citations)
  • CodeExpression (equivalent to a MyST {eval} https://mystmd.org/guide/notebooks-with-markdown#myst-inline-expressions)
  • Note (this includes footnotes which MyST supports see https://mystmd.org/guide/typography#footnotes)

[rowanc1]

Expressing so much excitement for this work. Thank you both and excited to chat through how I can help next week!

[nokome]

Some notes from me before we discuss later today...

Plugin integration

• I've been working on adding codecs as something that plugins can provide (currently it is limited to kernels and assistants)
• This was progressing fine until I hit a circular dependency in our internal Rust crates - that requires some refactoring by me to avoid that
• Due to a recent change in the serialization of the Cord type (a string with character level attribution) there will need to be a few changes to the decoding functions in the PR (but they are minor)
• I need to do some more thinking around how we pass the internal node_id to the plugin so it can be used in the encoding mapping. Currently we are (ab)using the id property - which is probably OK, but which might also conflict if the user is using id for something.

MyST directives for AI instructions and suggestions

Now, for the interesting stuff 😃 . We want to add MyST directives (block level) and roles (inline level) for Stencila's InstructionBlock and InstructionInline nodes. This would allow users to edit MyST documents in Stencila and make use of AI assistants. These custom directives would be parsed and translated into the instruction node types by this plugin.

I'm currently working on a refactor of instructions (allowing for more control, multiple suggestions and feedback on each), and our focus is on getting the syntax for blocks right first, so here are some very quick sketches of what those might look like:

:::{new}
:assignee: code-table
:reps: 3
:temp: 80
total counts by year and region
:::

:::{suggest}
:feedback: exclude last four years
<the suggested table>
:::

Stencila plugin for MyST

To allow users to use Stencila's assistants in other editors we could create a plugin to MyST https://mystmd.org/guide/plugins (this would effectively be the inverse of this plugin). This would be a different PR that would probably use our Node.js bindings to make use of core functionality for calling assistants etc.