📑 Add literalinclude directive

[rowanc1]

This supports a new directive, literalinclude as well as all of the additional options across RST and Sphinx.

RST documentation:

• https://docutils.sourceforge.io/docs/ref/rst/directives.html#including-an-external-document-fragment

Sphinx documentation:

• https://www.sphinx-doc.org/en/master/usage/restructuredtext/directives.html#directive-literalinclude

See executablebooks/jupyterlab-myst#189 and https://github.com/orgs/executablebooks/discussions/1026

There are a few more tweaks that will be necessary to bring this to jupyterlab-myst to expose the transform and extend to not having to read from disk, etc.

cc @nthiery @matthewfeickert @fperez who were asking for this over the last few months.

[rowanc1]

This fixes a portion of the bug in:

• code-block rendering does not work as expected jupyterlab-myst#174

Line number is now turned on if lineno-start is included.

[fwkoch]

This looks great - I didn't dig through all the logic around line numbers, etc, in detail, but testing and documentation both look thorough 👍

Despite all the logic complexity added, there was nice simplification around previously duplicated options/directives. My only comments are non-critical.

[nthiery]

I confirm that the dev version of myst supports smoothly the literalincludes
in my documents.

Since I could not yet test it with JupyterLab, I actually started mystmd for the
first time on a real project of mine. Wow, this is sooo fast. True real time
rendering. This a game changer for authoring.

Guys, it's amazing what you are building!!!

Can't wait to switch over from Jupyter-book for my whole toolchain.

[rowanc1]

⚡️💨🚀

Really glad you are up and running with the CLI -- we will get it over to Jupyter soon!

[fperez]

Pinging also @minrk who may have content using these.

Haven't had a chance to test it yet, but wanted to thank @rowanc1 & team for listening to our input! This is looking wonderful, will provide concrete feedback as soon as I have any. Hugely apreciated!

[nthiery]

Dear @rowanc1,

There seems to be a little glitch in the tex export when literal-including indented code.
Here is an example:

I have a file mystere-correction.cpp containing:

void mystereTest() {
    /// BEGIN mystereTests
    CHECK( mystere(0) == ""             );
    CHECK( mystere(1) == "*\n"          );
    CHECK( mystere(2) == "*\n**\n"      );
    CHECK( mystere(3) == "*\n**\n***\n" );
    /// END mystereTests
}

It's included as such:

:::{literalinclude} mystere-correction.cpp
:start-after: BEGIN mystereTests
:end-before: END mystereTests
:::

The generated TeX looks as such; mind the indentation of the first line:

\begin{verbatim}
CHECK( mystere(0) == ""             );
    CHECK( mystere(1) == "*\n"          );
    CHECK( mystere(2) == "*\n**\n"      );
    CHECK( mystere(3) == "*\n**\n***\n" );
\end{verbatim}

Let me know if more detailed instructions to reproduce would be helpful.

The generated html is fine.

[fwkoch]

@nthiery - I pulled your above comment to a separate issue so it wasn't lost: #614