Misunderstading markdown output to notebooks?

[renefritze]

Before I submit a detailed issue, I wanted to ask if I simply had some wrong expectations. After the sphinx-build run In the notebook .ipynb output I still have non-transformed myst-markdown. Such as Sphinx roles and myst substitutions. My expectation was that those would appear as their html equivalent in the cell markdown.
So are those expectations just wrong, or am I potentially doing something (fundamentally even) wrong/running into a bug?

[chrisjsewell]

Do you mean this is Markdown output from code cells?

[renefritze]

No, markdown that's straight straight from the .md. My tryout is this and the generated tutorial_basis_generation.ipynb has unreplaced substitutions like {{ VectorArrays }} and sphinx roles (math, attr, ref, etc.). In the html output those are all fine.

[chrisjsewell]

Ok cheers, yes they should work

first can you clarify what version of myst-nb you are using, and secondly can you try these substitutions in a standard Markdown file (i.e. one that does not get converted to a Jupyter Notebook) to check whether this is a myst-nb specific issue (or with myst-parser)

[chrisjsewell]

oh wait you were expecting them to be replaced in the generated .ipynb?
No, no content gets converted via sphinx from .md to .ipynb, it is just a (bi-directional) change in format, as explained here: https://jupytext.readthedocs.io/en/latest/formats.html#myst-markdown
The conversion to published content (HTML, LaTeX, etc) comes later

[chrisjsewell]

There is some discussion about this (see e.g. https://github.com/QuantEcon/sphinx-tojupyter) but it not part of the core conversion (the notebooks are the source documents as opposed to the rendered document)

[renefritze]

oh wait you were expecting them to be replaced in the generated .ipynb?

Yes, I was.

No, no content gets converted via sphinx from .md to .ipynb, it is just a (bi-directional) change in format, as explained here: https://jupytext.readthedocs.io/en/latest/formats.html#myst-markdown
The conversion to published content (HTML, LaTeX, etc) comes later

Alright, thanks for clearing that up.

There is some discussion about this (see e.g. https://github.com/QuantEcon/sphinx-tojupyter) but it not part of the core conversion (the notebooks are the source documents as opposed to the rendered document)

Great, I'll look into that next. Thanks again!

[NIDS2020-instructor]

I would like to jump on this bandwagon as it may be exactly related to what i was searching for. So is that expected that a Jupyter notebook launched with a Jupyter kernel does not display 100% well compared to the Jupyter book html version associated with that notebook ?

I just started with Jupyter book and I downloaded the example (quantecon-mini-example), I was expecting to interact with the Jupyter notebook while displaying it in a similar way as the Jupyter book html page appears. So with quantecon-mini-example i was expecting to locally run Jupyter to interact with mini_book/_build/jupyter_execute/docs/python_by_example.ipynb and display a similarly beautiful rendering as on mini_book/_build/html/docs/python_by_example.html. However i can see a few un-rendered features, such as non interpreted local hyperlinks ((ourfirstprog)= and {ref}our first program <ourfirstprog>, etc.).

So from this discussion should i assume this is normal and not all features are expected to render well in a Jupyter notebook associated to a Jupyter book file ?

Then what would be the recommended approach to write a Jupyter Book so that to optimize compatibility between the Jupyter Book and its associated notebooks when one would like to sometimes read the book but also interact with the notebooks ? I installed jupytext but i am not sure of its use for this purpose.