Recommended Workflow for Academic Writing

[amichuda]

jupyter-book provides a great way to create reproducible reports in multiple formats and with myst-markdown, myst-nb and jupytext creating interactive and informative websites becomes easy and intuitive.

But in the context of academic writing, it's not as clear to me what the workflow would be to use jupyter-book if the end format was something like a latex PDF. Currently, even though I'm pretty python-centric, I use Rmarkdown and bookdown to write and render my job market paper because it has a preview of the code cells, kable for making nice looking tables and it's sort of set up well for a latex PDF from the start.

I want to go over to jupyter-book, but I'm finding it hard to figure out a sane workflow. Is it to write everything in a jupyter notebook, convert to myst-md, change tags, add figures, and the like and then convert to PDF? It seems okay, but that extra step of .ipynb -> myst-md seems a bit cumbersome. I know that there's no support as of now for live preview, but is there an easy to hack my way to getting it to work?

Not meant as a criticism at all, I'm just looking for ideas and I'd like to know what your experiences have been like in terms of academic writing.

[mmcky]

hi @amichuda -- we are all looking forward to the project developing to better support LaTeX outputs for academic writing. jupyter-book is currently plugged into the default sphinx pdf writer for generating the pdf (via LaTeX) but more development work needs to take place to better support academic writing (i.e. papers etc.). We are working to make LaTeX better in the medium term. Currently we are going through a harmonisation phase to get what we currently have working nicely together between html and pdf outputs through projects such as sphinx-multitoc-numbering to get consistent numbering between chapters and parts. We will then turn our attention to making pdf output more configurable.

Is it to write everything in a jupyter notebook, convert to myst-md, change tags, add figures, and the like and then convert to PDF?

With jupyter-book you can use ipynb or myst-md as source files (or a combination of both) -- but it is easier (in my view) to change tags via myst-md etc. I typically use vscode for writing myst-md files - I find this extension useful: https://marketplace.visualstudio.com/items?itemName=ExecutableBookProject.myst-highlight particularly for syntax highlighting.

There is a previewer but currently it doesn't render all myst syntax.

https://marketplace.visualstudio.com/items?itemName=ExecutableBookProject.myst-highlight#preview-enhancement

an update on its status: executablebooks/myst-vs-code#18

Contributing:

If you had a real-world public Rmarkdown/bookdown project it would be useful as a test case as new features are being thought through. I haven't used that workflow (myself) so it would also be a good learning case for me to see how that workflow builds etc. I would also like to learn a bit more about kable as it would great to improve table support.

[amichuda]

Thank you so much for the thoughtful reply. I appreciate your tips on your workflow. I actually have the vscode extension installed (I installed it as soon as I heard about the project, it's been great). I guess the only thing missing would be to have the preview also run the code in the code blocks, so that the preview showed the output as well as the writing.

In terms of a public bookdown project, I'm not sure I can share my draft (due to data privacy, as well as not having the time right now to make a reproducible version), but I think that the bookdown website, itself is a bookdown project whose source code can be found on GitHub:

https://github.com/rstudio/bookdown/tree/master/inst/examples

In regards to kable, the closest thing that python has is probably python-tabulate, but it's strength of the flexibility with which The table can be configured, as well as options for captions and labels for latex.

[mmcky]

thanks for the link @amichuda -- those examples will be useful and I will take a closer look at python-tabulate

[mmcky]

I guess the only thing missing would be to have the preview also run the code in the code blocks

Please raise a feature request for this on https://github.com/executablebooks/myst-language-support

[amichuda]

thanks for the link @amichuda -- those examples will be useful and I will take a closer look at python-tabulate

Of course, this is an awesome and happy to help when I can (will try out the hide-cell latex extensions ASAP).

I just checked and I think there's already one! executablebooks/myst-vs-code#16

[amichuda]

Just stumbled upon this as well, might be useful for @AakashGfude and jupyterbook-latex

https://jterrace.github.io/sphinxtr/singlehtml/index.html

[mmcky]

thanks @amichuda

[amichuda]

Hi all! Wanted to chime in on this and give some updates/thoughts.

I've just posted a (rough!) pull request to vscode-jupyter to allow for code cells in markdown files to be used in the python interactive window. This, I think, will allow us to, hopefully, make it more easy to write MyST notebooks directly without having to use Jupyter.

Once that is working, I the next step would be to create an LSP for MyST markdown so that we could get both markdown and python autocomplete. I think @chrisjsewell has his pretty awesome rst-language-server which from the looks of it has plans to be used or MyST at some point. I think that would make the whole workflow thing with MyST notebooks be pretty smooth and we can write notebooks with admonitions and rich MyST markdown without having to go to jupyter notebooks and then converting, and then adding code cell options.

If jupyterbook-latex continues to develop as it's been jupyter-book can be a pretty cool solution for academic writing! Provided we can add support for abstracts at some point, I guess.

In any case, this project is super awesome! Thank you for all the hard work