.md files with executable code

[rmhorne]

Hello everyone,

First, this a great tool! At the moment I am trying to work on a workflow/ pipeline using Quarto that does the following:

1. Allows note taking in Obsidian with Zotero integration and code snippets that create some output (graphs, charts, maps, etc).
2. Publish the results in Quarto.

Essentially I want to be able to use the normal Obsidian note taking process (with the on-the-fly creation of new notes, the network view, etc), add in code blocks (in my case python) that can be rendered in quarto, and then publish the results. I am ~90% of the way there, but I have run into some issues, and maybe this list (or the devs!) could help.

My biggest issue / roadblock is Quarto's limitation on running executable code in .md files. Although I can just change the extension on all of my .md files to .qmd when I want to render them (and the results are great!), doing so breaks my linkages within my documents and some of the functionality within Obsidian. It would be a lot more convenient if I could just use something like

engine: jupyter

in the project .yaml file (or a yaml block within the file itself) that allows Quarto to render code in a .md file. Is there any way to override the current block on .md code execution, or any future plans to allow it? I realize that this could also be solved in the other direction by more extensive support for .qmd files in Obsidian, but I think this kind of functionality might be useful for other workflows / processes that have code in .md files.

Thanks!

[cscheid]

Currently, we assume .md files are intermediate markdown files pretty extensively throughout the codebase. I wouldn't want to rely on using .md files as input because there's a risk some part of our codebase will step on those files. (This is something we will 100% fix in the future, but it'll be a while until we can get to the bottom of it.)

I would recommend you save those with some other extension, possibly through a project script that happens automatically ahead of quarto render.

[rmhorne]

Thanks, and I'm glad to hear that it is something that will be fixed in the future!

A project script could work - in general I was trying to construct this pipeline for an academic humanities / non technical audience, but a quick script to change extensions, then crawl through .md files and change the relative links there before rendering might be a good way forward.

[cscheid]

You can create custom project types in extensions, so your audience should need very little to no configuration to get things started: https://quarto.org/docs/extensions/project-types.html

[jeffkimbrel]

I've been dabbling with trying to better integrate executable documents in R with stand alone markdown in Obsidian, and too have been struggling. I used to have a better solution with R markdown, but haven't found a solution with Quarto. What worked for me in Rmarkdown was to have the .rmd file outside of Obsidian, but then render to a markdown file inside of my Obsidian folder structure. Knitting allowed me to send the file somewhere via the front matter...

knit: (function(inputFile, encoding) {
    rmarkdown::render(inputFile, 
      encoding = encoding, 
      output_dir = "/PATH/TO/OUTPUT/)
    })

Figures could be a little wonky, as the links to them weren't always preserved exactly, but I could live with it. I would also have a callout in the output file saying this was just a rendering, and where to find the "real" document should I need to edit and run again.

For Quarto, I've been having trouble getting a markdown file to be written as "complete" as it was previously, and I haven't been able to send it to a different folder. Any suggestions? I'm also open to other ways to get Quarto and Obsidian to integrate.

[mcanouil]

Not clear to me how what you are trying to do differs from the initial message here.

An answer has been provided by Quarto team stating that you can't do what you want, i.e., rendering md files, etc.

Regarding output dir, the R package is for now not up to date.
However, you could call Quarto CLI yourself using for example system().
quarto render file.qmd --output-dir PATH/TO/OUTPUT

[jeffkimbrel]

I was just trying to show I had solved the original question problem (as I understand it) by flipping the source and destination around. I think the original question was rendering from .md inside Obsidian to an external file, where I was suggesting instead rendering from .qmd outside of Obsidian to .md inside Obsidian. I'm not trying to execute .md, rather write to .md.

Thanks for the idea of just command line rendering... I live and die by the "render" button, but I don't need to be!

A last idea for the original message... There is an obsidian plug in (txt-as-md) that can be modified to show non-.md files in Obsidian... so you can just have your .qmd file itself live in your obsidian vault and see it as though it is markdown. I've asked a search plug-in to find them as well: scambier/obsidian-omnisearch#58

I also found this, but doesn't seem tested: https://github.com/odebroqueville/obsidian-qmd-as-md

[mcanouil]

Thanks for the detailed explanation.

If you want to output markdown, you can use gfm format (or any markdown based format from Quarto/Pandoc) such as the one described in #4436.

[jeffkimbrel]

Thanks, some good links to follow on there, particularly setting a destination for images.

[baptiste]

For the quarto publishing step, I think you could simply include the .md file in a dummy .qmd wrapper, as in

---
title: "including"
format: html
---

{{< include _includee.md >}}

where _includee.md has the content,

Quarto enables you to weave together content and executable code into a finished document. To learn more about Quarto see <https://quarto.org>.

## Running Code

When you click the **Render** button a document will be generated that includes both content and the output of embedded code. You can embed code like this:

```{r}
1 + 1
```

You can add options to executable code like this 

```{r}
#| echo: false
2 * 2
```

The `echo: false` option disables the printing of code (only output is displayed).

[mcanouil]

This does not solve the possible between files "linkage" issue described in the first post.

[baptiste]

it's unclear to me what linkages are discussed in that first post: I took it as meaning the links created internally by Obsidian, in which case I assume they should still work (but I'd need to see a concrete example to know how Quarto would interpret them – if they're just relative paths, then it should be fine, I think).

The other way around – quarto-generated output, etc. – well it's not clear from the OP that Obsidian would ever see them in this workflow, since that would require modifying in-place the original input md document (which cannot be run again, so a destructive operation). If a new document is created and imported in Obsidian for the rendered output, then it should be enough to specify the right flavour of markdown as quarto/pandoc output format.

[mcanouil]

Exactly, as I pointed out a moment ago #3715 (reply in thread)

PS: I am not at all familiar with Obsidian.

[mcanouil]

FYI (@cderv originally posted in #5449)

Obsidian related discussion https://forum.obsidian.md/t/quarto-support-open-source-scientific-and-technical-publishing-system/32625/36 and Obsidian plugin https://github.com/danieltomasz/qmd-as-md-obsidian

Just for reference for anyone looking for Obsidian and Quarto

[nichtich]

My failed workaround to this problem was to create symlinks from each file.qmd to file.md and explicitly exclude the .md files in config:

project:
  render:
    - "!*.md"

But this fails with an AlreadyExists error. Any suggestions how to allow keeping a .md file as source is welcome!

[cderv]

What are you trying to do exactly ? Maybe you should open a new discussion about your use case.

.qmd is the source file for Quarto, it is like .md but with special Markdown content that Quarto can handle, (like computational code block). It will create a .md from this .qmd containing all the results from your evaluation, so that it can be converted using Pandoc to an output format.

This is why you may have issue about already exist error if you have .md with same name as .qmd.

If you want to keep the intermediate .md produced by a .qmd rendering you can use keep-md: true - though it will be using some Markdown syntax specific to what Quarto needs for its feature.

Hope it helps