Should Quarto source text be markup instead of markdown?

[adamiturabi]

Description

The original case for markdown was that the source text file should be human readable. And for various historical and technical reasons, markdown became the source text format for Quarto.

But does anyone else feel that qmd files are not really read (in their final form) as documents, and so markdown is unnecessarily limiting? For example, there are restrictions and complications on (combinations of) table formatting, footnote paragraphs, bidi, spans, divs, etc.

Markdown still has the advantage that the source text cannot be too complicated and that makes it easier for Quarto/pandoc to render the output in different formats (HTML, PDF, etc.).

But would it be better if the source text for Quarto was a markup language meant to be read by the machine? And human readbility, while important, would be a secondary consideration. I don't mean something crazy like xml. Rather, something like what Typst is trying to do with Tex. A clean start with better fundamentals. And of course it will evolve and get more complex. But hopefully the better fundamentals will provide a good balance of power and maintainability.

Can the new qml at least be an option along with qmd?

One possible low hanging fruit could be the use of braces {} (or other convenient delimiter) to fundamentally support nested constructs.

Thoughts?

[mcanouil]

You can write raw HTML, raw LaTeX, raw Typst, and raw OpenXML.

If you are suggesting to create a fully new markup, that's very unlikely to happen in my opinion and not sure to see what this would bring exactly.
Quarto markdown is already an extended version of Markdown.

What you are suggesting would be a completely new tool / breaking change as it wouldn't be compatible with all rmarkdown/quarto documents of the past 10+ years.
Having to maintain two markup languages is also very unrealistic considering the size of the Quarto core team (3).

FWIW, you are opposing Markdown to markup languages, but Markdown is a markup language.

[adamiturabi]

You can write raw HTML, raw LaTeX, raw Typst, and raw OpenXML.

The issue is, correct me if I am wrong, that the raw HTML will only go the HTML output, not to a potential additional PDF or other outputs for example. So if you are targetting multiple outputs then it isn't very convenient.
I know that qmd has added new HTML format for tables that can potentially render into PDF but it doesn't seem to work well when combined with spans within the raw HTML.

What you are suggesting would be a completely new tool / breaking change as it wouldn't be compatible with all rmarkdown/quarto documents of the past 10+ years. Having to maintain two markup languages is also very unrealistic considering the size of the Quarto core team (3).

This definitely makes sense. I agree mine is a unrealistic proposal given the practical limitations. But I just wanted to get feelings on if it made technical sense. And thanks for the reply and for the awesome work!

FWIW, you are opposing Markdown to markup languages, but Markdown is a markup language.

Yes, technically that is true. But "down" in its name is meant to highlight its primary consideration: which is that the source formatting should first and foremost be human readable as a final format. My feeling is that Quarto users may not share this primary consideration.

[mcanouil]

You can use the conditional content feature, but if your idea was to write HTML for Typst or these kind of things -- this is also very unlikely to happen, especially because you seem to want to mix markups.

It's also a bit niche in my opinions

Note that if you want any content to be processed, you could eventually use Lua to transform a raw code block into AST, then let Quarto/Pandoc write the output, but beware that this is a rabbit hole without any guarantee it will work. This is pretty advanced and possibly messy.

Most users use simple markdown and use R/Python libraries to output complex content.

[adamiturabi]

because you seem to want to mix markups.

Not really. I was thinking that if there was a new markup with more of the power of HTML/Tex/Typst than what markdown provides, then it would be amenable to translate complex content into all of these final formats.

Note that if you want any content to be processed, you could eventually use Lua to transform a raw code block into AST, then let Quarto/Pandoc write the output, but beware that this is a rabbit hole without any guarantee it will work. This is pretty advanced and possibly messy.

Most users use simple markdown and use R/Python libraries to output complex content.

Do you happen to know any repos that do this kind of processing (into the AST, etc) for complex content? I can try to study their code and learn from it. Thanks again!

[mcanouil]

Almost all users using R use packages like tinytable, gt, kable, and more
For Python users, there is great-tables and other.

Regarding the AST processing, not to that extent because that's pretty advanced / deep waters which basically require a user to know the internals of Quarto (which is easier these days using LLMs), but that's still a rabbit hole for no clear reason to go there in the first place.

[cscheid]

Quarto's plain-text document format is going to be (some close variant of) Markdown for the lifetime of the project, including Quarto 2. This is not a discussion that I think is fruitful, so I'm not going to encourage it. You can always write project pre-render scripts to convert from your choice of language to QMD.