Improving the Typst base template to match equivalent PDF options

[christopherkenny]

Description

I've had to create several new template classes for my work in Quarto recently. I made the switch to using Typst for all of them, but still using Quarto as the frontend.

As a heavy Quarto + LaTeX user prior, it's clear that there are many fewer features available for -typst formats compared to -pdf formats. It looks like most of this is driven by (1) the relative youth of Typst (~5) compared to LaTeX (~40) and (2) the simplicity of the upstream Pandoc template.

Typst itself is a very intuitive alternative to LaTeX and (at least to me) is substantially easier to learn and use than LaTeX ever was/is. As such, I think it would be great if some features of the Typst templates could be caught up to -pdf outputs.
To this end, I've listed out many features where the implementation would be only a few lines and thus hopefully easy to maintain. Of course, in aggregate, this would add quite a few features at once.

I'm starting here with (1) default suggestions, (2) example syntax for how it would be implemented based on the YAML variables, and (3) details explaining the rationale behind any choices. If there are changes that would be helpful, I'm happy to make a (series of) pull requests to introduce these features to Quarto's templates.

One important thing to note: Some of these examples may need to be prefixed by # to start code while others may need to have # removed. Each of those is context-dependent on where the "best" place to insert such a fix would be. The rest of the content would stay the same.

Finally, I realize this is a long list of possibilities.
If you're not open to a pull request on some (or any) of these points, then this discussion just documents how to get the "equivalent" behaviors in Typst formats as in the other pdf formats.

The rest of this proceeds as follows:

• Introduces a general comment on passing variables Quarto -> Pandoc -> Typst.
• Compares the PDF options and Typst options. Sections are organized following headings on the PDF options page.
• Discusses the layout options that would require more work.

Quarto -> Pandoc -> Typst

Pandoc aggressively escapes characters that can cause issues for Pandoc's processing, but would be valid in Typst. This can make it hard to pass things like colors, emails, or urls from Quarto to Typst. Specifically, if you pass a variable from Pandoc (say $my-variable$) to Typst, then $my-variable$ gets escaped. So, if you pass email: chris@example.com and try to make "$email$", it becomes chris\@example.com. For things like emails, a simple Typst-evaluated regex might suffice, but it falls apart if you don't know all possible escapes.

Instead, if you make [$email$], it passes it as "context". This avoids nasty escape characters. Yet, context is messier to work with. The easiest way to extract text from context is a function like:

#let to-string(content) = {
  if content.has("text") {
    content.text
  } else if content.has("children") {
    content.children.map(to-string).join("")
  } else if content.has("body") {
    to-string(content.body)
  } else if content == [ ] {
    " "
  }
}

This short function comes from an old issue but handles this problem very well.

Adding a function like this would also fix printing errors in templates, like the IEEE one. If you render that template, you can see how the Pandoc escaping makes the email links print incorrectly and have invalid links.

Simple updates based on template options

Title & Author

subtitle

Default: none

if title != none {
    if subtitle != none {
        align(center)[#block(inset: 2em)[
            #text(weight: "bold", size: 1.5em)[#title]
            #parbreak()
            #text(weight: "bold", size: 1.25em)[#subtitle]
        ]]
        } else {
            align(center)[#block(inset: 2em)[
            #text(weight: "bold", size: 1.5em)[#title]
        ]]
    }
}

This example solution is from a prior discussion and was made by @mcanouil.

thanks

Default: none

if title != none {
    align(center)[#block(inset: 2em)[
        #text(weight: "bold", size: 1.5em)[
            #title #if thanks != none {
                footnote(thanks, numbering: "*")
                counter(footnote).update(n => n - 1)
            }
        ]
    ]]
}

Details: This may actually be one of the most complicated suggestions, as it interacts with the title and a counter. However, this is just adding a footnote and using the "*" numbering. We then reset the counter to 0, which could be as simple as counter(footnote).update(0). In case something gets stuck before the title, we can use n => n - 1.

Fonts

linestretch

Can be specified as any non-negative number.

Default: 1 (single spacing)

set par(leading: linestretch * 0.65em)

Details: 0.65em is the measurement for one line.

mathfont

Default: New Computer Modern Math

#show math.equation: set text(font: mathfont)

Details: New Computer Modern Math is shipped with Typst.

codefont

Default: DejaVu Sans Mono (shipped with Typst)

#show raw: set text(font: codefont)

Details: DejaVu Sans Mono is shipped with Typst.

Colors

linkcolor

Default: "#800000"

#show link: set text(fill: rgb(to-string(linkcolor)))

Details: Matches Quarto's PDF template default.
Related discussion: #9175

filecolor, urlcolor

Default: "#800000"

#show link: this => {
    if type(this.dest) != label {
        text(this, fill: rgb(to-string(filecolor)))
    }
}

Details: Try to match Quarto's PDF template default, but not possible in a simple way.

Note: These can't be separated without some regex, as far as I know.

citecolor

Default: "#0000FF"

#show ref: this => {
    text(this, fill: rgb(to-string(citecolor)))
}

Details: Try to match Quarto's PDF template default, but not possible in a simple way.
This catches both crossreferences and bibliographic references.

Note: These can't be separated, as far as I know.

toccolor

Default: "#0000FF"

#show link: this => {
    if type(this.dest) == label {
        text(this, fill: rgb(to-string(filecolor)))
    }
}

Details: Not 100% sure the default, but this looks right to me from the LaTeX.

Metadata

keywords

Default: [don't set]

#set document(keywords: keywords)

Details: Defaults to an empty array in Typst.

title-meta

Default: title (or [don't set])

#set document(title: title-meta)

Details: Defaults to none in Typst.

author-meta

Default: [don't set]

#set document(author: author-meta)

Details: Defaults to an empty array in Typst.

date-meta

Default: auto

#set document(date: date-meta)

Details: Set by Typst to current date by default

More difficult layout problems

layout

These have very simple adjustments in theory.
Each fix is a line or two, but they are currently hardcoded when converted from Quarto to Typst.
For example, see this line.

cap-location

Problem: This is hardcoded when creating the figures and tables.
Otherwise, changing the default should be as simple as combining both the table and figure suggestions.

#show figure.where(kind: "quarto-float-fig"): set figure.caption(position: cap-location)
#show figure.where(kind: "quarto-float-tbl"): set figure.caption(position: cap-location)

fig-cap-location

Problem: This is hardcoded when creating the figures.
Otherwise, changing the default should be a single line.

Default: "bottom"

#show figure.where(kind: "quarto-float-fig"): set figure.caption(position: fig-cap-location)

Note: Also listed in Figures in the PDF Options document.

tbl-cap-location

Problem: This is also hardcoded when creating the figures.
Otherwise, changing the default should be a single line.

Default: "top"

#show figure.where(kind: "quarto-float-tbl"): set figure.caption(position: tbl-cap-location)

Note: Also listed in Tables in the PDF Options document.

[mcanouil]

Thanks for the suggestions.

Note that a lot of the YAML options come directly from Pandoc and the template there.
For instance, Quarto updates Pandoc thus the templates at each new Quarto releases.

Also, I believe most of your suggestions should be applied on Pandoc side as I did for subtitle.

cc. @gordonwoodhull

FYI:

• Typst: 1.6 introduces Pandoc interpolated variable to typst-template.typ rather than typst-show.typ #10212

[gordonwoodhull]

This is a really helpful catalog of areas where Typst output could be brought up to the level of Latex. Thanks!

I agree with @mcanouil that most of these changes should be made on the Pandoc side. For instance, we will pick up his subtitle fix when we upgrade to Pandoc 3.2.1+ in Quarto 1.6.

I would suggest contributing the Pandoc features directly to Pandoc!

I am not sure, but I think the "more difficult layout problems" may require changes on the Quarto side. If you would be willing to do a little more background research to see where these changes need to be made, that would be super helpful.

Please keep in mind that while we have a long-term plan for how to review and accept changes that improve feature parity between formats, that is at least some months away. (Among other things, we need guidelines and processes on how to add tests, docs, and feature-format matrix entries.)

[christopherkenny]

Thank you both for the feedback. I can start an issue at Pandoc (edit: see jgm/pandoc#9956) that goes through the Pandoc pieces of this. I was not entirely clear how much templates differ between Pandoc and Quarto.

My sense is that the layout problems would require Quarto changes, but I will look into how much is required there. Notably, since Quarto uses its own figure type, then any layout directives need to be handled here, separately from Pandoc. (This is because you have to target them with figure.where(kind: "quarto-float-***"), as Quarto does not insert standard "table" or "image" kinds.)

[mcanouil]

I was not entirely clear how much templates differ between Pandoc and Quarto.

The idea is to have the smallest amount of differences to ease the process of Pandoc update.

Notably, since Quarto uses its own figure type, then any layout directives need to be handled here, separately from Pandoc. (This is because you have to target them with figure.where(kind: "quarto-float-***"), as Quarto does not insert standard "table" or "image" kinds.)

To be accurate, it's not figure. It's "whatever" including figures and tables.
See https://quarto.org/docs/authoring/cross-references-custom.html.

[christopherkenny]

Thank you, that's helpful to know.

[cscheid]

Hi @christopherkenny, thanks for the long post.

Let me add a bit of context to some of our design goals and decisions, so that you understand where we might land on eventual PRs you might make. Then I'll make some specific comments.

• We prefer "features" that work uniformly across formats
• We prefer our templates to feel familiar to users coming from "non-Quarto workflows" of that format
• We prefer our templates to feel familiar to users coming from "pure Pandoc workflows" for that format

These three goals mean that we want Quarto to work, in general, as close to a "strict Pandoc superset".

Specific comments

cap-location Problem: This is hardcoded when creating the figures and tables.

I'm not sure I understand: cap-location can very much be controlled by users authoring the document via YAML configuration. In case you mean something like "the control of a Typst feature should be possible in Typst", let me respond to that.

Quarto is very much designed in a way that doesn't prioritize the cleanliness of the intermediate output files. It's not that we're unwilling to make changes that make the output files cleaner; it's that we don't consider this to be particularly important. We instead choose to focus on uniformity of experience for Markdown and ipynb inputs, and will almost always prioritize that over cleanliness of output files.

Quarto does intend the intermediate outputs to be consumable by downstream tooling without requiring Quarto to be installed. The typical use case here is that we want the intermediate latex files to exist and be available for submission to a journal that requires LaTeX sources. If (hopefuly when!) journals start accepting raw Typst as their format submission, we intend Quarto's Typst output to be directly usable as input to such submissions. But Quarto does not (and will not) ever emit Typst code that is as clean as you could write by hand.

Next steps

I realize that this might mean that some of your proposed changes do not match our design goals.

I think the best way to incorporate your suggestions into Quarto is for you to start with the very smallest PR you can envision, so that we can together experiment with how that feature interacts with Quarto's other formats and our design goals.

[christopherkenny]

Thank you for the additional context. I've started an issue upstream about the pandoc template, which based on your comment and the others is the best course of action for any non-Quarto-specific edits. Unless you suggest otherwise, I think it might be wise to wait for any thoughts/considerations there before starting on any next steps here, given the feedback.

As for the cap-location point, I think I goofed. Sorry about that. I'm not sure exactly what I did, but I convinced myself that cap-location and the tab/fig-cap-location arguments did nothing. I'm glad those arguments work natively.

It may be worth clarifying the "hardcoded" comment. My best understanding from tracing though the code is:

• If no specific yaml position is given for the caption when a figure is created (in the .qmd file), then this gets processed to the default around here
  • The default will always take on a value because the last option is to take from crossref, as defined starting here.
• Whatever value is accessed at this point is the passed to position: within figure.caption() either here for a figure with subfigures or here otherwise

In effect, this means that any creation of a caption will always be accompanied by either position: top or position: bottom.
Typst should always respect that value then, as far as I know, since that's directly setting an option above and beyond a set or show rule.

This is likely where the Typst-Quarto tradeoff is:

• Using Typst, I would want any show-set rules to apply to all figures that I have not explicitly said to do something else with.
• Using Quarto, I would want it to pick sensible defaults, which may require that it explicitly says to do something.

[cscheid]

In effect, this means that any creation of a caption will always be accompanied by either position: top or position: bottom. Typst should always respect that value then, as far as I know, since that's directly setting an option above and beyond a set or show rule.

This is likely where the Typst-Quarto tradeoff is:

• Using Typst, I would want any show-set rules to apply to all figures that I have not explicitly said to do something else with.
• Using Quarto, I would want it to pick sensible defaults, which may require that it explicitly says to do something.

That's correct, and this is an instance of what I meant by Quarto not adhering to "the control of a Typst feature should be possible in Typst". If a feature can be controlled in the Markdown document (either through classes, attributes, or YAML metadata), then we don't consider it a bug that the resulting behavior is "hardcoded" in Typst.

To quote my original response again, "Quarto is very much designed in a way that doesn't prioritize the cleanliness of the intermediate output files.". I will say something even stronger: Quarto considers the .typ output to not be suitable for humans to read or edit. That is by design, and allows us sufficiently many favorable tradeoffs that we are not going to budge on that.

[christopherkenny]

Thank you for the additional information on that.

For what it's worth, I came across this behavior while working on a format extension which makes a PDF using Typst, not while trying to use the intermediate output for anything (but debugging). I found the behavior surprising, but this context is very helpful. Thanks.