Fancy HTML table not included in typst after latest dev version (works on latest 1.5.47)

[cderv]

This is not minimal but it is reproducible for me

---
title: fancy gt table
format: typst
---

```{r}
#| echo: false
#| message: false
library(tidyverse)
library(gt)
library(palmerpenguins)

spanners_and_header <- function(gt_tbl) {
  gt_tbl |>
    tab_spanner(
      label = md("**Adelie**"),
      columns = 3:4
    ) |>
    tab_spanner(
      label = md("**Chinstrap**"),
      columns = c("Chinstrap_female", "Chinstrap_male")
    ) |>
    tab_spanner(
      label = md("**Gentoo**"),
      columns = contains("Gentoo")
    ) |>
    tab_header(
      title = "Penguins in the Palmer Archipelago",
      subtitle = "Data is courtesy of the {palmerpenguins} R package"
    )
}

penguin_counts <- penguins |>
  filter(!is.na(sex)) |>
  mutate(year = as.character(year)) |>
  group_by(species, island, sex, year) |>
  summarise(n = n(), .groups = "drop")

penguin_counts_wider <- penguin_counts |>
  pivot_wider(
    names_from = c(species, sex),
    values_from = n
  ) |>
  # Make missing numbers (NAs) into zero
  mutate(across(.cols = -(1:2), .fns = ~ replace_na(., replace = 0))) |>
  arrange(island, year)

desired_colnames <- colnames(penguin_counts_wider) |> 
  str_remove('(Adelie|Gentoo|Chinstrap)_') |> 
  str_to_title() |> 
  set_names(nm = colnames(penguin_counts_wider))

penguin_counts_wider |>
  mutate(across(.cols = -(1:2), ~ if_else(. == 0, NA_integer_, .))) |>
  mutate(
    island = as.character(island),
    year = as.numeric(year),
    island = paste0("Island: ", island)
  ) |>
  gt(groupname_col = "island", rowname_col = "year") |>
  cols_label(.list = desired_colnames) |>
  spanners_and_header() |>
  sub_missing(missing_text = "-") |>
  summary_rows(
    groups = TRUE,
    fns = list(
      "Maximum" = ~ max(.),
      "Total" = ~ sum(.)
    ),
    formatter = fmt_number,
    decimals = 0,
    missing_text = "-"
  ) |>
  tab_options(
    data_row.padding = px(2),
    summary_row.padding = px(3), # A bit more padding for summaries
    row_group.padding = px(4) # And even more for our groups
  ) |>
  opt_stylize(style = 6, color = "gray")
```

This should give a nice fancy table and it does in latest 1.5 prerelease, but no more on main. On main, it is no table in the doc without any error or warning. Table is silently not included.

1.5.47	main

Not great for 1.5 release 😞

Let's find out what caused this ! I wondered if my environment is broken somehow... 😭

because I also see during typst rendering this

error: the argument '--v8-flags[=<v8-flags>...]' cannot be used multiple times

Usage: deno run [OPTIONS] [SCRIPT_ARG]...

[cderv]

Ok, so I believe this is a Windows only problem in there

quarto-cli/src/resources/filters/normalize/parsehtml.lua

Lines 53 to 59 in 1b2c802

	local quarto_path = pandoc.path.join({os.getenv('QUARTO_BIN_PATH'), 'quarto'})
	local jout = io.popen(quarto_path .. ' run ' ..
	pandoc.path.join({os.getenv('QUARTO_SHARE_PATH'), 'scripts', 'juice.ts'}) .. ' ' ..
	juice_in, 'r')
	if jout then
	return jout:read('a')
	else

And more over probably dev-version specific. Why ? because built version on windows is quarto.exe while dev version the binary is quarto.cmd and we should account for this.

I'll do some test to confirm and offer a fix.

But I believe built version of Quarto (pre-release and release) are ok related to this issue.

[gordonwoodhull]

Thanks for catching this! I think I remember discussing this before, but I guess we didn't fix it. 🤔

[cderv]

The problem here is that when we are calling quarto with io.popen, all the environment is kept. Meaning all QUARTO_* env var with set for various reason are passed to the script called. I don't think this is what we really expect.

We probably want to call quarto in a clean environment right ?

I am surprised this is not a problem for non-windows. Maybe io.popen is not executing the same, and only on windows environment is inherited.

I trying out different things here, but happy to discuss this if you have thoughts

[cderv]

I can't find an easy way to run in a clean env. I tried pandoc.system.with_environment() but it seems it has an issue windows when restoring the environment as I get

pandoc: unsetEnv: invalid argument (Param�tre incorrect.)

I'll try another workaround... 🤔

[cderv]

error: the argument '--v8-flags[=<v8-flags>...]' cannot be used multiple times

I manage to hack my way to resolve this, and I can confirm this is the issue here. Now the io.popen call to quarto run works as expected and juice is executed, with table being inserted now.

So this is the part to handle with a proper fix

• quick fix: Make sure that the --v8-flag is not set twice
• probably real fix: How to we execute in a clean environment ? Should we call deno run instead here to avoid this kind of problem ?

Talking about deno because this works

--- a/src/resources/filters/normalize/parsehtml.lua
+++ b/src/resources/filters/normalize/parsehtml.lua
@@ -50,8 +50,8 @@ function parse_html_tables()
       local jin = assert(io.open(juice_in, 'w'))
       jin:write(htmltext)
       jin:flush()
-      local quarto_path = pandoc.path.join({os.getenv('QUARTO_BIN_PATH'), 'quarto'})
-      local jout = io.popen(quarto_path .. ' run ' ..
+      local quarto_path = pandoc.path.join({os.getenv('QUARTO_BIN_PATH'), 'tools/x86_64/deno.exe'})
+      local jout = io.popen(quarto_path .. ' run --allow-read ' ..
           pandoc.path.join({os.getenv('QUARTO_SHARE_PATH'), 'scripts', 'juice.ts'}) .. ' ' ..
           juice_in, 'r')
       if jout then

[cderv]

I have fix for quarto.cmd - I'll make a PR for that, also because it unblocks me. I'll check on linux too to understand why it was not detected... Also our CI failed to detect that, but we do use quarto.cmd so it is a weird.

[cderv]

I'll check on linux too to understand why it was not detected...

This is not happening on linux because QUARTO_DENO_V8_OPTIONS and QUARTO_DENO_EXTRA_OPTIONS

Inside our bash script, we are just setting variables, which are not environment variables !! in our quarto.cmd script we don't do that - we read the env variable and set new ones, or just create them as env var.

If we do this, then we recreate the error that happens because of nesting calls in Windows

❯ QUARTO_DENO_EXTRA_OPTIONS='--v8-flags=--enable-experimental-regexp-engine,--max-old-space-size=8192,--max-heap-size=8192' quarto render test.qmd
error: the argument '--v8-flags[=<v8-flags>...]' cannot be used multiple times

Usage: deno run [OPTIONS] [SCRIPT_ARG]...

For more information, try '--help'.

[cscheid]

To make sure I understand well:

• the bug is that quarto run foo.ts from inside Pandoc fails on Windows
• somehow, that's interfering with QUARTO_DENO_EXTRA_OPTIONS.

We can no longer set --v8-flags in QUARTO_DENO_EXTRA_OPTIONS, because we always have to pass at least one such argument (to use v8's new regexp algorithm). That's the reason I added QUARTO_DENO_V8_OPTIONS. Using the latter should work.