Percent scripts: accept raw-string (r""") and '''-delimited markdown cells, as jupytext writes them

[fqueiro]

Description

Quarto renders Python percent scripts and the docs name jupytext as the reference implementation of the format, with markdown cells written either as # comments or as a multi-line string (""").

Jupytext writes markdown cells as strings when cell_markers = '"""' is set, and since 1.13.0 it prefixes the string with r whenever the cell contains a backslash, so that LaTeX such as \frac is not read as a Python escape sequence (changelog entry for jupytext/jupytext#836; src/jupytext/cell_to_text.py, "Markdown cells that contain a backslash should be encoded as raw strings"; expected output in tests/functional/others/test_raw_strings.py). Its reader accepts """, r""", R""" and the ''' forms (src/jupytext/cell_reader.py).

Quarto's parser recognises only a bare """ line:

// src/core/jupyter/percent.ts, markdownFromJupyterPercentScript
const isTripleQuote = (line: string) => !!line.match(/^"{3,}\s*$/);

so a cell jupytext wrote falls into the comment branch, which only strips a leading #, and both delimiter lines are rendered as prose. The same applies to a cell a user writes as r""" by hand to keep Python from interpreting the backslashes (a plain """ cell containing \xi or \underline is a syntax error in Python, so the raw prefix is the only way to keep such a file importable or runnable).

Minimal example

doc.py:

# %% [markdown]
# ---
# title: "Raw-string markdown cell"
# ---

# %% [markdown]
r"""
# A heading

Display math: $\frac{a}{b}$
"""

quarto render doc.py (1.8.25; the parser is unchanged in 1.10.18, 1.11.3 and main) produces a document whose body, as plain text, reads

r”“” A heading
Display math: \(\frac{a}{b}\) ““”

The heading is lost (the # was stripped as a comment prefix) and both delimiters appear as text, smart-quoted by Pandoc. Replacing r""" by """ renders correctly, which confirms the cause.

Proposed change

Accept an optional r/R prefix on the opening delimiter, and the single-quote form, matching jupytext's reader:

const isTripleQuoteOpen = (line: string) => !!line.match(/^[rR]?("{3,}|'{3,})\s*$/);
const isTripleQuoteClose = (line: string) => !!line.match(/^("{3,}|'{3,})\s*$/);

No other part of the pipeline is affected: the cell content is taken as markdown source either way.

Quarto version

1.8.25 on Windows 11; the relevant code is identical at tags v1.10.18 and v1.11.3 and on main.

[mcanouil]

@fqueiro please follow and use the bug report template in the future as detailed in the issue picker and contributing guidelines.

Thank you for the detailed report.

isTripleQuote in src/core/jupyter/percent.ts matches a bare """ line only, so a cell opened with r""" falls into the comment branch, the
leading # of the heading is stripped as a comment prefix, and both delimiter lines are passed through as prose.

Two corrections to the references, so the record is accurate:

• jupytext#836 was closed and never merged.
  The change that shipped in jupytext 1.13.0 is jupytext#850.
  The CHANGELOG links #836 because that is where the discussion started.
• You report Quarto 1.8.25, but src/core/jupyter/percent.ts does not exist in that release.
  At v1.8.25 the file is src/execute/jupyter/percent.ts.
  The regex is identical in both, so the conclusion stands.

Two notes on the proposed change:

• Raw f-strings must stay out.
  Rf""" was proposed in the #836 discussion and declined.
  jupytext's reader accepts the prefixes "", "r" and "R" only.
• jupytext limits the string-cell form to Python (cell_reader.py, if self.ext == ".py").
  Quarto's percent.ts runs the same check for .py, .jl, .r and .q.
  A bare """ opens a docstring in Julia, and ''' and the r prefix mean nothing in R or q, so the new delimiters need to be scoped to
  Python.
  The closing delimiter should also be required to match the opening one.

[fqueiro]

Thank you, and noted on the template for next time.

Both corrections are right: the jupytext change that shipped in 1.13.0 is jupytext/jupytext#850, and at v1.8.25 the parser lives at src/execute/jupyter/percent.ts. I agree with the three constraints: Python only, no raw f-strings, and a closing delimiter that matches the opening one.

In case it helps whoever picks this up, a change along these lines in markdownFromJupyterPercentScript does it, keeping the bare """ rule for .jl, .r and .q:

const tripleQuotePattern = language === "python"
  ? /^[rR]?("{3,}|'{3,})\s*$/
  : /^("{3,})\s*$/;
const openingQuote = (line: string) => line.match(tripleQuotePattern)?.[1][0];
const isClosingQuote = (line: string, quote: string) =>
  !!line.match(new RegExp(`^${quote}{3,}\\s*$`));
const cellContent = (cellLines: string[]) => {
  const quote = cellLines.length > 2 ? openingQuote(cellLines[0]) : undefined;
  if (quote && isClosingQuote(cellLines[cellLines.length - 1], quote)) {
    return asCell(cellLines.slice(1, cellLines.length - 1));
  }
  return asCell(cellLines.map(stripPrefix)); // commented
};

rf""" and f""" fail the optional-prefix pattern and fall back to the comment branch, as does a """ cell closed by '''. Against tests/unit/percent-format.test.ts this passes the existing cases plus new fixtures for """, r""", R""", ''' and r''' (each producing the same markdown as its comment-form twin), the rejected forms, and unchanged Julia and R behaviour. Happy to share those fixtures if useful.