pandoc 3.14

[jgm]

I'm pleased to announce the release of pandoc 3.14,
available in the usual places:

Binary packages & changelog:
https://github.com/jgm/pandoc/releases/tag/3.14

Source & API documentation:
http://hackage.haskell.org/package/pandoc-3.14

This release includes a lot of bug fixes and small improvements,
including improvements in dependencies, which you can read about
in the changelog. Here are the changes of special note:

• Pandoc now searches a path given in an enviornment variable
  PANDOC_DATA_DIRS when looking up data files (including
  templates, filters, defaults files, metadata files, etc.).
  The user data directory is still tried first. This change
  will make it easier to create distributions of pandoc with
  special customized data files.

• The LaTeX writer now uses custom macros for footnotes in
  tables and definition lists, fixing a bug with hyperlinks
  in the notes in these contexts. Those who use custom templates
  will need to import the definitions of these macros from
  the default LaTeX template.

API change:

• Text.Pandoc.Class exports new functions getDataDirs
  and getPandocDataDirs.

Thanks to all who contributed, especially new contributors
Alain Lesage, Asuka, Caio Vinícius, Jeremy Levartovsky,
Nacho García, tyler000000.

Click to expand changelog

• Support PANDOC_DATA_DIRS for data file lookup (Support $XDG_DATA_DIRS or equivalent for templates and lua filters #7922). If PANDOC_DATA_DIRS is set, the (absolute) directories it lists are searched, in order, for data files not found in the user data directory, before falling back to the default data files. This covers templates, filters, defaults files, metadata files, CSL styles, and other data files. If PANDOC_USER_DATA_DIRS is set, print its value in --version output.

• Reset sourcepos when parsing footnotes and grid table cells (rebase_relative_paths: a footnote or grid table ending a file is rebased against the next file #12001, extends fix for rebase_relative_paths extension: image reference inside final block object of file is incorrectly pathed to next input file of pandoc command #11888). Otherwise, when one of these ends a file, relative paths in it can be rebased against the next file on the command line.

• LaTeX reader:

  • Fix duplicated text after a control space at line end (markdown->latex duplicates entire line with combination of \itshape and \ #12051, Asuka).
  • Handle pandoc’s deferred footnotes (\pandocfootnotemark{n} and \pandocfootnotetext{n}{..}) as produced by the LaTeX writer for notes in tables, captions, etc.
  • Ignore definitions of \pandocbounded, \pandocfootnotemark, and \pandocfootnotetext (e.g. from pandoc’s template), which the reader handles directly. Previously these definitions were expanded, producing garbage and losing content.
  • Fix file-crossing bug in rawLaTeXParser (Raw LaTeX spanning ch9.md and ch10.md crashes: their names sort out of order #12072). The raw LaTeX parser didn’t work properly when crossing from one source file to another.

• Typst reader:

  • Handle columns: auto in grid (grid nested inside figure give an error with pandoc #12061).

• HTML reader:

  • Don’t drop ordered lists and rules inside footnotes (The HTML reader strips <ol>s within the HTML representation of footnotes #11852). Previously an <ol> inside the body of a note was treated as the list of notes, and its items were lost. Similarly, an <hr> inside a note was dropped.

• RST reader:

  • Strip minimum indentation from indented blocks (pandoc incorrectly strips leading whitespace in RST pre-formatted blocks #12071). Previously the indentation of the first line was stripped, so lines with less indentation lost all their leading whitespace. Now, as in docutils, the minimum indentation is stripped and relative indentation is preserved.
  • Require descriptions in option list items (Incorrect parsing of RST text starting with a hyphen #12063).

• ODT reader:

  • Support the styles extension (Styles extension for ODT in pandoc reader #12060).

• Docx reader:

  • Merge adjacent styled code blocks (docx and docx+styles readers process Source Code style differently #9183, Caio Vinícius). Without +styles, consecutive “Source Code” paragraphs are combined into a single code block by combineBlocks. With +styles, each paragraph is first wrapped in a Div carrying its custom-style attribute, so the CodeBlock case no longer matched and every line became its own Div [CodeBlock].
  • Don’t drop contents of customXml element (DOCX reader silently drops text inside inline w:customXml #12050).
  • Read all underline types (Docx reader ignores underline types other than single (double, thick, dotted, wave, words, …) #11988, Patrick Creighton).
  • Follow w:numStyleLink in abstract numberings (Docx Reader: List not converted in specific document #9222, Caio Vinícius).

• Pptx reader:

  • Don’t add spaces between text runs (Pptx reader: spaces added between text runs; line breaks dropped #12016, Nacho García).
  • Read the shapes inside group shapes (Pptx reader: content of grouped shapes is dropped #12017, Nacho García).
  • Read equations and markup-compatibility blocks (Pptx reader: a text box containing an equation is dropped entirely #12025, Nacho Garcia).
  • Read speaker notes (Nacho García, pptx and xlsx readers: speaker notes, charts, hidden slides, numbers and dates #12023). Each slide’s notes are now read into a notes div at the end of the slide’s blocks, the form the pptx writer turns into speaker notes, so pptx -> Markdown -> pptx keeps them.
  • Read charts as tables (Nacho García, pptx and xlsx readers: speaker notes, charts, hidden slides, numbers and dates #12023). Previously, a chart was read as a placeholder text. We now create a table from the chart’s cached data. The table is put in a chart Div whose type attribute holds the chart type. The chart’s title becomes the table captions. The chart code is in Text.Pandoc.Readers.OOXML.Charts.
  • Mark hidden slides with a hidden class on the header (Nacho García, pptx and xlsx readers: speaker notes, charts, hidden slides, numbers and dates #12023).
  • Handle merged table cells (Nacho García).
  • Parse bold, italic, sub/superscript and hyperlinks (Nacho García). Text runs were previously read as plain text, so their formatting and hyperlinks were lost.
  • Read inherited bullets, numbering and nested lists (Pptx reader: read inherited bullets, numbering and nested lists #12042, Nacho García).
  • Skip slide number placeholders (Pptx reader: skip slide number placeholders #12045, Nacho García).

• Xlsx reader:

  • Resolve package-absolute relationship targets (Xlsx reader: fails on workbooks written by openpyxl and pandas (absolute relationship targets) #12013, Nacho García).
  • Read inline strings and rich text as written (Xlsx reader: inline strings dropped; spaces added inside rich-text strings #12014, Nacho García).
  • Look up a cell’s font through its cell format (Xlsx reader: look up a cell's font through its cell format #12033, Nacho García).
  • Read merged cells as spans (Nacho García).
  • Render dates and times (Nacho García).
  • Render numbers as Excel’s General format (Nacho García, pptx and xlsx readers: speaker notes, charts, hidden slides, numbers and dates #12023).
  • Read charts (Xlsx reader: read charts #12043, Nacho García). Charts are now read as tables using the cached data.

• Docx writer:

  • Implement flip-flopping for emph (Roman Text in Italicized Titles #11961). Emph within emph becomes upright. This is important for correct formatting of titles in CSL, among other things. LaTeX does this automatically, and in HTML we achieve it via CSS.

• EPUB writer:

  • Fix id clash with added title header (EPUB writer: fix id clash with added title header. #12003, Alain Lesage).
  • Skip dates that cannot be normalized (EPUB writer: skip dates that cannot be normalized. #12002, Alain Lesage).

• Ipynb writer:

  • Make ANSI escape stripping linear (ipynb reader performance: JSON escaped double quote quadratic in time #7744, Jeremy Levartovsky).

• Vimwiki reader:

  • Support rejected todos (tyler000000).

• LaTeX writer:

  • Fix hyperlinks for deferred footnotes (LaTeX writer 3.12: footnote links in longtables point to missing destinations (xelatex) #11936). Notes in tables, captions, and definition list terms are rendered as a mark followed later by the note text. Previously we used \footnotemark{} and \footnotetext with manual adjustments to the footnote counter. But hyperref anchors each \footnotetext at the destination of the most recent \footnotemark, so when there were several such notes, all of the texts got the last mark’s anchor and the links for the other marks were broken. We now emit \pandocfootnotemark{n} and \pandocfootnotetext{n}{..}, where n is a unique id. These macros, defined in the default template when the deferred-notes variable is set, save the footnote mark and hyperref anchor at the mark and restore them at the text. This removes the need for counter arithmetic, works when the footnote counter is reset by chapter, and handles notes nested inside deferred notes (whose texts are now emitted after the text containing their marks). Note: Users of custom LaTeX templates, or of LaTeX fragments produced without --standalone, will need to include the definitions of \pandocfootnotemark and \pandocfootnotetext from the default template.
  • Allow repeated longtable captions (caption for table with header is not repeated on pagebreak in latex output #4558, Václav Haisman). If a table has a repeat-caption class, captions will be repeated on each page of a longtable.

• HTML styles template:

  • Add emph flip-flopping. So, emph within emph renders as upright, etc.
  • Move @print rules to end (HTML default styles: @media print resets for links, blockquotes and code are overridden by later rules #12064), so they aren’t overridden.

• Text.Pandoc.PDF:

  • PDF via LaTeX: do at most 3 runs when only a TOC is present (PDF via LaTeX: do at most 3 runs when only a TOC is present #12012, Alain Lesage).

• Text.Pandoc.Class:

  • Export new functions getDataDirs and getPandocDataDirs. [API change]

• Text.Pandoc.Citeproc:

  • removeFinalPeriod now acts across Link boundary.

• pandoc-lua-engine: raise lower bound for pandoc to 3.13 (pandoc-lua-engine-0.5.4.2 requires pandoc-3.13 - but doesn't tell #12066).

• Depend on texmath 0.13.3.5, commonmark 0.3.2, commonmark-extensions 0.2.8, citeproc 0.14.2, typst 0.12.1, emojis 0.1.6, skylighting 0.15.0.1.

• New bounds for crypton in pandoc-lua-engine and pandoc.

• Makefile:

  • Fix release-templates target so it pushes tags.
  • Remove default CABALOPTS. These can be put in cabal.project.local. This will ensure that the same options are used by HLS, so that rebuilds aren’t needed.
  • make check-cabal: exit with error status on outdated deps.
  • Reorganize check-cabal; add check-sdist target we can run separately.

• cabal.project: specify +http flag for all ghc versions.

• Minimize the test docx, odt, pptx files. This reduces the size of the test files (and release tarball) by 1.79 MB.

---

This discussion was created from the release pandoc 3.14.

[iandol]

Hi Jon, thanks as always!

If PANDOC_USER_DATA_DIRS is set, print its value in --version output.

There is a small typo in the change log, shouldn't it be PANDOC_DATA_DIRS?

[jgm]

Yes, good catch. Unfortunately I can't change the note in the release (immutable release), but I'll change the changelog in the repository.

[benz0li]

Unofficial Linux/RISC-V (64-bit, linux/riscv64) release assets are available at https://gitlab.b-data.ch/jgm/pandoc/-/releases/3.14.

[shunlog]

congrats on reaching version π!