Should LaTeX code used to create PDF differ from LaTeX output? Is this a bug?

[frabjous]

I noticed recently that if I have a markdown document with certain punctuation that is already or becomes “smart” punctuation, that I get different results when converting to LaTeX, then compiling with pdflatex, from when I have pandoc call pdflatex directly to produce a PDF.

Consider:

Hello … there.

I am “here”.

If I convert this to LaTeX (without suppressing the "smart" option) I get:

Hello \ldots{} there. 


I am ``here''.

The ellipsis has been replaced with \ldots{}, and the smart quotes with LaTeX’s original ASCII methods of representing them.

If I compile directly to PDF, however, using the --verbose flag, I can see in that case, that it passes the unicode smart quotes instead, without converting them to LaTeXisms:

[redacted verbose output...]
\begin{document}

Hello … there.

I am “here”.

\end{document}

This happens even if the smart option is explicitly specified with, e.g., pandoc file.md --verbose -t latex+smart -o file.pdf.

It seems strange to me that the LaTeX code it generates with LaTeX output should differ from the code it uses when it uses LaTeX to create the PDF.

I noticed this because I am using a template in which \ldots is redefined. That redefinition would apply to ellipses in the source if I used pandoc to convert to LaTeX then compiled the LaTeX with pdflatex but not if I used pandoc to create the PDF directly, even using pdflatex as the engine.

(I get the same results if I use three dots and dumb quotes in the original.)

I'm not sure if this is a bug or intended behavior, but it is confusing, since if something goes wrong when producing a PDF via LaTeX I usually want to examine the intermediate document, but this is unreliable when the code it uses to create PDFs differs from the code it generates by converting.

I am using pandoc 3.1.11.1 on Archlinux x86_64.

[jgm]

It's intentional. The problem is that with some babel settings, the ligatures we emit with latex when smart is enabled (as it is by default) can interact badly. See commit 0f5c56d and jgm/citeproc#54 for justification.

[frabjous]

Thanks for the explanation.