Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Escaped percent signs in LaTeX (\%) still are considered comments in certain contexts #3779

Closed
isaisabel opened this issue Jul 3, 2017 · 1 comment

Comments

@isaisabel
Copy link

In LaTeX, the symbol % is considered the start of a comment. However, when escaped (%) it is considered by the latex compiler to just be a normal % character. Pandoc does not always catch these escaped comment markers correctly, interpreting them as comments in certain contexts.

See the following example:

$ pandoc --version
pandoc 1.19.2.1
Compiled with pandoc-types 1.17.0.4, texmath 0.9, skylighting 0.1.1.4
Default user data directory: /Users/jtrahan/.pandoc
Copyright (C) 2006-2016 John MacFarlane
Web:  http://pandoc.org
This is free software; see the source for copying conditions.
There is no warranty, not even for merchantability or fitness
for a particular purpose.

The example1 latex input file:

$ cat example1.tex
\documentclass{article}
\newcommand{\fakeitemize}[1]{
    \begin{itemize}
        #1
    \end{itemize}
}
\newcommand{\testcmd}[1]{
    #1
}
\begin{document}
    \fakeitemize{
        \item Pandoc is 100\% awesome.
    }

    \begin{itemize}
        \item Pandoc is 200\% awesome.
    \end{itemize}

    \testcmd{
        Pandoc is 300\% awesome.
    }
\end{document}

The output from the first conversion through pandoc. As you can see, the text starting at the escaped percent sign is lost for all text that is passed through arguments into the custom commands:

$ cat example1.tex | pandoc -f latex -t html
<ul>
<li><p>Pandoc is 100 </p></li>
</ul>
<ul>
<li><p>Pandoc is 200% awesome.</p></li>
</ul>
<p>Pandoc is 300 </p>
@jgm
Copy link
Owner

jgm commented Jul 4, 2017

This is going to be fixed by the big rewrite of the LaTeX reader that I'm working on now.

jgm added a commit that referenced this issue Jul 5, 2017
This rewrite is primarily motivated by the need to
get macros working properly (#982, #934, #3779, #3236,
 #1390, #2888, #2118).

We now tokenize the input text, then parse the token stream.
Macros modify the token stream, so they should now be effective in any
context, including math. (Thus, we no longer need the clunky macro
processing capacities of texmath.)

A custom state LaTeXState is used instead of ParserState.
This, plus the tokenization, will require some rewriting
of the exported functions rawLaTeXInline, inlineCommand,
rawLaTeXBlock.
jgm added a commit that referenced this issue Jul 6, 2017
This rewrite is primarily motivated by the need to
get macros working properly (#982, #934, #3779, #3236,
 #1390, #2888, #2118).  A side benefit is that the
reader is significantly faster (27s -> 19s in one
benchmark, and there is a lot of room for further
optimization).

We now tokenize the input text, then parse the token stream.

Macros modify the token stream, so they should now be effective
in any context, including math. Thus, we no longer need the clunky
macro processing capacities of texmath.

A custom state LaTeXState is used instead of ParserState.
This, plus the tokenization, will require some rewriting
of the exported functions rawLaTeXInline, inlineCommand,
rawLaTeXBlock.

* Added Text.Pandoc.Readers.LaTeX.Types (new exported module).
  Exports Macro, Tok, TokType, Line, Column.  [API change]
* Text.Pandoc.Parsing: adjusted type of `insertIncludedFile`
  so it can be used with token parser.
* Removed old texmath macro stuff from Parsing.
  Use Macro from Text.Pandoc.Readers.LaTeX.Types instead.
* Removed texmath macro material from Markdown reader.
* Changed types for Text.Pandoc.Readers.LaTeX's
  rawLaTeXInline and rawLaTeXBlock.  (Both now return a String,
  and they are polymorphic in state.)
* Added orgMacros field to OrgState.  [API change]
* Removed readerApplyMacros from ReaderOptions.
  Now we just check the `latex_macros` reader extension.
jgm added a commit that referenced this issue Jul 7, 2017
@jgm jgm closed this as completed in 0feb750 Jul 7, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants