Moving from Sphinx 1.3.1 to 1.5.3 breaks LaTeX compilation of links rendered as code

[stefano-maggiolo]

Hi! Not urgent as we just used a workaround, but in case you care...

We have an extension that creates a link to a file in our GitHub repo, and renders it as code.

With 1.3.1 it worked. With 1.5.3 I get errors like these:

! Argument of \T1\. has an extra }.
<inserted text> 
                \par 
l.492 ...dev/cms/tree/v1.3.dev0/config/}{config/}}
                                                  . You want to copy them to...

Runaway argument?
! Paragraph ended before \T1\. was complete.
<to be read again> 
                   \par 
l.492 ...dev/cms/tree/v1.3.dev0/config/}{config/}}
                                                  . You want to copy them to...

I get the same error both locally (stock TeXlive distro in Ubuntu 14.04) and in ReadTheDocs.

[jfbu]

Can you please paste lines 485 to 495 of the foo.tex document raising error message like you reported ?

actually I expect this is from https://github.com/cms-dev/cms/blob/17a924e333025bc4e70e675731f09cf8a8305313/docs/Running%20CMS.rst#configuring-cms

Currently I can not get same LaTeX error, but I do observe (expected) problems with for example this

\sphinxcode{\href{https://github.com/cms-dev/cms/tree/v1/config/}{config/}}

in LaTeX file, as the link target in PDF contains some TeX macros and is not valid.

[jfbu]

I expect this to have same causes as those which gave #3200 (fixed at #3206). The \hyperref (or rather here \href) dislikes "active" characters (this is TeXnical...).

Perhaps the encode_uri in LaTeX writer should get is \detokenize. But needs evaluation.

[stefano-maggiolo]

Looks similar. The offending tex is: \sphinxcode{\href{https://github.com/cms-dev/cms/tree/v1.3.dev0/config/}{config/}}

[stefano-maggiolo]

[I already updated the repo to remove the code formatting from that macro, that's probably why you weren't seeing it.]

[jfbu]

no I downloaded the gh_links.py extension in the version you linked to initially. Not knowing which reST source to use I tried https://github.com/cms-dev/cms/blob/17a924e333025bc4e70e675731f09cf8a8305313/docs/Running%20CMS.rst#configuring-cms which generates

\sphinxcode{\href{https://github.com/cms-dev/cms/tree/v1/config/}{config/}}

Then I added manually what you indicated

\sphinxcode{\href{https://github.com/cms-dev/cms/tree/v1.3.dev0/config/}{config/}}

but at my locale, there is no LaTeX error on compilation; sure the hyperlinks are wrong as is to expected from the way \sphinxcode works. It might be that if Sphinx core changes to fix the hyperlinks then also it fixes the failed compilation you have. I tried on TeXLive 2013 in case something changed with hyperref package but still do not get LaTeX error. To better resolve your issue, it would help if you could create small project with gh_links.py extension which exhibits it. And at least answer my query about pasting more lines of offending LaTeX code, is this in some environment? have you customized LaTeX in any way?

[jfbu]

ah, I reproduce your error in an sphinxalltt environment. Was your link rendered as code originally inside contents of a parsed-literal directive ?

[jfbu]

arrrgh, apologies, all my testing was on a project I used for issue #3200 and it was left with latex_elements setting 'sphinxsetup': 'inlineliteralwraps=false' which explains why I did not see the failed LaTeX run. This being said; 'inlineliteralwraps=false' is not a cure to your issue as although the LaTeX build completes, the hyperlink is wrong.

[stefano-maggiolo]

Hi @jfbu - just to set expectations, I am fine with the current workaround (just render the link with the regular style). Thanks for looking into it!

[jfbu]

Sorry in advance if I am a bit longish here, but I am documenting for future, as immediate all-around fix does not present itself in an obvious way.

The underlying issue is that inline literals, using \sphinxcode macro, do not only achieve a style (use mono space font) but also do things like allowing TeX to insert linebreaks at some locations where it would not, or make sure backquote is rendered as ` and not as a curly quote etc... this makes it related to Verbatim contexts where also a new "catcode regime" is put in place.

But the \href LaTeX macro does not like such Verbatim or "exotic catcodes" contexts. It prefers to be at top level, and if in argument of a macro, the context should be the standard one for "catcodes". The \href LaTeX macros does not use the 1999 e-TeX extensions which would allow it to normalise its argument in a more powerful way than the TeX code it currently uses. The fix at #3206 for analogous issue with \hyperref macro wraps its argument in a \detokenize but this can not be immediately employed here by latex writer for \href because it is not compatible with things like \# or \% in the URI. And \#, \% must be employed in \href if it is not at top level.

Getting back to your issue. As workaround, add this to preamble:

latex_elements = {
    'preamble' : r"""
\makeatletter
\let\originalsphinxcode\sphinxcode
\protected\def\sphinxcode#1{{%
   \def\sphinxbreaksafteractivelist{\do\,\do\;\do\?\do\!}% drop / and .
   \def\sphinx@literal@nolig@list{\do\`\do\<\do\>\do\'}%   drop -
   \originalsphinxcode{#1}}}%
\makeatother
"""
}

You may need to drop \do\? also, if ? is used in the URLs.

Investigating this, I realized that the related but distinct issue #3317 (which as fixed as consequence of #3340) about URLs inside parsed-literals are still a problem there if the \href macro is not at top level but wrapped in any arbitrary LaTeX macro (like your extension does, with \sphinxcode, but now it could be any macro).

I hope workaround above will be enough for your project.

[jfbu]

I forgot to say the obvious:

Just use 'preamble' as above but with

\protected\def\sphinxcode#1{\texttt{#1}}

if you want to revert to Sphinx LaTeX 1.3.1 behaviour in this regard.

This may be the simpler solution for your use-case, but you lose wrapping at line-breaks and non-curly single quotes in PDF output for inline literals

[jfbu]

It seems I need to talk a lot before acting. I have fixed this issue at PR #3538. Thanks for reporting.

[jfbu]

fixed at 3128a43 for Sphinx 1.5.4. Please re-open in case for of (!) further problems.