recent unicode-math upgrade breaks PDF builds if dot character is active when entering math mode

[jfbu]

Description

This provokes a failed PDF build using XeTeX/LuaTeX

(edit: also when using \setmathfont{texgyrepagella-math.otf} or even no setmathfont at all, as long as unicode-math is loaded)

\documentclass{article}
\usepackage{fontspec}
\usepackage{unicode-math}
\setmathfont[Extension = .otf,BoldFont = *bold]{xits-math}

\begin{document}

{\small
\catcode`. \active
\def.{\ERROR}
\(\pm\)
}
\end{document}

I observe this with TeXLive unicode-math rev46300 2018-01-13 (hmm I god this date from tlmgr info unicode-math as cat-date: 2018-01-13 15:04:43 +0100 and doing this with rev45845 gives 2017-11-18 12:52:03 +0100 ; the date (2018-01-15 23:06) below is presumably the date when I updated my TL2017 and unicode-math with it...)

After

$ tlmgr restore unicode-math
Available backups for unicode-math: 45845 (2018-01-15 23:06)

$ tlmgr restore unicode-math 45845
Do you really want to restore unicode-math to revision 45845 (y/N): y
Restoring unicode-math, 45845 from /usr/local/texlive/2017/tlpkg/backups/unicode-math.r45845.tar.xz

above test file compiles fine with both Unicode engines.

I thought the problem was with luatex because a trace show me a directlua and I figured perhaps it should be \string. therein, but then I checked that also xetex build fails, and I did not investigate more.

edit sorry I got fooled due to many tasks to do by emacs/auctex not having reparsed my file variable, or did some other mistake and erroneously reported a xetex issue. This is only a luatex one, and then my guess is that \string. in the directlua calls will fix it. (not tested) Make it \string<space>. in case catcode is letter... ah no, then the space may be active, ok so \detokenize{.} hopefully dot will not have catcode of closing brace...

... or perhaps we need to understand if really it is directlua and why it would use the current catcode of dot...

Check/indicate

• Relevant for XeTeX
• Relevant for LuaTeX

[eg9]

I only get an error with LuaLaTeX. The console output for XeLaTeX is

> xelatex jfbug
This is XeTeX, Version 3.14159265-2.6-0.99998 (TeX Live 2017) (preloaded format=xelatex)
 restricted \write18 enabled.
entering extended mode
(./jfbug.tex
LaTeX2e <2017-04-15>
Babel <3.16> and hyphenation patterns for 84 language(s) loaded.
(/usr/local/texlive/2017/texmf-dist/tex/latex/base/article.cls
Document Class: article 2014/09/29 v1.4h Standard LaTeX document class
(/usr/local/texlive/2017/texmf-dist/tex/latex/base/size10.clo))
(/usr/local/texlive/2017/texmf-dist/tex/latex/fontspec/fontspec.sty
(/usr/local/texlive/2017/texmf-dist/tex/latex/l3packages/xparse/xparse.sty
(/usr/local/texlive/2017/texmf-dist/tex/latex/l3kernel/expl3.sty
(/usr/local/texlive/2017/texmf-dist/tex/latex/l3kernel/expl3-code.tex)
(/usr/local/texlive/2017/texmf-dist/tex/latex/l3kernel/l3xdvipdfmx.def)))
(/usr/local/texlive/2017/texmf-dist/tex/latex/fontspec/fontspec-xetex.sty
(/usr/local/texlive/2017/texmf-dist/tex/latex/base/fontenc.sty
(/usr/local/texlive/2017/texmf-dist/tex/latex/base/tuenc.def))
(/usr/local/texlive/2017/texmf-dist/tex/latex/fontspec/fontspec.cfg)))
(/usr/local/texlive/2017/texmf-dist/tex/latex/unicode-math/unicode-math.sty
(/usr/local/texlive/2017/texmf-dist/tex/latex/unicode-math/unicode-math-xetex.s
ty
(/usr/local/texlive/2017/texmf-dist/tex/latex/l3packages/l3keys2e/l3keys2e.sty)
 (/usr/local/texlive/2017/texmf-dist/tex/latex/filehook/filehook.sty)
(/usr/local/texlive/2017/texmf-dist/tex/latex/base/fix-cm.sty
(/usr/local/texlive/2017/texmf-dist/tex/latex/base/ts1enc.def))
(/usr/local/texlive/2017/texmf-dist/tex/latex/unicode-math/unicode-math-table.t
ex))) (./jfbug.aux) [1] (./jfbug.aux) )
Output written on jfbug.pdf (1 page).
Transcript written on jfbug.log.

Package: unicode-math 2018/01/13 vv0.8k (tlmgr revision 46300)

[jfbu]

@eg9 thanks, I have edited my post. Somehow, I got confused due to not doing my tests from command line but changing engine via an emacs buffer variable, but I must have done a mistake and thus reported a xetex problem. It is specifically luatex, and I have indicated in my edit how I think this could get fixed.

[wspr]

Thanks for picking this up -- it's as simple as:

\catcode`. \active
\def.{\ERROR}
\directlua{fontspec.mathfontdimen(font.current(),"AxisHeight")}

I think if I change \directlua to \lua_now:n (better to do so, anyway) it will fix the problem.

[wspr]

Hmm, no, that doesn't fix it. I'll continue to look into it.

[jfbu]

\documentclass{article}
\usepackage{unicode-math}
\begin{document}
\catcode`. \active
\def.{\ERROR}
%\directlua{fontspec.mathfontdimen(font.current(),"AxisHeight")}
\directlua{\detokenize{fontspec.mathfontdimen(font.current(),"AxisHeight")}}
\end{document}

compiles without error for me.

It is strange to me that the \directlua call coming from unicode-math would pick up the current catcode of dot character.

[wspr]

Yes, I didn’t have long to look earlier, and when I saw the dot being seen as active even after it had been tokenised as an other I could tell something strange was happening...

[davidcarlisle]

I didn't fully trace but it's just for the first use of math at that size, the catcode triggers because it's as if an fd file had been read with <8.5->"[xits-math.otf]:mode=base;script=math;language=DFLT;"<6-8.5>"[xits-math. when the . was active It's not an external fd file but printed from \directlua so comes to the same thing. I would guess that (somewhere:-) you can print using the option to use a normalised latex catcode table rather than the default of using current catcodes.

…

On 25 January 2018 at 13:16, Will Robertson ***@***.***> wrote: Yes, I didn’t have long to look earlier, and when I saw the dot being seen as active even after it had been tokenised as an other I could tell something strange was happening... — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#443 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABNcArPcrOEKHdsc8CrywepDYKmhH-hLks5tOH6ngaJpZM4RrUlh> .

[jfbu]

\nfss@catcodes is ignored in this process of pseudo fd file?

[wspr]

Ah, this has to be fixed in fontspec, sorry. There's a tex.sprint buried in there which didn't have a catcode regime specified, and it was outputting a length with a dot in it. When I get around to the next release of fontspec, it will be fixed; in the mean time, you can fix locally by applying the changes here:

latex3/fontspec@5372058

(At some point I looked into removing that function but wanted to keep things simple. I'll leave this issue open until that happens, at which point I'll add a test file.

[jfbu]

I confirm that the latex3/fontspec@5372058 patch fixes the failed test lualatex build, when I apply it to TeXLive2017 fontspec 2.6g TL revision number 45732