# Luaotfload ignores OpenType attributes in mathmode #204

Closed
opened this issue Mar 13, 2014 · 27 comments

### Hugovdberg commented Mar 13, 2014

 I want to use Calluna as the base font for my documents, and eulervm in mathmode. However, the euler numerals look really ugly with Calluna, while Calluna numerals look rather beautiful with euler. To get this to work I use fontspec to load the fonts, and a bit of magic by egreg@stackexchange to change the numerals to Calluna's Lining,Monospaced numerals, see also http://tex.stackexchange.com/q/165238/46598 This bit of code works perfectly using XeTeX, but LuaLaTeX forgets the +lnum and +tnum attributes as soon as it enters mathmode and uses the default numerals. For most fonts this is fine since they have Lining figures by default, although these are usually proportional, so not ideal but not too wrong. However, Calluna (and EB Garamond, which is more readily available) have oldstyle numerals by default which is not preferable in mathmode. As you can see using the code below the attributes are set just fine by lualatex+fontspec, and using xetex the attributes are respected as well in mathmode, but lualatex will produce the wrong numerals. \documentclass{scrartcl} \usepackage{amsfonts} \usepackage{eulervm} \usepackage{siunitx} \usepackage[no-math]{fontspec} \setmainfont[ Ligatures={Common,TeX}, Numbers={OldStyle,Proportional}, ]{EBGaramond12-Regular.otf} \newfontfamily{\liningmain}[ Ligatures={Common,TeX}, Numbers={Lining,Monospaced}, RawFeature=+tnum, ]{EBGaramond12-Regular.otf} \setsansfont[ Ligatures={Common,TeX}, Numbers={Lining,Proportional}, ]{TeX Gyre Heros} \setmonofont[ Ligatures={NoRequired,NoCommon,NoContextual}, Numbers={Lining,Monospaced}, ]{TeX Gyre Cursor} % A trick for extracting the family information % which works independently of the chosen font \begingroup \def\getfamily#1#2#3#4#5{#4} \edef\x{\endgroup \def\noexpand\liningdefault{\expandafter\expandafter\expandafter \getfamily\csname liningmain \endcsname}}\x \DeclareSymbolFont{liningmath}{\encodingdefault}{\liningdefault}{m}{n} \DeclareSymbolFontAlphabet{\mathlining}{liningmath} \Umathcode0="7 \symliningmath 0 \Umathcode1="7 \symliningmath 1 \Umathcode2="7 \symliningmath 2 \Umathcode3="7 \symliningmath 3 \Umathcode4="7 \symliningmath 4 \Umathcode5="7 \symliningmath 5 \Umathcode6="7 \symliningmath 6 \Umathcode7="7 \symliningmath 7 \Umathcode8="7 \symliningmath 8 \Umathcode9="7 \symliningmath 9 \sisetup{ math-rm = \mathlining, } \usepackage{polyglossia} \setmainlanguage[variant=british]{english} \begin{document} \section{testing} \fontname\font -- 0123456789 {\liningmain\fontname\font -- 0123456789} \begin{tabular}{ll} \hline Mode & Result\\ \hline Text mode & 1234567890\\ Math mode & $$1234567890$$\\ \verb|\SI| text mode & \SI{1042358769}{\metre}\\ \verb|\SI| math mode & $$\SI{1234567890}{\metre}$$\\ \verb|\num| text mode & \num{1234567890}\\ \verb|\num| math mode & $$\num{1234567890}$$\\ verbatim & \verb|1234567890|\\ \hline \end{tabular} \end{document} A more concise document, as provided by egreg as well, to see how luatex ignores the font specification is given below, using the font specifications as output in the above code: \input ifxetex.sty \ifxetex \font\test="[EBGaramond12-Regular.otf]/ICU:script=latn;language=DFLT;+liga;mapping=tex-text;+lnum;+tnum;" \else \input luaotfload.sty \font\test="[EBGaramond12-Regular.otf]:mode=node;script=latn;language=DFLT;+liga;+tlig;+trep;+lnum;+tnum;" \fi \test 1234567890 \textfont0=\test $1234567890$ \bye
Member

### phi-gamma commented Mar 14, 2014

 \font\test="[EBGaramond12-Regular.otf]:mode=node;script=latn;language=DFLT;+liga;+tlig;+trep;+lnum;+tnum;" ... \textfont0=\test $1234567890$  Afaik node mode conflicts with TeX math mode. From the manual:  The downside is that the intricate operations required for \identifier{node} mode may slow down typesetting especially with complex fonts and it does not work in math mode.  If you change the definition to file:EBGaramond12-Regular.otf:mode=base;+lnum;+tnum; then you get lining figures in the second example. I’ll have more thorough look at it later.
Member

### phi-gamma commented Mar 14, 2014

 As I thought, the same behavior with Context: \definefontfeature [testfeatures-node] [lnum=yes,tnum=yes,mode=node] \definefontfeature [testfeatures-base] [lnum=yes,tnum=yes,mode=base] \definefont [testfontnode] [file:EBGaramond12-Regular.otf*testfeatures-node] \definefont [testfontbase] [file:EBGaramond12-Regular.otf*testfeatures-base] \starttext {\testfontnode 1234567890\endash\textfont0=\font \m {1234567890}}\par {\testfontbase 1234567890\endash\textfont0=\font \m {1234567890}}\par \stoptext  I’d stick with base mode in math mode since that’s documented behavior of the fontloader. If you want it changed you’re going to have to take the issue upstream.
Author

### Hugovdberg commented Mar 18, 2014

 Sorry for my late reply, but setting the Renderer to Basic for that math font does work only partially, like about half the time, the font specification string consistently says mode=base, but then it rather randomly applies OpenType features, sometimes lining, sometimes oldstyle, sometimes monospaced, sometimes proportional. I'm not sure where this would stem from, it seems to me the minimal plain TeX code with manual font selection works ok all the time, so why it doesn't in a normal document I don't know. Do you have any idea where this could best be asked?
Member

### phi-gamma commented Mar 19, 2014

 but then it rather randomly applies OpenType features, That sounds weird. It can happen in cases where the selected features are ambiguous like for example in this issue reported recently: http://tug.org/pipermail/luatex/2014-March/004784.html But without a minimal example containing only an absolute minimum of Latex code I’m not really in a position to debug anything.
Author

### Hugovdberg commented Mar 19, 2014

 Hereby a minimal working example, I formatted it into a table for easy comparison, and added the font specification strings to check whether they are the same all the time. That string itself is typeset with the liningmain font to show that it's not just math mode that messes it up, but it's really in the font loading: \documentclass{article} \usepackage{amsfonts} \usepackage[no-math]{fontspec} \setmainfont[ Ligatures={Common,TeX}, Numbers={OldStyle,Proportional}, ]{EBGaramond12-Regular.otf} \newfontfamily{\liningmain}[ Ligatures={Common,TeX}, Numbers={Lining,Monospaced}, Renderer=Basic, ]{EBGaramond12-Regular.otf} \begingroup \def\getfamily#1#2#3#4#5{#4} \edef\x{\endgroup \def\noexpand\liningdefault{\expandafter\expandafter\expandafter \getfamily\csname liningmain \endcsname}}\x \DeclareSymbolFont{liningmath}{\encodingdefault}{\liningdefault}{m}{n} \DeclareSymbolFontAlphabet{\mathlining}{liningmath} \Umathcode0="7 \symliningmath 0 \Umathcode1="7 \symliningmath 1 \Umathcode2="7 \symliningmath 2 \Umathcode3="7 \symliningmath 3 \Umathcode4="7 \symliningmath 4 \Umathcode5="7 \symliningmath 5 \Umathcode6="7 \symliningmath 6 \Umathcode7="7 \symliningmath 7 \Umathcode8="7 \symliningmath 8 \Umathcode9="7 \symliningmath 9 \begin{document} \begin{tabular}{p{.15\textwidth}p{.85\textwidth}} \hline Mode & Result\\ \hline Normal font & \fontname\font\\ Lining font & {\liningmain\fontname\font}\\ Text mode & 1234567890\\ Math mode & $$1234567890$$\\ \hline \end{tabular} \end{document}
Author

### Hugovdberg commented Mar 21, 2014

 I just realised this could be minimized even further to show that the problem is not in the declaration of the symbolfont etc. \documentclass{scrartcl} \usepackage{amsfonts} \usepackage[no-math]{fontspec} \setmainfont[ Ligatures={Common,TeX}, Numbers={OldStyle,Proportional}, ]{EBGaramond12-Regular.otf} \newfontfamily{\liningmain}[ Ligatures={Common,TeX}, Numbers={Lining,Monospaced}, Renderer=Basic, ]{EBGaramond12-Regular.otf} \begin{document} Normal font:\\ \fontname\font Lining font:\\ {\liningmain\fontname\font} \end{document}
Member

### phi-gamma commented Mar 24, 2014

 As I explained in the thread I cited earlier, the apparent “randomness” in applying features is due to Lua hashes making no guarantees about the order of keys when iterated over. Font features are passed on to the fontloader as a hash table, hence the varying order. Btw. I can only reproduce this with the 2.4 (CTAN) branch. However Hans modified the ordering of features without announcing it, thus the development Luaotfload chooses features consistently. Node mode is currently broken though and you’d need a patch to make it run. Otherwise you’ll have to wait for v2.5.
Author

### Hugovdberg commented Mar 24, 2014

 How should the order in which the options are iterated over influence whether or not lining figures are selected? As far as I can see there are no conflicting options in my code? Is there an ETA for the 2.5 version? I guess I will just keep compiling until it by accident selects the correct features. Op 24 mrt. 2014 21:18 schreef "Philipp Gesang" notifications@github.com: As I explained http://tug.org/pipermail/luatex/2014-March/004785.html in the thread I cited earlier, the apparent "randomness" in applying features is due to Lua hashes making no guarantees about the order of keys when iterated over. Font features are passed on to the fontloader as a hash table, hence the varying order. Btw. I can only reproduce this with the 2.4 (CTAN) branch. However Hans modified the ordering of features without announcing it, thus the development Luaotfload chooses features consistently. Node mode is currently brokenhttp://www.ntg.nl/pipermail/ntg-context/2014/077485.html though and you'd need a patch to make it run. Otherwise you'll have to wait for v2.5. Reply to this email directly or view it on GitHubhttps://github.com//issues/204#issuecomment-38495251 .
Member

### phi-gamma commented Mar 24, 2014

 How should the order in which the options are iterated over influence whether or not lining figures are selected? The previous implementation had a race condition between features: only one would be applied, and which one depended on the order in which they were found in the table. I guess I will just keep compiling until it by accident selects the correct features. You should be able to avoid the effect by choosing only one of the conflicting features, say lnum with onum. Is there an ETA for the 2.5 version? Well, it’s called the texlive2014, so there is kind of a deadline. Though I’m not sure all planned features will be implemented by then.
Author

### Hugovdberg commented Mar 24, 2014

 I understand a race condition can occur if you select both lnum and onum as that would say oldstyle lining figures, but in my case the mainfont is oldstyle proportional (no race condition here) and the new liningmain font is lining monospaced (also no race condition as far as I can see). So I think the effect shouldn't occur at all. What race condition do you see in my code? Op 24 mrt. 2014 22:09 schreef "Philipp Gesang" notifications@github.com: How should the order in which the options are iterated over influence whether or not lining figures are selected? The previous implementation had a race condition between features: only one would be applied, and which one depended on the order in which they were found in the table. I guess I will just keep compiling until it by accident selects the correct features. You should be able to avoid the effect by choosing only one of the conflicting features, say lnum with onum. Is there an ETA for the 2.5 version? Well, it's called the texlive2014https://github.com/lualatex/luaotfload/branches , so there is kind of a deadline. Though I'm not sure all planned features will be implemented by then. Reply to this email directly or view it on GitHubhttps://github.com//issues/204#issuecomment-38501461 .
Member

### phi-gamma commented Mar 24, 2014

 What race condition do you see in my code? I didn’t say it was in your code. It was in the fontloader. If you run this snippet with version 2.4: \ifdefined \directlua \ifdefined \luatexsuppresslongerror \input luaotfload.sty \font \tlnum = "file:EBGaramond12-Regular.otf:mode=base;+lnum;+tnum at 10pt \fi {\tlnum bar 1234567890, \fontname \font} \par \bye  you randomly get either the lnum or the tnum feature applied which is due to how font features used to be implemented. As I said, with the current fontloader the result is stable.
Member

Author

Author

### Hugovdberg commented Mar 30, 2014

 Thanks a lot, it took some time to get it all to work. Apparently I didn't have docutils installed, so creating the manpage failed, but for some strange reason it did create the zip file. That Makefile could use a rewrite, because it doesn't really check for depencies the correct way, but I guess that's of really low priority. I think the new version is great in the sense that it at least consistently selects the wrong font features, but now at least I think I finally know what's really the core of the problem. What happens at the moment is that when you specify Numbers={Lining,Monospaced}, this is translated to +lnum;+tnum;, and when a glyph is requested (I'll use 8 in this example) luaotfload has to find out which glyph is the correct 8 to apply. In pseudocode what I think happens is: features = [lnum,tnum] // Or maybe even 'tnum' as each next specification overwrites previously selected number features? requested_glyph = 8 glyph_name = lookup_default_glyph_name(requested_glyph) // returns 'eight' glyph_name = lookup_glyph_in_table(glyph_name, features[0]) write_character(glyph_name)  However, it is perfectly possible that the first lookup in a table does not return the correct final glyph, and actually is what goes wrong in my case. What luaotfload actually should do is iterate over each substitution table and use the returned glyph_name of the previous pass to get to the final glyph_name (I tested this method on paper, by following the substitution tables of both fonts and ended up at exactly the right glyph every time). In pseudo code: features = [lnum,tnum] requested_glyph = 8 glyph_name = lookup_default_glyph_name(requested_glyph) // returns 'eight' foreach feature in features glyph_name = lookup_glyph_in_table(glyph_name, features[0]) next write_character(glyph_name)  This results in the following glyph names for EBGaramond12-Regular.otf: Features = [lnum,tnum] =================================== input & table & output ----------------------------------- eight & lnum & eight.lnum eight.lnum & tnum & eight.tablining =================================== Features = [tnum,lnum] ============================================ input & table & output -------------------------------------------- eight & tnum & eight.taboldstyle eight.taboldstyle & lnum & eight.tablining ============================================ Features = [onum,pnum,lnum] =================================== input & table & output ----------------------------------- eight & onum & eight eight & pnum & eight eight & lnum & eight.lnum ===================================  I could add many more of these sequences, but as you can see a single lookup is insufficient to determine the final glyph. Furthermore, as long as no conflicting options are set (as in the last example) the order in which the features are processed doesn't matter at all (that's where my initial confusion over a race condition stemmed from) as both orders of lnum+tnum result in the same glyph. However, since people might use \addfontfeature it's probably best to really process options in the order in which they are set anyway. I really hope this actually is the problem, and it seems to me this should be rather straightforward to implement, because at the moment it is impossible for me to select {Lining,Monospaced} as that always takes two subsequent lookups to get the correct glyph.

### dohyunkim commented Apr 1, 2014

 Just a comment. The order of the applying opentype features should be regardless of how users specified it, as the information about the order of lookup tables is contained in the opentype font. It turned out that EBgaramond12-Regular.otf has the following order of lookup tables: lnum onum tnum pnum  Currently texlive2014 branch of luaotfload seems to work as expected under node mode, respecting this order. But under base mode, it does not seem to apply those opentype features.
Member

### phi-gamma commented Apr 1, 2014

 ········· Just a comment. The order of the applying opentype features should be regardless of how users specified it, as the information about the order of lookup tables is contained in the opentype font. It turned out that EBgaramond12-Regular.otf has the following order of lookup tables: lnum onum tnum pnum  Currently texlive2014 branch of luaotfload seems to work as expected under node mode, respecting this order. But under base mode, it does not seem to apply those opentype features. That’s what I suspected, thanks for testing. In the end it might be limitation of base mode. I’m going to run some tests and then forward it to Hans (you can do that too if you wish). Best, Philipp
Member

### phi-gamma commented Apr 5, 2014

 For reference: http://www.ntg.nl/pipermail/ntg-context/2014/077631.html Fyi I’ve classified this as a feature request for the fontloader.

### adunning commented Oct 21, 2015

 As it hasn't been mentioned here, it may be worth pointing out that this issue has been deemed responsible for the rather serious problem of OpenType options being ignored in URLs with \urlstyle{same}; see http://tex.stackexchange.com/questions/264206/cant-get-oldstyle-figures-in-url-with-lualatex.
referenced this issue Oct 31, 2015

### jfbu commented Apr 30, 2016 • edited

 With some fonts one does not get old style figures in math mode (TeX Gyre Termes), but with others it is the other way around, one does not get lining figures in math mode (Vollkorn). tested with LuaTeX 0.95 + luaotfload 2.7-fix-1 from TeXLive 2016 pretest April 29, 2016. Here is with TeX Gyre Termes which comes with TeXLive. \input luaotfload.sty \font\test= {name:TeX-Gyre-Termes:+onum} \test 0123456789 $\fam0 \textfont0=\test 0123456789$ \nopagenumbers \bye Result: But it does work with context: \starttext \font\test= {name:TeX-Gyre-Termes:+onum} \test 0123456789 $\fam0 \textfont0=\test 0123456789$ \stoptext gives (Copied over from closed issue #347) Edit: the explanation (as pointed out in Philipp's comment next and a post by Hans Hagen on context list) is that in the context case, the low-level font declaration resulted in use of base mode. And contrarily to node mode, base mode may partially apply font features in math mode too. But with \starttext \font\test= {name:TeX-Gyre-Termes:+onum;+tnum;} \test 0123456789 $\fam0 \textfont0=\test 0123456789$ \stoptext one can observe that the figures will be Proportional, not Tabular. If we add mode=node; in the font specifier, we do get OldStyle Proportional figures in text; but in math mode we are back to the Lining Tabular figures which are the default for TeX Gyre Termes.
Member

### phi-gamma commented Apr 30, 2016

 ········· With some fonts one does not get old style figures in math mode (TeX Gyre Termes), but with others it is the other way around, one does not get lining figures in math mode (Vollkorn). tested with LuaTeX 0.95 + luaotfload 2.7-fix-1 from TeXLive 2016 pretest April 29, 2016. Here is with TeX Gyre Termes which comes with TeXLive. \input luaotfload.sty \font\test= {name:TeX-Gyre-Termes:+onum} \test 0123456789 $\fam0 \textfont0=\test 0123456789$ \nopagenumbers \bye Result: But it does work with context: \starttext \font\test= {name:TeX-Gyre-Termes:+onum} \test 0123456789 $\fam0 \textfont0=\test 0123456789$ \stoptext These examples are not equivalent. Luaotfload enables node mode and a number of features by default. Also, you don’t define fonts that way Context.

### jfbu commented Apr 30, 2016

 I don't know the correct way in context. If the correct way in context has the same issue as with Plain LuaTeX+luaotfload, then at least that will indicate where to look. If it does not have the same issue, then I guess it is also indication.

### yannis1962 commented Apr 30, 2016

 And here is my minimal example (using luatex, beamer, mathastext and the beautiful Fedra font): \pdfoutput1 \documentclass[9pt]{beamer} \usepackage[no-math]{fontspec} \usepackage{luatextra} \usetheme{AnnArbor} \usecolortheme{crane} \usefonttheme{professionalfonts} \setsansfont[ Path = /hom/yannis/texmf/cours/stats/, Extension = .otf, Ligatures = TeX, BoldFont = FedraSerifProABold, ItalicFont = FedraSerifProABookItalic, BoldItalicFont = FedraSerifProABoldItalic, Numbers = Lining ]{FedraSerifProABook} \usepackage[italic]{mathastext} \begin{document} \begin{frame}[fragile] \frametitle{An example of unwanted oldstyle figures in Fedra math} \Huge 0123456789 $0123456789$ $$0123456789$$ \end{frame} \end{document}  And the result is: As you can see, in inline text mode figures are lining, and in display mode they are old style. This makes no sense.

### jfbu commented May 3, 2016 • edited

 Using mode=base does not fix all issues. For example TeXGyreTermes:mode=base;script=latn;language=DFLT;+tlig;+tnum;+onum;  does not have tabular figures, but only old style figures. With mode=node it has both. And if the above is either with +tnum; or with +onum; it works. Edit: (brief summary for people starting at bottom of thread) The above observation in the present comment does not bring any new information to what has already been well documented in 2014 in this thread. back in 2014 when this thread started there was some unpredictability in how base mode applied features. This was fixed already in 2014, thus currently there is no more unpredictability but the fact that base mode seems to not apply multiple features has not evolved since 2014. Typically with base mode it may not work to try to get OldStyle+Tabular for example, or Lining+Proportional but this depends on the defaults in the font. And base mode produces in this regard same output be it in text mode or in math mode. No evolution since 2014. node mode does not work at all in math mode (in terms of applying opentype features). No evolution either on this side. from what I understand, this behaviour of base mode and node mode are upstream issues, unrelated to the font loader. (but this is confusing to me as the font loader itself appears to be an upstream component, after font forge libraries have been dropped). Unfortunately I don't know a github like interface allowing people like myself not being actively engaged in these topics and condemned to a very partial view to raise such issues to the developer attention. to recapitulate for math: node mode is completely inoperant viz opentype features and base mode only partially operant.

### u-fischer commented Aug 14, 2019

 I'm closing this issue. As mentioned in https://mailman.ntg.nl/pipermail/ntg-context/2014/077635.html the proper way to handle this "is to define a font where certain rendering is replaced by another but in the same slot". This (probably) can be done e.g. with fonts.handlers.otf.addfeature. The *num opentype features are for text, not for math.

### adunning commented Aug 19, 2019

 For issues such as the URL problem I mentioned earlier, then, does this need to be resolved elsewhere?

### u-fischer commented Aug 19, 2019

 @adunning imho yes. url's are text. Imho in luatex it should be possible to add break points without resorting to math. Be aware that this repo is no longer active. luaotfload development is now at https://github.com/latex3/luaotfload.