Using fonts from MathJax's CDN for Native MathML

[fred-wang]

Firefox 15 is going to be release today and could now use MathJax fonts for the MathML operators.

See https://groups.google.com/forum/?fromgroups=#!topic/mathjax-dev/9EyiDt_0ess

=> assigned to myself.

[fred-wang]

I haven't test the fix yet.

(this time I combined and repacked MathJax, although I don't know if that's convenient for merging branch)

[fred-wang]

There were extra single quotes in the CSS rules, that I have fixed in my latest commit. I tested this with Firefox Beta 15 and the Web fonts are correctly used:

http://devel.mathjax.org/testing/mathjax/tests/MozillaTortureTest/texvsmml.html

However, MathJax does not seem to scale correctly the math with respect to the surrounding text. I guess this is because it does not have access to the font metrics until the fonts are downloaded...

[dpvc]

Yes, I think you have touched on the issue. The way MathJax tries to match the size is to use a box of a particular height and width in ex's and measure the size of that box. I guess that the web fonts aren't loaded by that action, and even if they were, they might not have arrived yet, so MathJax gets the wrong size. I suppose I could have MathJax insert a dummy <math> element that would force the font to load, and wait a bit before starting to measure things, but MathJax would not know how long to wait. (That is why MathJax's font detection algorithm is so crucial to its own use of the fonts.)

[fred-wang]

Do you think that scaling the math with respect to the surrounding text is the only place where MathJax needs the fonts to be available? I'm wondering if it's really worth addressing this issue now (it is less serious than the bad rendering when users do not have the fonts installed).

Another solution would be to insert the math elements to let Firefox display them, wait a certain delay and render the math elements again. Note sure that would be better, though.

[dpvc]

I think you are probably right, it is not something to worry about at the moment, so I'm marking it ADDRESS LATER so we remember to come back to it for the release after this one.

[fred-wang]

I meant, we can take the fix at it is now (allowing the user to get a decent rendering without fonts installed) even if the scale is not applied correctly.

[dpvc]

Sorry, I'm working on too many things at once, and was getting confused about the plain FF usage of the fonts versus the MathJax NativeMML usage. Yes, we should be able to go ahead with the changes. It would also be possible, I suppose, to use a tester like the HTML-CSS output does to find out if the fonts are ready. The problem is, MathJax doesn't know what fonts the native MathML renderer will actually be using, so that probably doesn't help.

I also noticed that the width of the math can grow when the fonts arrive, so the space reserved for the math may be too small in the end. That is potentially a worse situation. I suppose MathJax could go back afterward and adjust the sizes after it is finished.

[fred-wang]

The problem is, MathJax doesn't know what fonts the native MathML renderer will actually be using, so that probably doesn't help.

I think we should add options similar to the HTML-CSS / SVG outputs to force the choice of a font. We can add font-family rules to change the font of the text and the other -moz-* rules for the stretchy symbols.

I also noticed that the width of the math can grow when the fonts arrive, so the space reserved for the math may be too small in the end. That is potentially a worse situation. I suppose MathJax could go back afterward and adjust the sizes after it is finished.

Yes, I also noticed that too, although I was not sure it was the same problem or not. IIRC this was to fix a bug when the math is zoom.

Anyway, I think your are right that we should handle this for after the next release.

[dpvc]

There is a font menu (hidden by default) that controls the HTML-CSS font usage. We could probably do a similar one for NativeMML output.

[fred-wang]

I was thinking to the configuration options, but probably a font menu would be a good idea too.

[dpvc]

OK, good point. Yes, font options in the configuration for NativeMML would be appropriate.

[fred-wang]

I've updated my branch to make the NativeMML font configurable. For the moment, I didn't do font detection or take the menu choice into account.

It seems that the font detection and verification is not so simple. Would it be best to move it to a separate file that would be shared between the HTML-CSS & NativeMML output? I guess just force a reflow after some arbitrary time to avoid the problem mentioned above, but that won't help to detect which local fonts is installed.

[fred-wang]

I wanted to put the FONTTEST variable in a specific extension that would be loaded by the HTML-CSS & NativeMML outputs but then I realized that it uses the HTMLCSS variable and specifically the HTMLCSS.FONTDATA. Is there a simple and reliable way to detect when the Web fonts arrive, that could be put in the NativeMML jax instead? It's a shame that browsers don't have an API to do that...

[dpvc]

The font testing is one of the most fragile components of MathJax, so "simple and reliable" is not really what I would call it. If you are wanting to deal only with the MathJax fonts, then the code could probably be simplified, and the section of HTMLCSS.FONTDATA that is needed (not much as I recall) could be included. But doing the detection in general is quite hard.

It might be worth looking at the google webfont code to see if they have developed anything better.

[fred-wang]

I've updated my branch again to recompute the scale and width after three seconds. Some things I'm not sure:

• I put the scale determination in one loop rather than three loops. I'm not sure if that changes the "this only requires one reflow". After three seconds, I do an additional loop at the end to modify the style on the each span. Anyway, the NativeMML rendering is very fast, so I can't see the difference.
• I don't know the best way to access the span to which we attach a style="width: xxx" in order to workaround the widthBug. I've tried something that seems to work.

Other remarks:

• We still need to update the horizontal stretchy braces. However, if the STIX/Asana fonts are installed, they will be used for these operators.
• I'm wondering if it's relevant to add a preview, given the speed of the NativeMML.
• rather than detecting the Web fonts, perhaps we can just verify whether this.defaultEx and this.defaultMEx have changed and update the rendering if it happens (e.g. one verification per second, during FONTTEST.timeout seconds).

[fred-wang]

https://github.com/fred-wang/MathJax/compare/master...issue301

[fred-wang]

I've updated the detection of changes to the default sizes as mentioned in my previous comment. I also added items for NativeMML in the font preference menu. In general, I tried to have configuration options consistent with the HTML-CSS ones, although of course the behavior may be slightly different depending on font support, native MathML support etc

=> Ready for review

[dpvc]

I haven't had the chance to look carefully at the changes, but you are right that changing the three loops to one destroys the "one reflow", and you now have a reflow for each math expression on the page. That was the reason for the three loops originally.

I will look more closely at the rest of it when I can.

[fred-wang]

IIUC, the reflow happens when the browser has to measure the offsetWidth. So that's not a problem if I keep a separate determineScalingFactor function, just that I need to keep the three loops.

[dpvc]

Right. It is not the separate function but the combining of the insertion of elements, measuring, removal all in one loop. The reason for three loops is to do all the insertion first, then all the measuring, then all the removal. That means the measuring only causes one reflow.

[fred-wang]

So as discussed with Peter on IRC, the plan would be to check all the issues "Ready for review" and merge them, so that we can run the tests today and tomorrow.

The issue301 branch allows the users to choose a font via the menu but I think this is not fundamental (the most important feature here is supporting Web fonts in native MathML here). Hence I think the modification to the menu can be removed before the merge, so that it won't be a problem with the localization.

The Open Type font branch will certainly be merged after the beta and also modifies the menu. Hence that will be a good time to modify the menu for this issue301 branch too, and update the localization strings.

dpvc: can you review the changes to

unpacked/jax/output/NativeMML/config.js
unpacked/jax/output/NativeMML/jax.js

and merge them if they look ok?

[fred-wang]

I'm removing the "Ready For Review" flag for now. I'll update the branch against the latest version (especially with localization change) and add woff versions of Asana/STIX for native MathML as specified in

https://github.com/mathjax/MathJax/wiki/STIX-and-Asana

[fred-wang]

47b11b6

OK, I've created a new branch issue301-bis. This adds support for Asana, STIX and MathJax Web fonts for the native MathML (in browsers). This also fixes the issue with the horizontal braces in MathJax fonts, since Gecko can pick the glyphs needed from other fonts (while still using MathJax glyphs for the other constructions). This also works in Webkit, although it is not possible to decide which font to use for largeop/stretchy operators (probably a ::-math-stretchy CSS rule should be defined). Anyway, Webkit largeop/stretchy support is currently very limited.

=> Ready For Review

[fred-wang]

https://github.com/fred-wang/MathJax/compare/mathjax:develop...issue301-bis

I'm updating the title since it's not specific to Firefox

[dpvc]

I've been looking at this branch for a couple of days, and I think there are some issues to be addressed.

• The new font menu has a couple of problems:
  • The menu seems to be broken into two parts, one for HTML-CSS and one for MathML, but the variable used is the same for both, so you are only getting one value. This should really be two separate variables, so that setting the MathML font won't affect the HTML-CSS font.
  • The auto option should be in each of the two, and for NativeMML, I think it should mean not setting any styles.
  • Once we get the open-type-fonts finalized, the font menu will need to include SVG fonts as well. I'm wondering if we shouldn't make submenus by output jax (e.g, Settings->Fonts->MathML->STIX)
• Should the font menu be made available by default? Otherwise, it is hard for users to make the selection. This menu was really only intended for use in debugging. One problem with it is that selecting a font reloads the page, and that might need to be changed before it can be made public.
• The About MathJax data may need to be modified to display the selected MathML font, either in addition to the HTML-CSS font, or as well as it. There may need to be some reorganization of the information there.
• A small item is that the font names in undefinedFamily (and other font-family lists) that have more than one word should be enclosed in single quotes. E.g., MathJax_Math, 'Asana math', STIXGeneral. Some of the older browsers have trouble without these, but I guess it probably isn't a problem any longer. Better safe than sorry, though.
• I note that you have used the original (large) font files for this. Are we wanting to use the web fonts from open-type-fonts for this rather than the original fonts?
• You have defined styles for MJX-tex-oldtyle (and the other similar ones) only for when the MathJax TeX fonts are selected. Should these be defined for all fonts (to use the MathJax_Caligraphic font)? I don't think you can use the STIX or Asana fonts for these directly, since the characters either aren't available or aren't in the positions of the ASCII alphabet and numbers.
• The @font-face definitions may interfere with the HTML-CSS ones (for the same font-family names) if you switch between renderers. I suspect that because of the built-in CSS rules for FF, you need to use these names here. I suppose they could be changed in the HTML-CSS output.
• Will listing all the fonts here cause all the fonts to be downloaded, or are they downloaded only if they get used? I know that IE downloads everything that is mentioned, but I don't know what happens with other browsers.
• MathJax_Caigraphic is a typeo; it should be MathJax_Caligraphic (L176 - L179 of NativeMML/jax.js).
• You only reference STIXNonUnicode, STIXSizeOneSym, STIXGeneral, MathJax_Main, and MathJax_Caligraphic in your CSS rules; so why the @font-face declarations for all the others? It seems these should not be needed (unless these fonts are used in the FF CSS rules somewhere). I would be surprised if the STIXIntegralsUp and other such variants were used, however.
• You don't need to include MathJax_WinChrome or MathJax_WinIE6, which certainly will not be needed.
• The check for scaling factor doesn't really test whether the web fonts have arrived. This is because the elements being measured do not depend on those fonts in any way. I really don't see how this test is going to be made to work since you don't know which fonts are being used by the browser, and so can't really test for their appearance. In addition, if more than one is requested, how do you know which one you have responded to?

I suppose it would be possible to run through all the math elements to see if their widths have changed in order to update the width that was set on the containing span, and it probably wouldn't take all that long to do it, but it does seem rather drastic. I can't think of another reliable way to do it, and even then, it is not clear when you should stop looking. One approach is to use a short initial time (so you respond quickly to fonts arriving), but then make the delay longer and lager as you try again, until it gets long enough to give up trying. (That is how the timeouts work for HTML-CSS polling for fonts, for example).

In any case, I'm a rather uncomfortable with the current approach that really doesn't give any reasonable guarantee of detecting the width changes.

I know that is a lot of items, but I think there are some important issues that need to be resolved before this can be merged.

[pkra]

Just to chime in.

• I don't think the font menu should be exposed by default for now -- page authors should choose the font. But we probably should document it for author debugging. If user feedback indicates or we see some accessibility needs (e.g., dyslexic fonts) then we can revisit this.
  • Alternatively, we could add a way to default to the MathJax fonts. Ideally, users could specify their preference in the browser but we can probably not read them (like the "bug" with FF setttings set to block all webfonts).
• I'm not sure the MathJax about popup is critical. I can't quite see why someone would configure different fonts for MathML/HTML/SVG outputs anyway. Let's deal with that at a later point.
• I think we should use the new webfonts -- those are part of MathJax and that should be the baseline.
• The conflict between MathML and HTML worries me. What are our options?
• Could either of you provide more details on the detection problem?

[dpvc]

I'm OK with not exposing the font menu, and having authors specify the fonts (as the do now for HTML-CSS output). I'm not sure what you mean by defaulting to the MathJax fonts, though.

The pop-up tells what font is selected for HTML-CSS, so I figured if there is a choice for MathML, it should tell that, too (so we can get information about it when users report problems). The reason there is a difference between HTML-CSS and NativeMML is that the HTML-CSS fonts distinguish between local and network versions (there are four choices: local STIX fonts, local TEX fonts, web TeX fonts, and image fonts), while NativeMML doesn't. That is, the choices for the two output modes aren't the same. So you can't use the same value for both, since the choices for one don't correspond to the choices for the other.

In terms of the using new web fonts, I'm not sure if that will work or not (which is why I was asking Fred about it). It may be that the current browser CSS rules preclude that (because of the naming or the math tables or something like that).

As for the conflict between NativeMML and HTML-CSS, the way the @font-face directives work is that they set up a font-family and link them to local and/or network fonts. The problem is that both NativeMML and HTML-CSS may set up @font-face directives for the same font-family (e.g., MathJax_Main). That could potentially lead to problems if the two aren't the same. But the font-family can really be anything you want, so it doesn't have to be the same as the actual name of the font in the font file. So one or the other of the two output jax could use a different font-family, like MathJax_Main-MML or MathJax_Main-HTMLCSS. (This is just another in a series of potential name conflicts between the components of MathJax; we have naming conventions for things like CSS classes and id's, and for properties added to the element jax object classes, and so on, but perhaps we need one for @font-face as well.) The only downside to using such a font-family is that if someone cuts and pastes some math as formatted text, the font name will be MathJax_Main-MML or MathJax_Main-HTMLCSS rather than MathJax_Main, and so even if they have the MathJax fonts installed, it won't show up properly in the pasted text. But they really can't copy and paste math very effectively that way now, so there is not much loss. In addition, the HTML-CSS output jax already uses postfixes for IE and Opera, which have problems handling styles and weights in their @font-face directives.

Finally, about font detection, the NativeMML output jax sets the size of its output SPAN tag explicitly, at least for Firefox, because Firefox doesn't always get the size of the <math> element correct (or didn't in the past). MathJax uses the size of an <mrow> that it adds just inside the <math> tag to get the size, because it turns out that that width is computed correctly. But if the fonts change after MathJax has done that (when web fonts arrive), then the size that it has set will no longer be correct, and the space reserved for the math will be off (leaving the math overlapping the text that follows, or leaving too much space after it). So Fred's code does a loop to check if web fonts have arrived, and recalculates the widths if they do.

The problem is that his code doesn't actually detect the incoming fonts (it is not measuring anything that depends on them), so the sizes won't be fixed. And even if it did, since more than one font might be needed, MathJax might need to detect several incoming fonts, and recalculate as each arrives. But MathJax doesn't know what fonts are actually being used by the MathML (unlike when HTML-CSS is used, where the font used for each character is painstakingly determined), so can't tell what fonts to wait for, or which equations might need resizing. So the whole process is a bit fragile.

My only suggestion would be to loop through all the math and update the sizes based on the sizes of the inner <mrow> if it is different, and to do that periodically until some time has passed and you don't think any more web fonts will be showing up. Not great, but the loop shouldn't take very long. My recommendation was to start out checking with fairly short intervals, but make the wait progressively longer (this is how the HTML-CSS checking works, and several other timing loops in MathJax). This makes it pretty responsive for fast-loading fonts, but less intrusive if you have to wait longer for the fonts.

Hope that clears up the issues.

[fred-wang]

• General remarks on how fonts are handled:

  1. the ideal case:
     1a) Follow the usal Web fonts behavior in browsers:
     • one can specify @font-face rules with "local" and Web fonts. The browser will try the local fonts first, do the rendering immediately and reflow if necessary once Web fonts arrive.
     • one can specify a list of fonts. The browser will use the first font that has the requested character.

    1b) Handling of mathematical fonts:
    - the local and Web versions are the same.
    - only a single font for a math font family (with possible Italic, Bold and BoldItalic styles).
    - read the Open Type Math table to provide a generic support
  

  2. how it is currently handled by the Native MathML:

  This is currently as in 1) except that they don't read the Open Type Math table and have private tables. WebKit only supports a few delimiters for stretchy constructions that can be done with Unicode characters only. Gecko has a generic Unicode support for stretchy operators too and otherwise uses tables for each supported font (indexed by font name): the old STIX set, the MathJax fonts and the Asana font. I'd like this to be rewritten more cleanly to remove the tables and implement a generic support based on the Open Type Math table instead. That will considerably simplify the handling of fonts, make it more consistent and allow to support much more fonts.

  The plan of the STIX consortium to drop the old font set and to merge STIX-Main and STIX-Math sounds a good idea for that purpose. All the other families already only use a single math font. We don't mind about the font size because the whole rendering process is much faster than MathJax's one and can be done immediately with local fonts before the Web fonts are available. On the other hand, having a single font to handle makes things much more convenient to handle (for internal processing or user's CSS).

  3. how it is currently handled by the MathJax HTML-CSS:

  1a) MathJax uses distinct rules for local and Web fonts and can not mix several families yet. I'm wondering if keeping the distinction local/Web is really relevant, I think everybody would like to use the local versions before trying the remote ones (what people do want is to avoid using e.g. the local STIX fonts and force the use of TeX Web fonts). Also it would be much better if MathJax could do some cleaver selection algorithm like the one used by browsers with font-family. So I'm guessing in the future we would like to remove all these availableFonts, preferredFont, webFont, imageFont config options and only have a single list that specify the list of fonts to try with the preference indicated by the order. That would make consistent the handling of the NativeMML/SVG/HTML-CSS output and simplify the configuration for the users. The menu would then just control the preferred font i.e. the first element of the list.

  1b) Because of Javascript limitations (can not access some glyphs, can not read open type table, do not know metrics, slow layout...) we must create our own fonts and private tables for font metrics and stretchy op. Because we don't have a generic support for local fonts, we must wait that the Web fonts arrive before starting the layout and so it's best to use several small fonts in that case. This is in contradiction with the "ideal" handling of mathematical fonts.

=> One consequence of this is that the NativeMML can not (and must not) use our custom Web fonts. Also, for Gecko the Web font name should currently be exactly the same as the local font. Because Gecko uses the MathJax/STIX family that have several small fonts, we have to specify all these rules in the CSS which is a bit tedious. So I've written a script to generate these rules for my MathML-fonts add-on that just uses the list of files even if they are not all used in Gecko or WebKit ; and I've just used a macro to convert these rules to the MathJax format. This will be much more simpler when only a single font is necessary as it's the case for Asana. I would have guessed that browsers are clever enough not to download the uncessary fonts but I admit that I didn't checked.

• Regarding the font menu and about MathJax. I think in the future we would like the menu to be enabled by default. One MediaWiki dev has recently submitted a patch (for their local install) to enable that menu. My initial goal was to make the menu and config options consistent between the three output modes, so that the users would have core config options to define the fonts for all the output and similarly only one menu to select the font. Unfortunately by the previous overview, consistency is not likely to happen soon so perhaps an intermediate solution would be to keep per-output configs and to have font names enabled/disabled when the output mode is changed.
• Regarding the font detection. First as indicated by Boris Barzky (https://bugzilla.mozilla.org/show_bug.cgi?id=491668#c12) trying to retrieve metric properties that are specific to HTML on the MathML element is not correct and gives wrong values. So the span is necessary or we should use getBoundingClientRect instead. Anyway, I think Davide convinced me that this automatic resizing should just be removed for the MathML output. I've never really be satisfied with the result, it's too random and it has always given too large sizes for me. I think we should let the browsers handle the sizes themselves (as we do for MathPlayer). Gecko may currently give smaller size because the default stylesheet changes the font-family on the . However users can rely on standard stylesheet if they want to increase that size or use a font-family for the surrounding text that has a size similar to the one for the math (especially when more math fonts are supported, they can use the font-family that has been specially designed to be used with the math font). Having MathJax computes that size may just disturb the rendering e.g. if the font for the surrounding text is itself a Web font.

[fred-wang]

I've created a simplified branch based on the one of issue 564 (that removes automatic resizing of text). Now I'm only using @font-face CSS rules to load MathJax Web fonts (and Asana to draw horizontal braces). I've removed all the config and menu features. So the only remaining issue is to rename the @font-face rule of the HTML-CSS output. What should I modify exactly for that?

[fred-wang]

Differences with issue #564 :
https://github.com/fred-wang/MathJax/compare/issue564...issue301-ter

[fred-wang]

At the moment, I'm leaning toward giving up this for MathJax 2.3. Hopefully in the release that comes after, the Gecko width bug and the lack of good support for stretchy operators in WebKit will be fixed ; so we can do that properly.

[pkra]

Your call, Fred. If this needs more work on the browser side, I'm sure we can figure that out.

[pkra]

This discussion is out of date given the developments in Gecko (Opentype Math tables etc). Not sure if we should open a new issue to reflect the changed requirements.

[pkra]

Marking this abandoned.