Incorrect widths of math elements in Gecko

[fred-wang]

Follow-up of Peter's report on the dev list, there are issues with width of math elements in Gecko. MML.math has the following block:

if (nMML.widthBug && !mtable.nMMLforceWidth && mtable.nMMLlaMatch))) {
       //
       //  Convert size to ex's so that it scales properly if the print media
       //    has a different font size.
       //
      parent.style.width = (math.firstChild.scrollWidth/nMML.ex).toFixed(3) + "ex";
     }

As I read it, this will not be executed if there is not any mtable. I think we disabled the correction when we added the mlabeldtr workaround in 0891402#L5L621 (issue #356). The condition should probably be:

    nMML.widthBug && !(mtable.nMMLhasLabels && (mtable.nMMLforceWidth || !mtable.nMMLlaMatch))

But then this gives too large widths. It seems to be due to the conversion to ex which was introduced in debc240#L3L618 (issue #386)

[pkra]

Thanks, Fred!

[fred-wang]

Peter's original test case was test/sample-mml but it's somewhat difficult to reproduce. Here are some test cases related to this bug:

<p>Gecko's bug 415413 (for the Zoom we have a workaround):</p>
  <table>
    <tr>
      <td style="border: 1px solid black">
        <math><mi>f</mi><mi>f</mi><mi>f</mi><mi>f</mi><mi>f</mi><mi>f</mi><mi>f</mi><mi>f</mi><mi>f</mi><mi>f</mi></math>
      </td>
    </tr>
  </table>

<p>Gecko's bug 415413 with mlabeledtr</p>
<math>
  <mtable>
    <mlabeledtr>
      <mtd><mtext>LABEL</mtext></mtd>
      <mtd style="border: 1px solid black">
        <mi>f</mi><mi>f</mi><mi>f</mi><mi>f</mi><mi>f</mi><mi>f</mi><mi>f</mi><mi>f</mi><mi>f</mi><mi>f</mi>
      </mtd>
    </mlabeledtr>
  </mtable>
</math>

<p>Gecko's bug 415413 with mlabeledtr</p>
<math>
  <mtable side="left">
    <mlabeledtr>
      <mtd><mtext>LABEL</mtext></mtd>
      <mtd style="border: 1px solid black">
        <mi>f</mi><mi>f</mi><mi>f</mi><mi>f</mi><mi>f</mi><mi>f</mi><mi>f</mi><mi>f</mi><mi>f</mi><mi>f</mi>
      </mtd>
    </mlabeledtr>
  </mtable>
</math>

[fred-wang]

So here is how it looked like in v2.1:

The "ex" changes of issue386 made the widths too large:

The changes for the mlabedledtr workaround removed the width correction in general:

[fred-wang]

So I've committed a patch that restores the width correction of the math in Gecko's native MathML but removes the incorrect fix of issue #386 in that case (I guess the latter is less likely to happen than the former). However, the width may still be wrong for mtable cells so we definitely need to fix bug 415413 upstream.

I suspect that in general, each time we measure some width in native MathML (not only Gecko), this width is likely to change after Web fonts are loaded. So in particular we'd rather implement mlabeledtr upstream and remove the workaround introduced in #356 in the future (or we'll have to do ugly measure verifications on each math, which will be a performance loss for the native MathML). Having #301 to ensure correct stretching of operators is more important to me and anyway people will always be able to use Web fonts themselves.

[dpvc]

I am not able to reproduce the problem that you are seeing. With the v2.1, the v2.2, and the develop branches, I get correct results for your test file.

Your image looks like it is using the MathJax fonts, is that correct? Are they locally installed, or web-based?

[dpvc]

Sorry, I forgot to switch to MathML output (D'oh!)

[fred-wang]

They were local fonts. The result may well be system-dependent. For example I didn't see Peter's original bug without trying with a larger zoom. That may depend on default font preference of Firefox (font-family and font-size). Probably that worked on your system and so didn't see the issue when fixing the bug.

[fred-wang]

Sorry, I forgot to switch to MathML output (D'oh!)

OK :-)

[dpvc]

Here's what I've found out about the problem. It used to be that the <math> element itself had the wrong width, which is why we added the <mrow>, and that even with the <mrow> the math element didn't get the right width. The latter seems to have been fixed (at least with the test cases I've tried). So it seems that <math> now has the correct width. The problem, however, is that that width does not propagate up to the enclosing element properly (at least when that element is "shrink-wrap" element like a <td> or <mtd> or an element with display:inline-block).

The following code shows the issue:

<!DOCTYPE html>
<html>
<head>
<title>Firefox MathML size errors</title>
</head>
<body>

<p>
<span style="display:inline-block; border: 1px solid red">
  <math id="math1">
    <mrow style="border: 1px solid black">
      <mi>f</mi><mi>f</mi><mi>f</mi><mi>f</mi><mi>f</mi><mi>f</mi><mi>f</mi><mi>f</mi><mi>f</mi><mi>f</mi>
    </mrow>
  </math>
</span>
</p>

<script>
math = document.getElementById("math1");
span = math.parentNode;
mbb = math.getBoundingClientRect();
sbb = span.getBoundingClientRect();
document.write("math scrollWidth = "+math.scrollWidth+"<br>");
document.write("math clientWidth = "+math.clientWidth+" (wrong)<br>");
document.write("math bbox width = "+(mbb.right-mbb.left)+"<br>");
document.write("span scrollWidth = "+span.scrollWidth+"<br>");
document.write("span clientWidth = "+span.clientWidth+" (wrong)<br>");
document.write("span offsetWidth = "+span.offsetWidth+" (wrong)<br>");
document.write("span bbox width = "+(sbb.right-sbb.left)+" (wrong)<br>");
</script>

<p>
<math>
  <mtable>
    <mtr>
      <mtd style="border: 1px solid red">
        <mrow style="border: 1px solid black" id="math2">
          <mi>f</mi><mi>f</mi><mi>f</mi><mi>f</mi><mi>f</mi><mi>f</mi><mi>f</mi><mi>f</mi><mi>f</mi><mi>f</mi>
        </mrow>
      </mtd>
    </mtr>
  </mtable>
</math>
</p>

<script>
math = document.getElementById("math2");
mtd = math.parentNode;
mbb = math.getBoundingClientRect();
sbb = mtd.getBoundingClientRect();
document.write("math scrollWidth = "+math.scrollWidth+"<br>");
document.write("math clientWidth = "+math.clientWidth+" (wrong)<br>");
document.write("math bbox width = "+(mbb.right-mbb.left)+"<br>");
document.write("mtd scrollWidth = "+mtd.scrollWidth+"<br>");
document.write("mtd clientWidth = "+mtd.clientWidth+" (wrong)<br>");
document.write("mtd bbox width = "+(sbb.right-sbb.left)+" (wrong)<br>");
</script>

</body>
</html>

The output for me is the following:

Note that the true width of the math is 76 pixels, while the width of the containing element (<span> or <mtd>) is only 53. The container has a red border and the math has a black one. The second example shows that the same is true internally within the math, in this case the <mtd> also fails to get the width of its content correct.
This has nothing to do with MathJax, as this example doesn't load MathJax.

MathJax tries to compensate for the first error by transferring the scrollWidth of the top-level <mrow> to the <span> that contains it. MathJax doesn't currently try to adjust the width in the second case (the <mtd> one), though it does add an explicit <mrow> to overcome other problems. (Fred points this out above where he says the upstream bug needs to be fixed.)

Fred correctly points out that this correction was lost in the change for the <mlabeledtr> support. I suspect he is also right that the ex versus px change is probably the reason for the wrong amount of space being reserved for the expression. I want to look into this further, as it suggests that the determination of the ex value is wrong. This could also account for your seeing font sizes that are too big.

[dpvc]

I suspect that in general, each time we measure some width in native MathML (not only Gecko), this width is likely to change after Web fonts are loaded.

The only widths that are being measured (currently) is the size of the <mrow> inside the <math> element. And yes, you are right that if web fonts are used, that means the size could change after we have set it. That is one of my concerns about using web fonts for MathML. Since Gecko doesn't propagate the sizes of the math element properly, we have to correct the size of the container (since it is a span with display:inline-block, one of the cases that Gecko gets wrong), otherwise all the math will have the wrong amount of space, not just math that uses web fonts.

we'd rather implement mlabeledtr upstream and remove the workaround introduced in #356 in the future (or we'll have to do ugly measure verifications on each math, which will be a performance loss for the native MathML).

I'm not sure I understand the linkage between the <mlabeledtr> workaround and the measurements for the Gecko bug with wrong widths. There is no special measuring in the mlabeledtr code, so that patch is independent of setting the widths of the container spans, as far as I can tell. Yes, having <mlabeledtr> work in Gecko would be great, and I'm all for that, but I don't see how it affects the decision about setting the size of the <span> that contains the resulting <math> element. That has to be done in either case.

The only solution I can think if is the one I suggested in #301 and that you suggest above: looping through the math periodically to see if the width has changed and updating it. It is ugly, but it is not that inefficient (it is merely checking if the top-level <mrow>'s width is no longer the same size as the containing <span>'s width. Running through the math elements to check this should be reasonably fast, and if one does all the checking before making any changes, you should not get the multiple reflow problem in the situation where something has changed.

Certinaly fixing the measurement errors in Gecko is a better solution, but we have to work with the Firefox browser as it currently stands (and will still have to support some older versions even once it is fixed, at least for a while).

[fred-wang]

There are two distinct Gecko bugs here:

• clientWidth, clientHeight, offsetLeft & offsetTop values are incorrect. Boris Zbarsky says they are not supposed to be used on MathML see https://bugzilla.mozilla.org/show_bug.cgi?id=491668#c12.
• Intrinsic widths of some token elements are incorrectly computed (bug 415413). This affects the layout of elements like table cells.

[fred-wang]

since it is a span with display:inline-block, one of the cases that Gecko gets wrong),

Why do we have to put each math element in an inline-block span?

I'm not sure I understand the linkage between the workaround and the measurements for the Gecko bug with wrong widths. There is no special measuring in the mlabeledtr code, so that patch is independent of setting the widths of the container spans, as far as I can tell. Yes, having work in Gecko would be great, and I'm all for that, but I don't see how it affects the decision about setting the size of the that contains the resulting element.

I think I remember that in mlabeledtr we take the width of the parent table at some point but didn't check in details. Anyway, my point is that we should ideally not measure anything in the NativeMML code since that's a performance loss and interfere with the user's CSS. So things like #564 must be fixed now. Of course if there are bugs, we have no other choice than to workaround them and to temporarily measure sizes.

Another bug in Gecko affecting height:

• https://bugzilla.mozilla.org/show_bug.cgi?id=219873

[dpvc]

Note that we don't use any of clientWidth, clientHeight, offsetLeft, or offsetTop from MathML elements in MathJax. Never have.

I certainly don't know anything about how Gecko works internally, but it doesn't to me that this problem is about the widths of token elements. Those all seem to have the correct widths. Even the math element has the correct width. It is that that width doesn't seem to be recognize by the parent element. Unless what you are saying is that the parent element asks the math element for its width, and the math element asks the mrow, and the mrow asks the token elements, and they report something wrong, and the mrow adds up the wrong values and reports it back to the math element, and it reports back the wrong sum to the parent. In that case, I don't see how the scrollWidth and the bounding rect size can be right. But, as I said, I don't know anything about the internals of Gecko; it just seems weird.

[dpvc]

I think I remember that in mlabeledtr we take the width of the parent table at some point but didn't check in details.

There is a check to see if there is an explicit width that is a percentage, as the work around is different in that case, but no actual measuring of elements is performed.

[fred-wang]

Note that we don't use any of clientWidth, clientHeight, offsetLeft, or offsetTop from MathML elements in MathJax. Never have.

I just copied the bug report name, but the comment holds for all the offset_, scroll_ and client* metrics I think. I remember that in the Zoom code we use one on the math.

I certainly don't know anything about how Gecko works internally, but it doesn't to me that this problem is about the widths of token elements. Those all seem to have the correct widths.

The intrinsic widths of these tokens are incorrect, not the final ones that are returned by Javascript. That means it will affect the width of elements like table cells that require these intrinsic widths to decide their layout. So finally, the final widths of the cells get wrong, but that's because of the incorrect intrinsic widths of tokens.

[dpvc]

Another bug in Gecko affecting height:

Personally, I think the current behavior is correct. It's like a span with a large image in it, it's height is still that of a standard line. But that may be a matter of viewpoint.

[fred-wang]

There is a check to see if there is an explicit width that is a percentage, as the work around is different in that case, but no actual measuring of elements is performed.

OK, I didn't check in details.

[fred-wang]

Personally, I think the current behavior is correct. It's like a span with a large image in it, it's height is still that of a standard line. But that may be a matter of viewpoint.

OK, that was just to point it out. But I don't think it affect MathJax.

[dpvc]

I just copied the bug report name, but the comment holds for all the offset_, scroll_ and client* metrics I think.

Comment 14 on that bug report indicates scrollWidth is defined for all elements (and this seems to be supported by the CSSOM View Model documentation. According to that, the clientWidth should be defined and 0 (my mistake saying 0 was wrong).

[fred-wang]

@dpvc you can take this bug if you wish. At the moment, the best I can see is 110b149, given that the print bug in native MathML is less likely to show up.

[dpvc]

Yes, I'll look into it and see if I can figure out what is wrong. The px width will certainly work (other than the print problem when the media scaling is different from the screen scaling), but I'd like to figure out why the ex width is off.

[dpvc]

I found the problem with the widths when given in ex's: the span being styled was inside the one that had the font-size style use for scaling the math, and that extra scaling wasn't taken into account. I have fixed that, and the sizes used for the containers now match the size of the math for me. See if that is the case for you as well.

I've also added a work-around for the problem with the <mtd> widths as well, so the table cells should now get the proper width also (so my second example in the test file above should work properly in MathJax).

I still have to add the option to disable automatic scaling, and to do the loop that checks if web fonts have changed the sizes of the math. I'll post again when I have that taken care of.

[fred-wang]

I'm not sure the loop that checks if the surrounding font size has changed is necessary for MathML, since I expect the incorrect width to be fixed in future versions of Gecko and the workaround to be removed in the next version when we deprecate old browsers. However, this will still be necessary for the HTML-CSS / SVG output, if that's what you mean.

[dpvc]

My issue558-noscale branch adds a matchFontHeight option to the HTML-CS and NativeMML output jax to control whether font height matching will be performed. (Note that it doesn't make sense to do this with SVG, since we aren't using system fonts, so the size matching comes automatically.)

[dpvc]

OK, these last two commits (to the issue558-loop branch of my fork of MathJax) include support for handling pages that include web fonts in the text around the math, or in CSS controlling the native MathML rendering. This is done through a loop that checks if the ex size of the surrounding font or the MathML font has changed, and if not whether the width of the math has changed (presumably due to the loading of a web font). If a change is detected, the width reserved for the math is reset, and the font-siz recalculated (if necessary).

There is a global parameter matchWebFonts that controls whether the output jax perform these tests. It is off by default (since it performs a loop after MathJax has typeset anything). There are also fontCheckDelay and fontCheckTimeout (for each output jax) that controls the initial delay between checks (this gets longer for each loop), and the total time elapsed when the loop should be terminated.

One pass through the loop is pretty fast (60 equations used about 15 to 20 milliseconds for me), and it doesn't seem to interfere with scrolling and other user interaction.

For the HTML-CSS output, if a change is detected, the output is re-rendered. This is because the output is sensitive to font-size changes, particularly at small font sizes, and also because dimensions using absolute units (like inches or pixels) need to be recomputed when the surrounding font's em-size changes. This may need more performance data to see if that is reasonable or not.

The SVG output doesn't need such a loop since it scales automatically (its size is given in units of ex, so a change to the surrounding font changes it automatically). Technically, we should re-render it on a font change as well, since the absolute units would need to be adjusted. But it probably isn't worth it, so I didn't add the loop. If you think it should be added for completeness, I can do that. Perhaps being able to disable the loop in individual output jax would be useful as well.

[dpvc]

I'm not sure the loop that checks if the surrounding font size has changed is necessary for MathML, since I expect the incorrect width to be fixed in future versions of Gecko and the workaround to be removed in the next version

The loop actually serves two purposes: detecting web font's within the MathML itself (and fixing the width values when that occurs), and detecting detecting font changes either within or without the MathML so that the math font can be scaled to match the surrounding text.

While the first of these may be fixed in the next Firefox release, I suspect the second will not be resolved for some time, so if the MathML uses web fonts (as you have planned to do), or if the page does, then I still think there is a need for this feature.

Of course, the page author can disable it (it is opt-in rather than opt-out anyway at the moment). I also added matchFontHeight option to NativeMML and HTML-CSS output to allow you to opt out of the font matching font-size setting.

[dpvc]

Here are the changes for the issue558-loop branch.

[dpvc]

Why do we have to put each math element in an inline-block span?

It is used for various event handlers for things like zoom click or hover events, and for the "discoverability" interface (which is not on by default, but is still there). For example, the border of this span can be made to "fade in" when the mouse is over the math, and a menu icon can appear at the upper right corner to give access to the contextual menu. This was designed to make the mobile interface easier, but the decisions about how to proceed with it where grounded when Robert went into the hospital.

[fred-wang]

While the first of these may be fixed in the next Firefox release, I suspect the second will not be resolved for some time

I hope the first to be fixed in Gecko Nightly soon. As for the second, this could just be modifying the CSS stylesheet not to change the font-family on the element, so that the font for the math matches the font of the surrounding text by default. However, a mechanism to automatically rescale the font size of the math will probably never be implemented. If a user decides to set the font-family or font-size on the math, then the native rendering engine must use that. While it could be considered a "feature" for a Javascript library as MathJax, it is just a CSS bug for a native rendering engine.

It is used for various event handlers for things like zoom click or hover events, and for the "discoverability" interface (which is not on by default, but is still there).

This is a very bad idea if linebreaking is implemented correctly in native MathML in the future and we'll have to reconsider this choice if we don't want to have yet another CSS compatibility issue. Consider for example

<p>Blah Blah Blah Blah Blah Blah
  <span style="border: 1px dashed red;"><math><mi>x</mi><mo>+</mo><mi>x</mi><mo>+</mo><mi>x</mi><mo>+</mo><mi>x</mi><mo>+</mo><mi>x</mi><mo>+</mo><mi>x</mi><mo>+</mo><mi>x</mi><mo>+</mo><mi>x</mi><mo>+</mo><mi>x</mi><mo>+</mo><mi>x</mi><mo>+</mo><mi>x</mi><mo>+</mo><mi>x</mi><mo>+</mo><mi>x</mi><mo>+</mo><mi>x</mi><mo>+</mo><mi>x</mi><mo>+</mo><mi>x</mi><mo>+</mo><mi>x</mi><mo>+</mo><mi>x</mi><mo>+</mo><mi>x</mi><mo>+</mo><mi>x</mi><mo>+</mo><mi>x</mi><mo>+</mo><mi>x</mi><mo>+</mo><mi>x</mi><mo>+</mo><mi>x</mi><mo>+</mo><mi>x</mi></math></span>
  Blah Blah Blah Blah Blah Blah <span style="display: inline-block; border: 1px dashed red;"><math><mi>x</mi><mo>+</mo><mi>x</mi><mo>+</mo><mi>x</mi><mo>+</mo><mi>x</mi><mo>+</mo><mi>x</mi><mo>+</mo><mi>x</mi><mo>+</mo><mi>x</mi><mo>+</mo><mi>x</mi><mo>+</mo><mi>x</mi><mo>+</mo><mi>x</mi><mo>+</mo><mi>x</mi><mo>+</mo><mi>x</mi><mo>+</mo><mi>x</mi><mo>+</mo><mi>x</mi><mo>+</mo><mi>x</mi><mo>+</mo><mi>x</mi><mo>+</mo><mi>x</mi><mo>+</mo><mi>x</mi><mo>+</mo><mi>x</mi><mo>+</mo><mi>x</mi><mo>+</mo><mi>x</mi><mo>+</mo><mi>x</mi><mo>+</mo><mi>x</mi><mo>+</mo><mi>x</mi><mo>+</mo><mi>x</mi></math></span>
  Blah Blah Blah Blah Blah Blah
</p>

or equivalently, with pure HTML:

<p>Blah Blah Blah Blah Blah Blah
  <span style="border: 1px dashed red;">x + x + x + x + x + x + x + x + x + x + x + x + x + x + x + x + x + x + x + x + x + x + x + x + x</span>
  Blah Blah Blah Blah Blah Blah <span style="display: inline-block; border: 1px dashed red;">x + x + x + x + x + x + x + x + x + x + x + x + x + x + x + x + x + x + x + x + x + x + x + x + x</span>
  Blah Blah Blah Blah Blah Blah
</p>

and here how they render in Gecko:

while an inline frame splits into several lines and behaves well with respect to the surrounding text, an inline-block frame does not split and the linebreaking of the MathML happens inside that inline-block itself, without interaction with the surrounding text. To say it simple, it's just like if we were putting all the equations inside a HTML table with a single cell, preventing any interaction with the surrounding text...

[pkra]

Great to hear that the FF nightly might fix this. How much does the NativeMML output differentiate between browser versions?

Weighing accessibility (zoom, discoverability) against line breaking is a tough call. Perhaps we can solve some of these problems by using page-wide tools, e.g. zoom libraries.

[dpvc]

It is probably worth making this a separate issue for future consideration. Changing the display:inline-block span would require changes in a variety of other locations in the code, so is non-trivial.

[dpvc]

Perhaps since the web-font-testing loop is off by default, that code should be made into an extension rather than be included in the core? (If so, however, that would also mean that the future use of web fonts for NativeMML output would also have to be an extension, since it would rely on this one.)

[fred-wang]

• there is a "console.log(element); " debug logging
• yes, I think we should make the web-font-testing loop an extension. I still believe the ideal goal will be to make MathJax entirely compatible with the browser layout engine (at least for the MathML mode). So hopefully the scale & width hacks for native MathML won't be necessary anymore in the future and in that case we will be able to add Web fonts without a testing loop.
• I think for consistency we should do the web-font-testing loop for SVG too. Having a single option for all the output modes makes sense to me: if people use Web fonts for the text on their page they are likely to use them whatever the output mode. However, it's easy to change that later if necessary (for example people might want to use Web fonts for the native MathML only).

[dpvc]

Sorry about the debug message. I thought I had caught them all.

OK, I'll turn the loops into an extension.

About SVG, remember that the font height matching is automatic in that case, so the only advantage to doing this will be to adjust the size of dimensions given in absolute units within the math. Since those are seldom used, and it requires re-rendering the math to do it, I'm not sure it is worth it. But I can certainly add it.

[fred-wang]

About SVG, remember that the font height matching is automatic in that case, so the only advantage to doing this will be to adjust the size of dimensions given in absolute units within the math. Since those are seldom used, and it requires re-rendering the math to do it, I'm not sure it is worth it. But I can certainly add it.

Yes, I got that. To be honest, I don't really have a preference but that seemed weird to not have the option for SVG. In that case, perhaps use one config option per rendering mode so that people can e.g. enable it for HTML but not SVG. And people who really need it for SVG could enable it too.

[dpvc]

OK, sounds good.

[dpvc]

I've moved the code to an extension, and made it possible to configure the renderers separately. This is the issue558-loop branch.

=> Ready for Review

[fred-wang]

I have not checked in details, but I'm OK with the changes in general so let's merge them to start the testing.

=> Ready For Release

[dpvc]

=> Merged

[fred-wang]

There seem to be a couple of failures do to spacing changes. I wonder if this is caused by #558