equation images

[PaulTopping]

MathML support in browsers is spotty with not much hope of improvement in the short term. MathJax, an open source JS lib that can render MathML, does a good job but has performance and reliability issues. Given this situation, it would be nice to use responsive images for equations.

It would be nice to allow a server to produce responsive equation images at request-time. This would allow the server to avoid storing a lot of images for each equation. Instead, it would generate the image directly from the client properties and the MathML.

If the server also had access to the text column width, ambient point size, font, and text color, it could use this information to generate the best image. MathML allows a renderer to do line-breaking. A long equation in a narrow text column could be broken in many lines, resulting in a tall, narrow equation image. The server could also return the speech text for the equation for accessibility purposes. This would allow it to customize the speech based on the particular user's disabilities and preferences.

Since many notations are text-based (chemistry has several such notations), such a mechanism would be useful for more than just mathematics.

[marcoscaceres]

Hi @PaulTopping. I agree that it could be useful, as it it's not foreseen that the MathML situation will get better any time soon :(

Any of the currently proposed solutions could be used to address your use cases. For the accessibility cases, those would indeed be great. Adding the ability to download audio equivalents would be quite helpful, I imagine.

Is there anything in the current document that should be changed to address the things your mention?

[PaulTopping]

Hi @marcoscaceres. Are you suggesting that the current proposal would make the ambient properties (column width, font name/size/color of surrounding text, etc.) available to the server? If so, I am surprised. Could you point me to the place in the proposal where that is addressed?

[marcoscaceres]

No, not so much at that level of detail. But client-hints (link below) could certainly help in providing some of the info you need (viewport width, pixel density, etc. and it's specifically designed to talk to CDNs). Also, as the server serves the fonts, sizes, etc. in the CSS, it has the context to make the decisions, no?

https://github.com/igrigorik/http-client-hints/blob/master/README.md

[PaulTopping]

Thanks for pointing me at this document. Unfortunately, this doesn't help as the hints would need to reflect different data for each image. If I understand the CH mechanism, the hints are only to allow the user agent to communicate its capabilities or user options to the server. Of course, if CH could have replaceable parameters to do this task then we'd really have something! For example,

CH-SITE: %column-width% %font-name% %font-size%

This would cause the browser to replace these parameters in each request with ambient properties of the image site. Perhaps this would be better done with URL query parameters. Unfortunately, there is no way to get the browser to place the ambient info in the parameters.

[PaulTopping]

You mention that the server knows the CSS it delivers so that it has the context to return the correct images. While this is theoretically true, it is not practical. It forces the server to simulate a user agent's processing of all CSS. Some might even come from another server. You also have to factor in the user agents' defaults for fonts, sizes. How can the server know what font, size, etc. the user agent assigns to H1, H2, etc? This actually depends on what fonts are installed on the user's machine.

[marcoscaceres]

Those are valid points - thanks for thinking that through and the clarifications.

However, I'm still not sure what action you would like us to take on this bug? We see sites that currently mix images and text (through the art direction use case). It's not ideal, obviously (for the reasons you mentioned). I'm also worried that the use case you are bringing up is very particular to a community (it's pretty rare to run into math equations in everyday browsing - but that depends on what community one is in, obviously).

So, if you have concrete suggestions for changes to the use cases document, I'm all ears :)

[PaulTopping]

The community that cares about math may be much larger than you think. Every grade school student is expected to learn math. At least half of most colleges' departments must use math in their educational content. Virtually all technology, engineering, medical companies have equations in their content. There is a lot of math already on the web but there would probably be more if browsers handled it better.

I am going to have to read the spec more carefully before giving any concrete suggestions. I was hoping that you people would have some ideas. And you did, of course, which is appreciated.

Obviously some of the things I've been talking about can probably be done with JavaScript but that means the equations will not be able to take advantage of browsers' pre-fetching optimizations. I think that is my main motivation in this "bug".

Even with JS code, there is still the difficulty in getting this ambient font information at the image site. As I mentioned, it is hard to simulate all of what goes on in CSS, etc. in the page in order to figure this out. Obviously, the browser has this information. It would be nice if there was a way to access those values from JS code. I will admit that there's a lot going on in the HTML5 world that I need to catch up on. Perhaps this has been addressed in some proposed API that I am just not aware of.

[marcoscaceres]

So, the core of the problem is still that we need better MathML support in browsers (which is outside the scope of our group). To use the responsive image solutions to solve that problem would only be a temporary stopgap.

However, with regards to another potential solution (again, just off the top of my head here) - if the HTML and the SVG shared the CSS file, then you could maybe get what you need?

[PaulTopping]

I have been campaigning for better MathML support in browsers. There has been a lot of talk about this lately: http://news.cnet.com/8301-1023_3-57610854-93/google-subtracts-mathml-from-chrome-and-anger-multiplies/. Agreed that it is outside the scope of this group.

What we've been talking about is a stopgap measure somewhat but there will always be text-based notations for which there is no markup language support in standards or no implementation in browsers. I do believe that the kind of thing I'm looking for is useful way beyond math.

We could generate SVG for the math and I believe that it can reference CSS parameters, though I am less sure the latter is sufficient. It is definitely something we are looking at.

[nwtn]

I think the SVG approach is certainly your best bet at a stopgap.

While I am a huge believer in needing better MathML support, I know how hard we've struggled in this group to get the existing use cases taken seriously by browser vendors, and I believe the existing use cases are (or at least will be perceived to be) far more common than the one you're raising here. I fear that any push to include equations as a responsive image use case will be dismissed at the browser level, at least at this stage.

The responsive image proposals have been an exercise in restraint and taking baby steps. The priority now is getting something implemented that takes care of the big three use cases (DPR switching, viewport switching, and art direction); everything else will need to be treated as a later addition/enhancement to whatever solution happens. Unfortunately, I think the equations fall into this camp.

I'm going to close this issue for now. I do think it's an interesting and valid use case, though, and something we should maybe revisit in the future. (Or, hopefully, we won't have to if MathML support improves.)