Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Recommended technique for Math formulas in EPUB #845

Closed
rdeltour opened this issue Sep 1, 2016 · 27 comments
Closed

Recommended technique for Math formulas in EPUB #845

rdeltour opened this issue Sep 1, 2016 · 27 comments
Labels
Cat-Accessibility Grouping label for all accessibility related issues Spec-A11yTechniques The issue affects the EPUB Accessibility Techniques WG Note Status-Deferred The issue has been deferred to another revision

Comments

@rdeltour
Copy link
Member

rdeltour commented Sep 1, 2016 •
edited
Loading

Recent discussions (EPUB a11y call on August 24 + follow-up thread on the mailing list) have revived the longtime question of how best to represent math formulas in EPUB. This issue intends to track the various discussions and solution candidates.

Background

The issue of math formulas in EPUB consists of at least two distinct (but related) challenges:

  1. how to visually render the formula
  2. how to make it usable through assistive technologies

Historically, EPUB has promoted MathML as the preferred way to represent math. However, MathML is poorly supported by numerous Reading Systems. Many of them do not even parse the MathML to render the fallback image defined in MathML's altimg attribute. Many content publishers –even the ones that can produce MathML– end up commenting out the MathML and just use images of formulas instead.

It must also be noted that MathML is considered by some experts as a failing appraoch to represent math on the web (see notably @pkra's thoughts on the topic).

As for accessibility, the situation is mixed. For some very simple formulas, simple alternative text is probably enough. For more complex content, MathML is reasonably well supported by major screen readers (sometimes with 3d party libraries like MathPlayer), despite being limited by the inherent lack of semantics in presentation MathML. Some specialized JS-based renderers (like MathJax's accessibility extensions) may also be relevant alternatives.

All in all, there is no ideal solution. What is needed is one (or several?) best practices that can be recommended to content publishers in the short-to-mid term.

Candidate techniques

The constraints are:

  • the formula must be visually rendered by both legacy and newer Reading Systems
  • legacy reading systems cannot be updated
  • the solution is accessible (even if MathML is not supported / not available)
  • authors must be able to reasonably easily create content from a MathML representation (most publishers do have MathML content).

Option 1 - data-mathml atttribute

The formula is represented with an img element, with alternative text in its alt attribute. Additionally, the equivalent MathML markup is available (escaped) in a data-math attribute that can be used by a javascript renderer.

Example:

  <img alt="my equation" src="my-formula.jpg" data-math="…escaped mathml…"/>

Pros:

  • works on legacy RS
  • requires no specification work

Cons:

  • usage of the data-math attribute is a non-standard used of the HTML spec if it is intended for use by the RS (it's probably OK if the javascript is provided by the Author).

Option 2a - object element with MathML fallback

The formula is represented as an image in the object element, for which an alternative text can be defined with the aria-label attribute. The object can have MathML child content, which could then be used as a fallback by the RS or AT that prefers it.

Example:

<object data="image.jpg" aria-label="my equation description">
 <mathml>
   (...)
 </mathml>
</object>

Pros:

  • works on legacy RS
  • easy to author

Cons:

  • requires some slightly bended interpretation of HTML, which says that the object's fallback content can be shown only "if the user has indicated a preference" (for instance if "that content uses a format that the user finds more accessible").

Option 2b - object element with image fallback

The formula is represented as an external MathML document embedded with an object element, for which an alternative text can be defined with the aria-label attribute. The object can have an image as its child content, which could then be used as a fallback by the RS.

Example:

<object data="mathml-formula.xml" aria-label="my equation description" type="application/mathml+xml">
  <img src="image-formula.jpg" alt="my equation description"/>
</object>

Pros:

  • works on legacy RS and current browsers

Cons:

  • authoring inconvenience

Option 3 - Media Query

A new Media Query "detecting" MathML support. We could then assign display:none' to the math element and a non-none display to an image fallback sibling for legacy reading systems and have that new MQ assign a non-none display value to the math element and display: none to the image fallback sibling if MathML is implemented.

See @therealglazou's proposal.

Example:

<img class="formula-image" src="formula.jpg" alt="formula description"/>
<mathml class="formula-mathml">
   …
</mathml>
.formula-mathml {
  display: none;
}
@media (mathml) {
  .formula-mathml {
    display: inline;
  }
  .formula-image {
    display: none;
  }
}

Pros:

  • conceptually simple
  • similar MQs will likely be used for other feature detection (e.g. javascript)

Cons:

  • requires spec work in CSS
  • slightly more tedious to author

Option 4 - aria-details

The formula is rendered with an image, and the MathML is referenced from an aria-details attribute on the image.

Example:

<img class="formula-image" src="formula.jpg" alt="formula description" 
          aria-details="formula-details" />
<details id="formula-details">
  <summary>Example</summary>
  <mathml>
   …
  </mathml>
</details>

Option 5 - custom classes in package metadata

Special CSS classes for both MathML and non-MathML formulas are declared in Package metadata. Reading systems that support MathML would override the display property to render the MathML instead of the image.

Example:
Content Doc:

<img class="math-image" src="..."/>
<span class="math-mathml">
  <m:math>...</m:math>
</span>

CSS:

.mathml { display: none; }

Package:

<meta property="math:mathml">math-mathml</meta>
<meta property="math:no-mathml">math-image</meta>

A MathML-aware RS would override the CSS based on the values defined in Metadata above:

.math-mathml { display: inline !important; }
.math-image { display: none !important; }

Other references
@mattgarrish mattgarrish added the Spec-A11yTechniques The issue affects the EPUB Accessibility Techniques WG Note label Sep 2, 2016
@mattgarrish mattgarrish added this to the EPUB 3.1 milestone Sep 2, 2016
@mattgarrish mattgarrish mentioned this issue Sep 2, 2016
Closed
@danielweck
Copy link
Member

Additional offscreen design proposal: https://rawgit.com/mhakkinen/extdesc/master/Extended-desc.md.html

@mattgarrish mattgarrish removed this from the EPUB 3.1 milestone Oct 2, 2016
@danielweck
Copy link
Member

MathJax's Peter K. 's thoughts on MathML Media Query:
https://www.peterkrautzberger.org/0190/
@pkra

@danielweck
Copy link
Member

An old issue...but perhaps a good time to mention the more recent DAISY proposals:
https://github.com/daisy/epub-accessibility-tests/blob/master/content/epub30-test-0330/EPUB/Text/nav.xhtml
Test EPUB:
https://github.com/daisy/epub-accessibility-tests/blob/master/build/Advanced-Accessibility-Tests-Mathematics-v1.1.0.epub

@VeryGoodErotica
Copy link

VeryGoodErotica commented Mar 27, 2020 •
edited
Loading

What about this? It's based on 2a and the role="math" could be used as a trigger to let ePub viewers know that if they properly render MathML then do not use the image, use the MathML content. Obviously readers not updated will use the image even if they support MathML but that is not so bad.

<object data="equation.{svg,png}" role="math" aria-label="y equals b INVISIBLE TIMES x to-the-power-of k">
  <math xmlns="http://www.w3.org/1998/Math/MathML" display="inline">
  <mi>y</mi>
  <mo>=</mo>
  <mi>b</mi>
  <mo>&#x2062;<!-- invisible times --></mo>
  <msup>
    <mi>x</mi>
    <mi>k</mi>
  </msup>
  </math>
</object>

@VeryGoodErotica
Copy link

VeryGoodErotica commented Mar 27, 2020 •
edited
Loading

Tested, it works in both Calibre and on my Kobo. ePubCheck complains that role=math is invalid for object so maybe next version of ePub should have a epub:type="math" that is used to let viewers know to render the MathML instead if they support it? with the MathSpeak in a SVG desc node like MathJax does?

@VeryGoodErotica
Copy link

Guess what I'm suggesting is option 2a but add an epub:type="math" attribute in next ePub version that viewers can optionally use to trigger a preference of the MathML if they properly support it.

Screen readers can be triggered by an aria-labelledby in the svg node,

@VeryGoodErotica
Copy link

sorry I haven't finished my coffee.

Option 2a with the epub:type="math" and an aria-label="MathSpeak string" in the object node.

@GeorgeKerscher
Copy link

GeorgeKerscher commented Mar 27, 2020 via email

@laudrain
Copy link

laudrain commented Mar 27, 2020 via email

@GeorgeKerscher
Copy link

GeorgeKerscher commented Mar 27, 2020 via email

@VeryGoodErotica
Copy link

Thank you very much for those working on this.

The solution I came up is currently working in the ePub viewers I personally have access to, but I definitely would prefer a method with proper accessibility testing behind it.

@mattgarrish mattgarrish added the Cat-Accessibility Grouping label for all accessibility related issues label Aug 26, 2020
@mattgarrish
Copy link
Member

Until we have a technique that reasonably works everywhere, I don't think there's anything we can do with this issue. I'm going to put a deferred label on it as I don't see it being resolved in our current charter timeline. We can return to it whenever there's more clarity (assuming mathml support doesn't overtake the need for this).

@mattgarrish mattgarrish added the Status-Deferred The issue has been deferred to another revision label Mar 8, 2021
@OriIdan
Copy link

OriIdan commented Mar 9, 2021 via email

@avneeshsingh
Copy link

We developed Math techniques and have done extensive testing on it. Which will improve further with Chrome's support for MathML.
The EPUB with the examples of best practices for Math is at:
http://epubtest.org/books/Accessibility-Tests-Mathematics-v1.1.1.epub

And explanation of the techniques is at:
http://diagramcenter.org/best-practices-for-authoring-math.html

So, we can consider closing this issue.

@alexshpilkin
Copy link

@avneeshsingh It appears that your report is, to a large extent, focused on accessible maths, which is undeniably important, but also not nearly the whole problem.

And... Do I understand correctly that the proposed solution for reasonably supported maths in EPUB books is... PNG images with no possibility of progressive enhancement? (OK, PNG images with <details>, in some cases.) That is, something the editors of the only decently typeset ebook with non-trivial mathematics that I’ve seen (The Feynman Lectures on Physics) explicitly rejected as unbearably ugly. And the alternative requires MathML support in the reader on pain of making the book impossible to understand.

I can understand that as a faithful representation of the current state of the reader world, but it’s still just so sad.

@iherman
Copy link
Member

iherman commented Mar 31, 2021

I can understand that as a faithful representation of the current state of the reader world, but it’s still just so sad.

It is. But there is some light in the tunnel. W3C is discussing setting up a new MathML Working Group whose main goal is to consolidate a profile of MathML that can, and will be, available in browsers. If that happens, then the PNG fallback become less prominent.

Fingers crossed :-)

@alexshpilkin
Copy link

@iherman wrote:

I can understand that as a faithful representation of the current state of the reader world, but it’s still just so sad.

It is. But there is some light in the tunnel. W3C is discussing setting up a new MathML Working Group whose main goal is to consolidate a profile of MathML that can, and will be, available in browsers. If that happens, then the PNG fallback become less prominent.

I’m not even particularly against a PNG fallback; I mean, having several representations of the same formula isn’t pretty and does add some bloat, but that’s what tooling is for. However, the discussion above leads me to believe that having a reasonably compatible, no-scripting-required MathML to SVG to PNG fallback isn’t possible with current readers, not with <svg:switch>, not with <epub:switch>, not with <picture>, not with <object>, just not possible at all in a single EPUB. If you want compatibility, it’s bare PNGs or bust. (The web is easier in that you can just code your fallback logic in JavaScript and it’ll usually work.)

Preferably such a fallback would allow for yet another alternative representation targeted at screen readers (although, TBH, a significant portion of maths that I care about best exists in a two-dimensional form—commutative diagrams, deduction trees, even similar algebraic expressions with related terms lined up—and I’ve never seen coherent guidelines as to how deal with these in an accessible way). Ideally it would even allow an additional, separate representation for copying (AsciiMath or TeX code, probably). But so far even the non-accessible fallback described above seems impossible; I’ve subscribed to this thread to see myself proven wrong on this count and haven’t succeeded so far. :(

@Artoria2e5 Artoria2e5 mentioned this issue Apr 5, 2021
Open
@mattgarrish
Copy link
Member

Closing this issue, but feel free to continue to discuss. The issue here is generally poor support for MathML and we can't solve that in the techniques at this time.

@jenstroeger
Copy link

Is there any status update on this? There’s a discussion Best Practices for Authoring MathML in EPUB from DAISY but I can’t find a definite guideline or recommendation?

@avneeshsingh
Copy link

@jenstroeger In addition to the best practices document that you mentioned above, DAISY knowledge base also has a section for MathML.
We will be revising the mentioned best practices sometime this year to reflect latest support of browsers and assistive technologies.

@jenstroeger
Copy link

Thank you @avneeshsingh — so what’s the recommendation today to target current and likely dated reading systems?

The equation in my particular case is a simple inlined E=mc2 which I could just note down/mark up as

<span role="math">E = mc<sup>2</sup></span>

to ensure that current reading systems can reliably handle it?

@avneeshsingh
Copy link

avneeshsingh commented May 18, 2024 via email

@jenstroeger
Copy link

It would be interesting to know that more than 50% of the reading systems that we tested support MathML.

Do you know how those “more than 50% tested” correlate to the devices in use out in the wild? What if 80% (just making up a number) of the devices in use make up the other 50%? 🤔

@avneeshsingh
Copy link

avneeshsingh commented May 18, 2024 via email

@GeorgeKerscher
Copy link

GeorgeKerscher commented May 18, 2024 via email

@gregoriopellegrino
Copy link
Contributor

The equation in my particular case is a simple inlined E=mc2 which I could just note down/mark up as

<span role="math">E = mc<sup>2</sup></span>

Hi @jenstroeger, we have discussed this possible approach (that looked promising) in both the DAISY TIES group and W3C. Using role="math" to mark formulas in plain text does not seem like a good solution. The direction we give now, even for inline formulas, is to transform them to MathML.

@GeorgeKerscher
Copy link

GeorgeKerscher commented May 19, 2024 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Cat-Accessibility Grouping label for all accessibility related issues Spec-A11yTechniques The issue affects the EPUB Accessibility Techniques WG Note Status-Deferred The issue has been deferred to another revision
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests