ARIA interaction with MathML

[NSoiffer]

[Links updated 17/4/22]

With JAWS and NVDA, the current situation is that aria-label, etc., are ignored in MathML. I think the same is true in Orca. I don't know if it is true using VoiceOver. Making use of aria-label in math is problematic because there are multiple math braille codes and there is no way to know which is the appropriate one to use if the author/authoring tool is clever enough to be able to generate an appropriate aria-braillelabel (in English speaking countries, there is Nemeth and UEB technical). Transcribing aria-label to braille is definitively wrong in Nemeth and UEB, the two braille codes I know. It is almost certainly wrong for the other codes because AFAIK, they are all meant to describe the characters in the math, not how they are spoken.

The MathML WG is working on providing a means to add author intent to a document. The idea is to allow authors to disambiguate the meaning of some math that has multiple interpretations and hence be spoken differently. There are still details the WG is hashing out, but the current intent syntax along with some examples is found here. In brief, the proposal is to add an intent attribute that takes as a value a simple functional syntax. For example, (0, 1) might represent a point which would have intent="point($a, $b)" where $xxx refers to descendant element with the attribute arg='xxx'. The same notation could also represent an open interval and would be encoded intent=open-interval($a, $b). Like aria-label, this attribute would override the speech for the math[1]. Unlike aria-label, this has no effect on the braille[2]. Where aria-label and intent most directly overlap is when intent is added to a leaf element such as mo as in

<mrow>
  <mi>n</mi>
  <mo arg="op" intent="factorial">!</mo>
</mrow>

One can easily imagine using aria-label instead of intent in the above example except for the problem with braille. Note: the suggested approach for intent would include the use of intent on mrow also.

The MathML WG wants to work with the ARIA WG to come up a plan on how ARIA should interact with MathML. This could be something like "ARIA is ignored inside of MathML except on non-MathML elements inside of the math element". Or it could be a modification to the accessible name computation. Or it could be something else.

The MathML WG meets at the same time as the ARIA group, so if we want to have a joint meeting to discuss this, is should be relatively easy to get people together.

For TPAC in 2021, the WG prepared a MathML Accessibility Gap Analysis which might be useful background information for those not familiar with the challenges of math accessibility.

[1] The names used for intent are meant to spoken if the AT doesn't now anything about them. Higher level math has thousands, if not tens of thousands of function names/notations. Using intent with a suitable value (e.g., <mi intent="ackermann-function">A</mi>) allows authors to express how the value should be spoken. However, if AT knows something about the name, it is free to override the name. For example intent="fraction($num, $denom)" might be spoken as "fraction num over denom end fraction" and not "fraction of num comma denom". If the fraction is numeric, might be spoken as "two thirds", etc. The MathML WG is developing a "core" set of names all AT should know.

[2] Most braille is presentational, but there are a few exceptions. For example, in Nemeth, : is brailed a little differently if it is meant to be a ratio as opposed to anything else (such as a mapping). intent could be used by a braille generator in that case.

[NSoiffer]

Any updates? Any interest in a joint call to talk through the issue?

[dginev]

If we were able to answer only one high-level question between the ARIA and Math groups, I would offer the strategic one:

Q: What degree of involvement should each group have in designing markup for "accessible math syntax" ?

• A1: exclusive to ARIA
  • where ARIA has interest in some e.g. aria-intent analog to MathML Intent
• A2: exclusive to MathML
  • we continue work on MathML intent
  • and explicitly recommend not using aria-* attributes.
• A3: ARIA+MathML synergy with ARIA as-is
  • we continue work on MathML intent
  • and use ARIA attributes as already prescribed
  • requiring no further involvement by the ARIA group
• A4: ARIA+MathML synergy with both groups open to new enhancements
  • requires an in-depth discussion between the two groups to see where each has interest.

If math syntax is currently outside of the bandwidth of the next ARIA milestone, we would need to make an informed choice between A2 and A3 in the Math WG. Which is workable - but it would be very helpful if we get an official thumbs up from participants here.

Maybe this is a productive bump to get the issue moving forward? Given the ARIA 1.4 milestone assignment, I think I am already getting an early hint. Thank you!

[cookiecrook]

We discussed briefly in the ARIA call today. If I understand correctly, this is an implementation problem, not an ARIA problem.

If MathML is sufficiently unambiguous in these scenarios, there is a bug (or bugs) to be resolved in either the user agents or assistive technologies. For example, WebKit+VO supports MathML natively with at least Nemeth, UEB, and 8-dot Computer Braille... (Admittedly there may be unknown or other outstanding bugs.) If JAWS and NVDA (mentioned above) don't convey factorials appropriately that's a bug for those screen readers or for the UAs/browsers that they are sitting on. Isn't it?

Otherwise, if MathML is ambiguous in this or other scenarios, that would be a semantic problem for MathML WG to solve. I'm not a MathML expert, but my understanding is that ! is unambiguously factorial in an mo context. If the not (!=) usage may be misinterpreted here, working around it in ARIA would not be the right solution.

[NSoiffer]

Math is sometimes ambiguous, especially since mathematicians (and others) long ago ran out of new symbols so they tend to repurpose them. ! is likely pronounced "factorial", but not always. I'm sure @dginev can easily come up with five other meanings/pronunciations (he works on a project involving arXiv that has millions of scientific articles). [I'm not suggestion he do this unless someone wants proof] The introduction of intent is meant to allow authors to convey alternate meanings/vocalizations of functions, variables, and symbols (the braille very likely doesn't change though).

I believe this is a spec issue because there are specifications for accessibility API mappings along with accessible name calculations that include the use of aria-label. If aria-label is allowed (encouraged???) inside of MathML, then this potentially interacts with the proposed intent attribute and hence how this is resolved is a spec issue.

In practice, I think ARIA is ignored in MathML at the moment (although I'm not sure about Safari+VoiceOver). Perhaps because Apple has extended their accessibility API to include MathML tags, the accessible name computation is affected on Apple OSs because of the MathML? On other platforms, there is not complete coverage for math/MathML in the accessibility platform, so I think conflicts are avoided because the mappings aren't used at the moment to get speech for math, but could be a problem in the future.

[cookiecrook]

Math is sometimes ambiguous, especially since mathematicians (and others) long ago ran out of new symbols so they tend to repurpose them. ! is likely pronounced "factorial", but not always. I'm sure @dginev can easily come up with five other meanings/pronunciations (he works on a project involving arXiv that has millions of scientific articles). [I'm not suggesting he do this unless someone wants proof] The introduction of intent is meant to allow authors to convey alternate meanings/vocalizations of functions, variables, and symbols (the braille very likely doesn't change though).

Thanks. We don't need additional proof that ambiguity exists.

Since this ambiguity exists in the Math, I think the other members of the ARIA working group would agree that ARIA is not the place to solve the issue. If a MathML intent attribute solves the ambiguity problem, that's a reasonable path for the MathML stakeholders to consider.

The issue of whether certain ARIA attributes should be allowed in some contexts is worth considering separately. I have argued that a native MathML solution provides a better experience than a the common polyfill approaches, but without better MathML support in other browsers, we're at an impasse. Ideally a polyfill like MathJAX would only override the experience where native support was missing. Last I checked, it overrode the rendering in Safari, thereby breaking the native braille, speech, and sub-formula exploration support run VoiceOver.

[cookiecrook]

A combination of A2 and A3 sounds most practical. Where a MathML intent becomes supported, there may be less reason to use polyfills and DOM-level accessibility overrides. Until that time, A3 may be necessary to ensure fallback support.

In either case, I think this issue can be closed. Do you agree?

[NSoiffer]

Thanks for the feedback.

I agree it can be closed. If/when someone raises the issue of ARIA and MathML interaction with a specific use case, we can explore how to resolve it then.