Intent: Comparison with SRE #421
Comments
|
I'm not sure I understand your workflow, but i think the target intent values can be improved/simplified
so I think you should get "E equals m c squared" from |
Me neither -- the workflow is not established yet. I am just thinking about how one can transition from the current format to intent without losing semantics.
It will be hard to figure out what will be helping and what not algorithmically. For the SRE the infixop seems to be helpful. looking at the speech output for the respective subtree <mrow data-semantic-type="infixop" data-semantic-role="implicit" data-semantic-annotation="clearspeak:unit" data-semantic-id="7" data-semantic-children="2,5" data-semantic-content="6" data-semantic-parent="8" data-semantic-speech="m c squared">
Thank you. This made me think of trying it out with https://nsoiffer.github.io/MathCATDemo/ apparently, there are no intent attributes needed to generate "E equals m c squared" So for this example, what would expose in the final output would be And we would not need any intent attributes here due to intelligent defaults. As a next step, I will see if I discover differences in the speech output of SRE and MathCat to understand when intent values are helpful. |
|
@physikerwelt could you elaborate on
To my understanding, SRE can continue using any What will be important if you want to use SRE with intent is that a native capability is implemented in SRE that can recognize input trees that have already been annotated with the "intent" attribute. And then using that value directly. For example, if I provide the input MathML <mi intent="indicator-function">𝟏</mi>SRE would need to pass the "indicator-function" to the speech string (maybe with an optional internationalization step), rather than run its own analysis to guess what 𝟏 meant. But if I only provide In both cases, to my understanding SRE, as well as any other AT tool, should be welcome to deposit as many |
|
@dginev thank you for the clarification. It will, however, be hard to generate test cases, as there is no MathML with intent. For example, in Wikipedia, we only have TeX, and the possibility for semantic annotations such as https://en.wikipedia.org/w/index.php?title=Special:MathWikibase&qid=Q35875 I was hoping that one could generate the intent from these sources. For example, I just annotated the definition for indicator function in Wikipedia. Now I would find a method that amends <math display="block" xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \mathbf {1} _{A}(x):={\begin{cases}1~&{\text{ if }}~x\in A~,\\0~&{\text{ if }}~x\notin A~.\end{cases}}}">
<semantics>
<mrow class="MJX-TeXAtom-ORD">
<mstyle scriptlevel="0" displaystyle="true">
<msub>
<mrow class="MJX-TeXAtom-ORD">
<mn mathvariant="bold">1</mn>
</mrow>
<mrow class="MJX-TeXAtom-ORD">
<mi>A</mi>
</mrow>
</msub>
<mo stretchy="false">(</mo>
<mi>x</mi>
<mo stretchy="false">)</mo>
<mo>:=</mo>
<mrow class="MJX-TeXAtom-ORD">
<mrow>
<mo>{</mo>
<mtable columnalign="left left" rowspacing=".2em" columnspacing="1em" displaystyle="false">
<mtr>
<mtd>
<mn>1</mn>
<mtext> </mtext>
</mtd>
<mtd>
<mrow class="MJX-TeXAtom-ORD">
<mtext> if </mtext>
</mrow>
<mtext> </mtext>
<mi>x</mi>
<mo>∈<!-- ∈ --></mo>
<mi>A</mi>
<mtext> </mtext>
<mo>,</mo>
</mtd>
</mtr>
<mtr>
<mtd>
<mn>0</mn>
<mtext> </mtext>
</mtd>
<mtd>
<mrow class="MJX-TeXAtom-ORD">
<mtext> if </mtext>
</mrow>
<mtext> </mtext>
<mi>x</mi>
<mo>∉<!-- ∉ --></mo>
<mi>A</mi>
<mtext> </mtext>
<mo>.</mo>
</mtd>
</mtr>
</mtable>
<mo fence="true" stretchy="true" symmetric="true"></mo>
</mrow>
</mrow>
</mstyle>
</mrow>
<annotation encoding="application/x-tex">{\displaystyle \mathbf {1} _{A}(x):={\begin{cases}1~&{\text{ if }}~x\in A~,\\0~&{\text{ if }}~x\notin A~.\end{cases}}}</annotation>
</semantics>
</math>with the intents that might be derived from the special page. |
This is indeed a very central problem for MathML Intent, not just for this issue. Without a solid authoring workflow, and existing tools that support that authoring soon, the spec will be a "wish" more than anything. I think we should have at least a latexml capability and a mathjax extension that can emit "intent" attributes from some well-chosen LaTeX macro dialect. I don't know which tool is currently used to generate MathML for wikipedia (I remember you wanted to work on a new alternative?), but having that emit "intent" from a couple of new macros will also go a long way towards a real workflow. I had briefly brainstormed a sample idea of what such TeX authoring may look like at this comment. But we need an actual solid implementation, maybe even a CTAN package with a fixed name that various conversion tools could point to, the way e.g. mathjax has a named extension for "amscd" and latexml has a named binding for "amscd.sty.ltxml", both of which target the same amscd.sty on CTAN. |
Ok. While this is not the answer, I wanted to hear it shows that I still do not understand the intent attribute. Reading the feedback from the TPAC meeting indicates that I am not the only person. Thus I think it would not be a waste of time if I understand why we use an attribute rather than encoding intent hints using content MathML to be able to communicate this to others.
@AndreG-P @Hyper-Node and I are working on a native PHP implementation. We will thus be able to generate anything. However, we are following a strictly test-driven development cycle. With the TDD approach, we could generate valid (strict) content MathML, as well as valid HTML attributes. In addition, it seems hard but doable to implement a checking software that uses intent grammar and checks if an intent attribute is valid based on the grammar rules. However, if we develop an intent grammar checker, we might introduce bugs in it. Thus it would be great to have the opportunity to cross-validate this with other implementations. However, before investing here, it would be great to understand what the benefits of these very complicated attributes are. To me, it seems that either the data-* attributes from SRE as well as content MathML are much closer to today's data structures supported by browsers.
I would like to step one step back and understand what we need to capture before discussing the encoding. In the codemeta project, we use the term cross-walks to express the ability to translate from one encoding to another. Maybe we can start with and end-to-end analysis of the example of indicator-function you provided above. Without any intent information mathoid (SRE) currently generates the following speech string:
whereas mathcat generates
With my current annotation in Wikidata one can infer that indicator function and can be explained as a from line 142 in the open-intent table one can derive from column c that it is an indexed function. This information can also be exported to wikidata as the links to the wikidata items are already implicitly in the table via the wikipedia links. In this particular example, it will be difficult, however since the indicator function is only in column F as an alternative, but let's ignore this detail for now. With this information, mathcat should now be able to figure out that times are incorrect. However, what should SRE generate in that case? What has been generated does not seem wrong (just a bit more verbose). To come back to the title of the issue... If we wanted that SRE picks up semantics from Wikipedia we would, regardless of the format, be able to specify test cases with tuples of MathML input and expected speech output. |
Well, I don't have great news here either - Content MathML has an even worse authoring problem than intent, when it comes to informal mathematical documents written by human authors. One of the excellent design choices in intent is that it allows "progressive remediation", via allowing "partial annotation" -- you can have a formula with 200 token elements and only annotate 1 of those elements with "intent". To create Content MathML for the same expression you would need to map each of the 200 tokens to some Content equivalent. Authoring that is an even greater challenge (again, for informal mathematical documents, such as the ones on arXiv and wikipedia). |
Ok. I was not aware of that. I assumed one could use semantics elements to group presentation and content together. Why is that not possible?
|
Let me mention that when that column was populated with information, I did not aim at creating a grammar rule set. It was mostly to allow easy grouping in pivot tables, so that we can get a human-friendly overview of the different kinds of notations ("this view shows infix operators", "this view shows all indexed functions" etc). I think Paul also had a demo using that kind of sorting capability. I think automatically inferring intent is a much harder task than consulting a list of notations. There is context, both external and internal to the expression, and there is always a chance of cascading failures triggered by unknown notations. The main added benefit of the intent attribute, and the lists, should be that when an authoring tool creates MathML with "intent" set, the ATs will use that attribute value to create more conceptual/natural speech strings.
I remember it being discussed during a meeting, and the group had wide consensus at the time that the ergonomics of fine-grained |
|
Oh, and a question. For the feature you describe here:
Is it accurate to state that neither SRE, nor Content MathML currently have a mechanism to carry that "explanation"/"description" ? This is the kind of auxiliary information that seems to be the target of the We also currently don't have a place to add such information in intent. So it would need to be a custom (i.e. non-standard, platform-specific) wikidata/wikipedia feature, unless we introduce some new MathML markup for it. |
Ok. So one needs to have something like <mi intent="indicator-function($a1)">𝟏</mi>to give MathCat a chance to understand that this is not times, correct?
This does not seem convincing to me. Adding a new attribute seems much more overhead than using existing elements. I think if the MathML WG really intends to add a new attribute, we must have a better answer to the question. In particular, for Mathoid I think it would be more reasonable to add very few shallow content MathML elements and invest time to make a pull request for SRE to be able to improve the semantics based on the content MathML annotation. |
I think this is good enough. I hope there is a way to provide this information only if it is requested. Putting all these descriptions in the DOM would add too much data volume. |
|
You should consider participating more in prototyping the For applying an indicator function, the intent syntax would be: <mrow intent="$idfun($set,$element)">
<msub>
<mi arg="idfun" intent="indicator-function">𝟏</mi>
<mi arg="set">A</mi>
</msub>
<mo>(</mo>
<mi arg="element">x</mi>
<mo>)</mo>
</mrow>and a hypothetical alternative via Content MathML would be: <semantics>
<mrow>
<msub>
<mi>𝟏</mi>
<mi>A</mi>
</msub>
<mo>(</mo>
<mi>x</mi>
<mo>)</mo>
</mrow>
<annotation-xml encoding="MathML-Content">
<apply>
<csymbol cd="intent-open">indicator-function</csymbol>
<ci>A</ci>
<ci>x</ci>
</apply>
</annotation-xml>
</semantics>That tree gets more verbose if we also include the id/xref mechanism for cross-referencing the individual pieces (so that AT knows it was indeed the bold 1 that was presenting the indicator-function). see my follow-up comment for a slightly larger example. As to ARIA, their group resolved to recommend to us to invent in MathML itself any potential mechanisms for accessibility that we may need, so that we capture the types of needs specific to math syntax. That conversation happened in ARIA #1723, you're welcome to review it. I think for now we have accepted their guidance in that regard. But if we do not provide any markup that matches |
|
Just a reminder why There are a few other negatives of I should note that AFAIK, |
|
Its important to point out that both |
The hypothetical content MathML looks valid to me according to the current MathML 3 spec and only slightly more verbose to one or the other verbosity metric. So I would really appreciate, if we could just allow this as a valid intent encoding. An even less verbose notation is LaTeX which brings us back to what we see in a typical HTML source today. Moreover, the DLMF/DRMF projects are good staring points to put more semantics with minimal typing into LaTeX. If verbosity is our goal we should just allow MathML with only LaTeX annotations. This is similar to what TEI does. Implementing validation for the proposed intent grammar which somehow breaks the canonical XML structure seems a nightmare to me. Putting a grammar inside an attribute seems to me like developing against the XML framework. This is my personal viewpoint. When others want to use the attribute this is fine, I just don't see myself implementing this, yet. |
|
@physikerwelt replying to your comments out of order
This seems to be the fundamental objection, that Looking at @dginev's But that markup is very fragile especially in any context where the mathml is source and needs to be edited (as opposed to simply being re-generated very time) . Also as the ids have document scope having so many ids in all Mathml makes re-use difficult. This was really the original motivation for That said, we don't have many tools generating intent, and even fewer using it, and browsers don't currently pass it to AT at all. |
|
It is cool that we come to the bone of the issue, a bone that @physikerwelt long bragged about. I would like to suggest that we propose BOTH parallel markup and attributes, as alternative encodings of intents. That is, I would like to suggest to introduce a small statement about an I know for sure, however, that general utility is low, just as is parallel markup: desirable but rarely done. |
|
@polx the generic mapping
to Is already used in the content-to-strict transformation to map any attributes (other, class, ...) not allowed in strict-content/openmath. Having made such a mapping for |
|
I think development the generation and consumption of attribute values new on the complexity level of x-path for production use is nothing one can do on a voluntary basis in an reasonable amount of time. I don't see myself editing intent attributes with an editor as I would probably become frustrated about my own typos, missing or duplicated arg values and so on. It would probably be helpful to understand if there are people willing to edit anything else than LaTeX. Currently in my echo chamber I have the feeling that the workflow starts from LaTeX and if anything is wrong the LaTeX is changed and MathML is regenerated. If this is the workflow, verbosity of the generated MathML source code is not the most pressing problem. According to my knowledge the backwards translation from MathML to LaTeX has not been fully developed. So I think bootstrapping intent editing for MathML generated from LaTeX is not a low hanging fruit even if we had an intent package on ctan. However editing MathML is to my mind of lower priority than getting tools to consume intent and generate better speech output in comparison to the output without intent. I see basically two strategies, wait for someone to implement attribute based intent generation and verification tools or to abuse content MathML to encode intent now. |
|
I see it the same way: for more than 99% of authors, all that will happen is they The authored source is converted to MathML with intent (and maybe also 'isa'). If the author cannot get the correct MathML+intent, they will view that as Nobody will edit the MathML. Correction: 99.9% of the people creating MathML+intent will not edit the MathML. Let's not have the other 0.1% have an outsize voice in the decisions we are making. |
|
@davidcarlisle wrote:
That is a fair comment, I should also offer a counter-example. I hand-crafted the smallest example I could think of that can exemplify the partial annotation approach, notably relying on the id/xref technique, but also requiring use of the The key new snippet is: <apply>
<csymbol cd="intent-open">multinomial-coeffficient</csymbol>
<share href="id-for-top"></share>
<share href="id-for-bottom"></share>
</apply>Find the full example with more comments at this gist: The attribute variant is 25 lines, single tree. Admin note: continued work on this "mixed tree" alternative should probably get a dedicated new issue and discussion. I hope bootstrapping these examples gets various participants here (de)motivated enough to do some prototyping. |
|
I'm not sure what SRE is here (Site Reliability Engineering?),
but the main focus of intent is disambiguation for accessibility.
From that pov, most of the data you've supplied in the SRE form seems
(I'm guessing) to be already reflected as reasonable default
interpretations of the MathML itself, so doesn't necessarily need to be
carried over into intent (although making "equality" (aka "equals)
explicit is welcome!). The most significant disambiguation, I think, is
superscript as power, but it isn't clear where that came from.
|
|
I can see authors writing
\abs{A}
\card{A}
\order{A}
\det{A}
all of which expand in a PDF to |A|, but the macro definition contains
extra information which enables conversion to MathML with intent
(or other attributes) which disambuates the meaning.
|
|
SRE = Speech Rule Engine. Volker Sorge's code for pronouncing math,
used by MathJax (and hence will be used often, by many people).
…On Tue, 11 Oct 2022, bruce miller wrote:
I'm not sure what SRE is here (Site Reliability Engineering?),
but the main focus of intent is disambiguation for accessibility.
From that pov, most of the data you've supplied in the SRE form seems
(I'm guessing) to be already reflected as reasonable default
interpretations of the MathML itself, so doesn't necessarily need to be
carried over into intent (although making "equality" (aka "equals)
explicit is welcome!). The most significant disambiguation, I think, is
superscript as power, but it isn't clear where that came from.
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because youcommented.[AABTULA3S6XWEOP6PQYWPLTWCULBVA5CNFSM57ZJ7QTKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOJ
PZUJBY.gif] Message ID: ***@***.***>
|
|
Can this be closed @physikerwelt ? |
|
I would keep it open until we have agreed on a close to final the intent syntax, and then check how this compares with the SRE syntax. |
|
@physikerwelt What is the remaining goal for the comparison with SRE? It would make sense to compare if we wanted to adapt the Intent syntax to be a better fit with SRE's I was actually hoping we would get the SRE author himself to attend a W3C Math meeting and give us his perspective on the Intent effort, but I understand that wasn't possible to arrange for the previous meeting with AT vendors (as documented in #461). Maybe finding a path to such a meeting could be a good motivation for keeping this issue open? I would be in favor of closing here unless we can redirect the discussion in some practical direction. |
|
To me the goal is to justify the need of intent and demonstrate what the advantages of intent in comparison to the previous SRE format. If one thinks that this is not useful, we can close the issue. |
|
I have asked Volker to join the meeting, but he's been busy and (maybe just my impression), not that interested in discussing intent. SRE adds lots of |
It seems like we have a final intent syntax, although we still are adding concept names. @physikerwelt: is there something more that you are waiting on? Can we close this issue? |
|
The comparison between SRE and intent is still an interesting scientific micro contribution, but not relevant to this group. |
Goal: To check if the intent attribute is practical, I would like to investigate if we can map SRE-enriched MML to MathML with intent attributes. For Wikipedia, we currently use SRE to enrich MathML and plan to implement a native rendering method that supports intent attributes. Before we jump into the details I would like to investigate if we can compare the semantics in the current format and the new intent attribute.
Example: Currently, the input$x^n$ will be rendered as
In the MathML spec draft we see
If we ignore the additional mrow, replace power with superscript and use the variable names from SRE we get
So one initial rule to transform SRE enrichment to intent is
if data-semantic-type="identifier" add arg html attribute with the value of data-semantic-idif data-semantic-type="superscript" add intent html attribute with the value of power($data-semantic-chilren[0],$data-semantic-chilren[0])This would almost work, except that we need to add some random but fixed NameStartChar to comply with the intent grammar. If we choose
aour SRE-generated intent would readLet's look at a more involved example. Currently, mathoid generates from the input$E=mc^2$ the following output
after some days of coding this could become
Is that really what we want? While the alttext "upper E equals m c squared" is biased towards English (and a few other languages) I still think this (current output) is more helpful for people that do not see in comparison to the intent enriched presentation tree. I don't really see how one could encode this information with intent and arg attributes in the presentation part of the MathML tree. Especially squared is something people would probably say, even though there is no 1:1 correspondence to the presentation tree.
Via some option tuning one can also generate deep speech annotations:
The text was updated successfully, but these errors were encountered: