\mmlToken does not evaluate TeX in its argument #2706

mkuron · 2021-06-13T19:51:04Z

Issue Summary

The \mmlToken macro cannot be used to wrap TeX into custom MathML. It can currently only be used to wrap plain text.

Steps to Reproduce:

Here is sample code and a screenshot. The first and third line render as expected because they are plain (Latin and Greek) text, while the second line leaks TeX code to the page. I would expect lines two and three to look identical.

<!DOCTYPE html>
<html>
<head>
<meta charset="UTF-8">
<script src="https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-chtml.js"></script>
</head>
<body>

\(\newcommand{\vec}[1]{\mmlToken{mi}[mathvariant="bold-italic"]{#1}}\)

\begin{align}
a &= \vec{a} \\
\alpha &= \vec{\alpha} \\
α &= \vec{α}
\end{align}

</body>
</html>

Technical details:

MathJax Version: 3.1.4
Client OS: macOS 11.4
Browser: Safari 14.1.1, Chrome 91.0.4472.101

Notes

@dpvc's answers in #2671 and #2595 led me to \mmlToken. The documentation on it is somewhat sparse.

I assume this can be worked around by implementing a custom \vec macro on the JavaScript level, but I have not succeeded at doing that. My TeX code is coming from an external source and I can't easily change that to deliver α instead of \alpha, so I need to get the formatting right via MathJax.

The text was updated successfully, but these errors were encountered:

dpvc · 2021-06-15T14:58:00Z

You are correct that the argument to \mmlToken is raw text, and is not processed as TeX. The contents of a token element in MathML must be text, and things like \alpha produce <mi>α</mi>, which is not allowed in a token element, so the argument can't be processed in that way.

It is also true that \mathbfit will not affect \alpha, just as it doesn't in actual TeX. But you can use \boldsymbol to make a bold version of \alpha (and many other symbols). So perhaps using

\let\vec=\boldsymbol

would do what you need. (Or \newcommand{\vec}[1]{\boldsymbol{#1}} if you want to be more semantic about it. You can use the macros property of the tex configuration block to add this definition to your configuration:

MathJax = {
  tex: {
    macros: {
      vec: ['\\boldsymbol{#1}', 1]
    }
  }
};

mkuron · 2021-06-16T08:40:12Z

The \vec example above was a bit simpler than what I actually had in mind. Maybe we should convert this issue into a feature request along the lines of "There should be something like \mmlToken that evaluates its argument as TeX".

I'm trying to get an ISO 80000-2 style, which would look something like this (with a custom definition of \ifstrequal that comes from lwarp):

\newcommand{\vec}[1]{\ifstrequal{#1}{\nabla}{\mmlToken{mo}[mathvariant="bold"]{\unicode{x2207}}}{\mmlToken{mi}[mathvariant="bold-italic"]{#1}}}
\newcommand{\tens}[1]{\mmlToken{mi}[mathvariant="bold-sans-serif"]{#1}}

That is, vectors should be bold-italic and operators like \nabla should be bold-upright.

Your suggestion of \boldsymbol is actually pretty close, I was not previously aware that it produced upright nablas. The nabla does end up in <mi> MathML tags though (irrespective of the use of \boldsymbol), while I thought it should be <mo>. \boldsymbol does not work on \alpha when the latter is defined using \(\def\alpha{\unicode{x1D6FC}}\), which for some reason is inserted by lwarp.

zorkow · 2021-06-16T10:24:48Z

I believe that is exactly what the physics package implements. Can you simply use that? For example use

  MathJax = {
    loader: {load: ['[tex]/physics']},
    tex: {
          packages: {'[+]': ['physics']},
          physics: {
            arrowdel: true
          }}
  };

and you can then get the gradient operator with

\[\grad\]

Note that the arrowdel option is implemented in the upcoming 3.2 only (also see issue #2449). In older versions you cannot specify arrowdel and will only get the bold, upright Nabla without vector arrow. But 3.2 should be out soon.

dpvc · 2021-06-16T13:16:01Z

@zorkow, I think he wants the nabla without an arrow, so arrowdel should be false.

@mkuron:

Your suggestion of \boldsymbol is actually pretty close, I was not previously aware that it produced upright nablas.

It should produce the same thing as the original symbol, but in bold, so if it was italic before it would be bold italic, and if was upright, it would just be bold.

The nabla does end up in <mi> MathML tags though ... while I thought it should be <mo>.

In TeX, \nabla has TeX class ORD (not OP), and so MathJax places it in <mi mathvariant="normal"> rather than <mo>. One reason to make this choice is that MathJax tries to make its MathML work for native MathML renderers (not just for MathJax). While MathJax uses TeX spacing rules (and would render it correctly either way), other renderers use the MathML spacing rules, and using an <mo> for nabla would cause the wrong spacing (from the TeX point of view).

\boldsymbol does not work on \alpha when the latter is defined using \(\def\alpha{\unicode{x1D6FC}}\)

The symbols in the Math Aphanumeric Symbols unicode block specify both a character and its weight and style, and the MathML standard explicitly states that these should not be affected by mathvariant attributes. So U+1D6FC will always render as an italic lowercase alpha, regardless of the mathvariant. It is unfortunate that lwarp redefines them in this way, as that does disable the \boldymbol functionality. I suppose an argument could be made that \boldsymbol should modify the Math Alphanumerics as well.

Maybe we should convert this issue into a feature request along the lines of "There should be something like \mmlToken that evaluates its argument as TeX".

I can mark this as a feature request, if you want, but it is not likely to be something we pursue (though it could be a community contribution). As I mentioned above, the contents of a token element must be text, not other MathML elements, and "evaluating ... as TeX" in MathJax means converting to MathML. With the definition from lwarp, \alpha would produce <mtext>𝛼</mtext>, which can't be placed inside an <mi>. Perhaps you will say that \mmlToken should take just the text content of the processed TeX; that might be reasonable, but what do you do with \mmlToken{mi}{\frac{a^2}{b} + \sqrt{1-x}}? Will it be <mi>a2b+1ࢤx</mi>, and does that make sense?

mkuron · 2021-06-16T13:33:09Z

Thanks for the explanations.

Perhaps you will say that \mmlToken should take just the text content of the processed TeX; that might be reasonable

I guess it should be restricted to TeX elements that translate into a single Unicode code point, e.g. \nabla, \alpha, or \unicode{x1D6FC}. But given all the ambiguities, I can see why you wouldn't want to do this. Maybe I should preprocess my equations and replace all \alpha, \nabla, \unicode{...}, etc. with the corresponding Unicode characters before passing them to MathJax.

zorkow · 2021-06-16T13:34:33Z

@zorkow, I think he wants the nabla without an arrow, so arrowdel should be false.

That's even easier, as it should already work now. \grad will give you a bold, upright Nabla.
For 3.2 arrowdel is false by default.

The upgreek package new in v3.2 would at least take care of upright Greek characters. It will not help with bold, though.

dpvc · 2021-06-16T14:07:34Z

Maybe I should preprocess my equations and replace all \alpha, \nabla, \unicode{...}, etc. with the corresponding Unicode characters before passing them to MathJax.

That would work, but another option would be to redefine \mmlToken to do more preprocessing on the argument string. For example, here is a configuration that would allow you to process the greek letters, \nabla, \unicode, and any other symbols you care to define.

MathJax = {
  tex: {packages: {'[+]': ['mmltoken-update']}},
  startup: {
    ready() {
      const {Configuration} = MathJax._.input.tex.Configuration;
      const {CommandMap} = MathJax._.input.tex.SymbolMap;
      const BaseMethods = MathJax._.input.tex.base.BaseMethods.default;
      
      const mmlSymbol = {
        nabla:  '\u2207',
        alpha:  '\u03B1',
        beta:   '\u03B2',
        gamma:  '\u03B3',
        /*  etc.  */
      };
      const mmlSymbolRE = new RegExp(`\\\\(${Object.keys(mmlSymbol).join('|')})`, 'g');

      new CommandMap('mmltoken-update', {
        mmlToken: 'MmlToken'
      }, {
        MmlToken(parser, name) {
          const kind = parser.GetArgument(name);
          const attr = parser.GetBrackets(name, '');
          const text = parser.GetArgument(name)
            .replace(mmlSymbolRE, (match, c) => mmlSymbol[c])
            .replace(
              /\\unicode\{(.*?)\}/g,
              (match, n) => String.fromCodePoint(n.charAt(0) === 'x' ? parseInt(n.slice(1), 16) : parseInt(n))
            );
          parser.string = `{${kind}}[${attr}]{${text}}`;
          parser.i = 0;
          BaseMethods.MmlToken(parser, name);
        }
      });
      Configuration.create('mmltoken-update', {
        handler: {macro: ['mmltoken-update']}
      });

      MathJax.startup.defaultReady();
    }
  }
}

you can fill in the rest of the mmlSymbol list to suit your needs.

On the other hand, you might want to redefine \vec rather than \mmlToken so that you can do the argument checking in javascript itself, rather than through TeX. Something like

MathJax = {
  tex: {packages: {'[+]': ['bold-vec']}},
  startup: {
    ready() {
      const {Configuration} = MathJax._.input.tex.Configuration;
      const {CommandMap} = MathJax._.input.tex.SymbolMap;
      
      const mmlSymbol = {
        nabla:  '\u2207',
        alpha:  '\u03B1',
        beta:   '\u03B2',
        gamma:  '\u03B3',
        /*  etc.  */
      };
      const mmlSymbolRE = new RegExp(`\\\\(${Object.keys(mmlSymbol).join('|')})`, 'g');

      new CommandMap('bold-vec', {
        vec: 'BoldVector'
      }, {
        BoldVector(parser, name) {
          const base = parser.GetArgument(name)
            .replace(mmlSymbolRE, (match, c) => mmlSymbol[c])
            .replace(
              /\\unicode\{(.*?)\}/g,
              (match, n) => String.fromCodePoint(n.charAt(0) === 'x' ? parseInt(n.slice(1), 16) : parseInt(n))
            );
          const [kind, mathvariant] = (base === '\u2207' ? ['mo', 'bold'] : ['mi', 'bold-italic']);
          parser.Push(parser.create('token', kind, {mathvariant}, base));
        }
      });
      Configuration.create('bold-vec', {
        handler: {macro: ['bold-vec']}
      });

      MathJax.startup.defaultReady();
    }
  }
}

should do the trick.

mkuron · 2021-06-20T12:43:11Z

Thanks, your redefined \vec seems to be doing exactly what I was hoping for.

dpvc added the Expected Behavior This is how MathJax works label Jun 15, 2021

dpvc added Code Example Contains an illustrative code example, solution, or work-around v3 labels Jun 16, 2021

mkuron closed this as completed Jun 20, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

\mmlToken does not evaluate TeX in its argument #2706

\mmlToken does not evaluate TeX in its argument #2706

mkuron commented Jun 13, 2021 •

edited

Loading

dpvc commented Jun 15, 2021 •

edited

Loading

mkuron commented Jun 16, 2021

zorkow commented Jun 16, 2021

dpvc commented Jun 16, 2021 •

edited

Loading

mkuron commented Jun 16, 2021

zorkow commented Jun 16, 2021

dpvc commented Jun 16, 2021 •

edited

Loading

mkuron commented Jun 20, 2021

\mmlToken does not evaluate TeX in its argument #2706

\mmlToken does not evaluate TeX in its argument #2706

Comments

mkuron commented Jun 13, 2021 • edited Loading

Issue Summary

Steps to Reproduce:

Technical details:

Notes

dpvc commented Jun 15, 2021 • edited Loading

mkuron commented Jun 16, 2021

zorkow commented Jun 16, 2021

dpvc commented Jun 16, 2021 • edited Loading

mkuron commented Jun 16, 2021

zorkow commented Jun 16, 2021

dpvc commented Jun 16, 2021 • edited Loading

mkuron commented Jun 20, 2021

mkuron commented Jun 13, 2021 •

edited

Loading

dpvc commented Jun 15, 2021 •

edited

Loading

dpvc commented Jun 16, 2021 •

edited

Loading

dpvc commented Jun 16, 2021 •

edited

Loading