Find a way to support custom Math expressions

[BoboTiG]

Sample output of the english Wiktionary parsing:

<math> ERROR with \begin{align}\frac{\pi}{2} & = \prod_{n=1}^{\infty} \frac{ 4n^2 }{ 4n^2 - 1 } = \prod_{n=1}^{\infty} \left(\frac{2n}{2n-1} \cdot \frac{2n}{2n+1}\right) \\[6pt]& = \Big(\frac{2}{1} \cdot \frac{2}{3}\Big) \cdot \Big(\frac{4}{3} \cdot \frac{4}{5}\Big) \cdot \Big(\frac{6}{5} \cdot \frac{6}{7}\Big) \cdot \Big(\frac{8}{7} \cdot \frac{8}{9}\Big) \cdot \; \cdots \\\end{align} in [Wallis product]

<math> ERROR with f:\Z_2^n \to \Z_2 in [bent function]

<math> ERROR with \frac = \frac in [circle of Apollonius]

<math> ERROR with \and in [conjunction]

<math> ERROR with n\in\N in [elementary symmetric polynomial]

<math> ERROR with (x_1, x_2, \ldots, x_m, y)\in\R^{m+1} in [graph]

<math> ERROR with f: \R^m\rightarrow\R in [graph]

<math> ERROR with a_i \isin R in [group ring]

<math> ERROR with \R^3 in [halfway model]

<math> ERROR with \begin{align}\rho(g, h) (0,x_1,\ldots,x_k) &= g(x_1,\ldots,x_k) \\\rho(g, h) (y+1,x_1,\ldots,x_k) &= h(y,\rho(g, h) (y,x_1,\ldots,x_k),x_1,\ldots,x_k)\,\end{align} in [primitive recursion]

<math> ERROR with \isin in [supermodular]

I guess Mediawiki is using custom symbols somehow. We should find a way to handle them.

[BoboTiG]

Sample output of the portuguese Wiktionary:

<math> ERROR with \Delta U = Q - \Tau in [calor]
<math> ERROR with \Tau in [calor]
<math> ERROR with \Delta U = Q - \Tau in [trabalho]
<math> ERROR with \Tau in [trabalho]

[BoboTiG]

If we find something, we should also update the Wiki.

[lasconic]

Mediawiki does some nasty tricks:
https://github.com/maxbuchan/viv/blob/0a90c08be554c069b43798db495d45666c893c9c/apps/mediawiki/htdocs/extensions/Math/math/texutil.ml#L95

[lasconic]

ERROR with f:\Z_2^n \to \Z_2 in [bent function]

\Z is not defined, it needs to be replaced : https://github.com/maxbuchan/viv/blob/0a90c08be554c069b43798db495d45666c893c9c/apps/mediawiki/htdocs/extensions/Math/math/texutil.ml#L425

<math> ERROR with \frac = \frac in [circle of Apollonius]

See https://github.com/maxbuchan/viv/blob/0a90c08be554c069b43798db495d45666c893c9c/apps/mediawiki/htdocs/extensions/Math/math/texutil.ml#L513

[lasconic]

Most of the errors can be fixed with simple string substitutions based on the list provided above. Just a couple are more problematic:

• In english primitive recursion : It uses \begin{align} and Tex complains Package amsmath Error: \\begin{align} allowed only in paragraph mode.
• In english Wallis product and circle of Apollonius : they use '\frac' and it should be rendered with an HTML table... according to https://github.com/maxbuchan/viv/blob/0a90c08be554c069b43798db495d45666c893c9c/apps/mediawiki/htdocs/extensions/Math/math/texutil.ml#L513 and https://github.com/maxbuchan/viv/blob/0a90c08be554c069b43798db495d45666c893c9c/apps/mediawiki/htdocs/extensions/Math/math/html.ml#L115
• In swedish terrasspunkt: The math expression contains square brackets... they are removed by our clean process...

Portuguese, French, the remaining english ones etc... they work ok with the PR to come.

[lasconic]

Note for later, to debug latex output, add "-d -1" in dvioptions

[lasconic]

I found why the few words using \begin{align} don't work.

See https://comp.text.tex.narkive.com/zTTDqtBD/help-with-amsmath-align-not-possible

render_expr = f"{expr}" if expr.startswith("\\begin{align}") else f"${expr}$"

would fix it but we need to change the IMAGE_CSS... Needs more work.

[Moonbase59]

Even if this is closed, let me add the ones my renderer found in the DE Wiktionary:

2022-02-04 02:05:54 ERROR: Couldn’t render 'N_{\rm A}'
2022-02-04 02:08:56 ERROR: Couldn’t render '\R'
2022-02-04 02:10:13 ERROR: Couldn’t render '\Zi\sqrt{5}'
2022-02-04 02:10:19 ERROR: Couldn’t render '\N'
2022-02-04 02:10:20 ERROR: Couldn’t render '\R'
2022-02-04 02:15:49 ERROR: Couldn’t render 'y(x) = e^{(x-x_0)A}\cdot y_0 + \int_{x_0}^x e^{(x-t)A}g(t){\rm d}t\ '
2022-02-04 02:18:11 ERROR: Couldn’t render '- 42 \in \R'

Mostly due to \R and \rm, I should think. Unfortunately, my renderer doesn’t spit out the article.

[lasconic]

You need to replace these :

ebook-reader-dict/wikidict/utils.py

Lines 478 to 486 in 8eb9ca4

	# see issue #1096
	expr = expr.replace("\\Z", "\\mathbb{Z}")
	expr = expr.replace("\\N", "\\mathbb{N}")
	expr = expr.replace("\\R", "\\mathbb{R}")
	expr = expr.replace("\\isin", "\\in")
	expr = expr.replace("\\and", "\\land")
	expr = expr.replace("\\Tau", "\\mathrm{T}")
	expr = expr.replace("\\infin", "\\infty")
	expr = expr.replace("\\rarr", "\\rightarrow")

See #1096 (comment)

Though, I feel like we are doing double work here...