Replace Tinymath with math.js

[gulderov]

Description

Migrate from tinymath to math.js

Tinymath has been deprecated for several years, and as part of the modernization plan (#4306), we need to replace it.

Math.js is an extensive math library for JavaScript and Node.js. It features a flexible expression parser with support for symbolic computation, comes with a large set of built-in functions and constants. It is maintained project with good docs.

The evaluate method can replace tinymath.evaluate with security in mind

Examples :
2 + 3
add(params.count, 3)
[a = params.count * 100, f(x) = sqrt(x), f(a)]

Examples that would NOT work:

parse, derivative and evaluate are disabled as risky:
[ h = parse('x^2 + x'), dh = derivative(h, 'x'), dh.evaluate({ x: params.count }) ]

Issues Resolved

closes #3655
closes #3656
closes #3657

Testing the changes

Open [eCommerce] Revenue Dashboard with sample data => Edit => Create new => New Visualization TSVB => Press + Add Series => Press + Add Metric => Last Aggregation Math => Variable name: count, Expression: params.count + 10 => Check graph

Check List

• yarn test:jest
• Update CHANGELOG.md
• Commits are signed per the DCO using --signoff

[codecov]

Codecov Report

Merging #4492 (f5f0a82) into main (80fe14d) will increase coverage by 0.00%.
Report is 1 commits behind head on main.
The diff coverage is 100.00%.

@@           Coverage Diff           @@
##             main    #4492   +/-   ##
=======================================
  Coverage   66.49%   66.50%           
=======================================
  Files        3402     3403    +1     
  Lines       65011    65021   +10     
  Branches    10401    10401           
=======================================
+ Hits        43230    43242   +12     
+ Misses      19212    19211    -1     
+ Partials     2569     2568    -1     

Flag	Coverage Δ
Linux_1	34.81% <100.00%> (+0.01%)	⬆️
Linux_2	55.31% <ø> (ø)
Linux_3	44.61% <ø> (+<0.01%)	⬆️
Linux_4	34.89% <ø> (ø)
Windows_1	34.83% <100.00%> (+0.01%)	⬆️
Windows_2	55.27% <ø> (ø)
Windows_3	44.61% <ø> (ø)
Windows_4	34.89% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Changed	Coverage Δ
...eseries/public/application/components/aggs/math.js	18.18% <ø> (ø)
...er/lib/vis_data/response_processors/series/math.js	87.23% <ø> (ø)
...ib/vis_data/response_processors/series/evaluate.ts	100.00% <100.00%> (ø)

... and 1 file with indirect coverage changes

[abbyhu2000]

Do we disable the following math.js functions due to security issues? How do you know these are the only functions that we need to disable?

[gulderov]

These functions are marked as unsafe by the mathjs documentation https://mathjs.org/docs/expressions/security.html

A user could try to inject malicious JavaScript code via the expression parser. The expression parser of mathjs offers a sandboxed environment to execute expressions which should make this impossible. It’s possible though that there are unknown security vulnerabilities, so it’s important to be careful, especially when allowing server side execution of arbitrary expressions.

The expression parser of mathjs parses the input in a controlled way into an expression tree or abstract syntax tree (AST). In a “compile” step, it does as much as possible preprocessing on the static parts of the expression, and creates a fast performing function which can be used to evaluate the expression repeatedly using a dynamically passed scope.

The parser actively prevents access to JavaScripts internal eval and new Function which are the main cause of security attacks. Mathjs versions 4 and newer does not use JavaScript’s eval under the hood.

[abbyhu2000]

Is this evaluate() function different than math.evaluate() from line 9?

[gulderov]

It is the same evaluate function. The idea is to disable risky functions on the user side. We can find a list of all risky functions in the mathjs documentation: https://mathjs.org/docs/expressions/security.html.

So it is possible to do: mathjs.evaluate('add(2, 2)').
At the same time, it is impossible to evaluate inside the evaluate function itself: mathjs.evaluate('evaluate("add(2,2)")').

[abbyhu2000]

Thanks @gulderov, no issues functionality-wise, but some questions regarding the disabled functions.

[AMoo-Miki]

Thanks for this PR.

parse, derivative and evaluate are disabled

I labelled this as breaking change; if that is not what you meant and the new library can be used just like the old one, lemme know and I will remove the label.

[gulderov]

Thanks @gulderov, no issues functionality-wise, but some questions regarding the disabled functions.

@abbyhu2000 Thank you so much!
I have answered all the questions. Please let me know if I need to provide more information

[gulderov]

Thanks for this PR.

parse, derivative and evaluate are disabled

I labelled this as breaking change; if that is not what you meant and the new library can be used just like the old one, lemme know and I will remove the label.

@AMoo-Miki, library change could be marked as breaking for sure
As I can see tinymath is more limited than mathjs and disabled functions are not possible in tinymath

It is not a drop in replacement for users. But in most cases it would work as is or with minimal changes

• clamp -> ['x=1', 'minimum=2', 'maximum=3', 'min(max(x, minimum), maximum)'] // 2
• first -> ['a=[1,2,3]', 'a[1]'] // 1
• last -> ['a=[1,2,3]', 'a[a.size()]'] // 3
• unique -> ['setDistinct([1,1,1,2])'] // [1, 2]

[abbyhu2000]

Thanks for the explanations!

[ananzh]

cypress fail but not caused by this PR.