Decimal math produces rounding error

[brs17]

(1) Issue/Bug Description:
Utilizing the = calculator function in shell and attempting to do math with decimal places provides inaccurate rounding errors (yay javascript?)

(2) Steps to reproduce (if you know):
Launch the shell launcher, =1.1-1, see incorrect answer (0.10000000000009)

(3) Expected behavior:
Expected to see 0.1.

(4) Distribution (run cat /etc/os-release):
Pop!_OS 21.04

(5) Gnome Shell version:
3.38.4-1ubuntu3~21.04.1ubuntu1pop1~1623339707~21.04~2a1865d

(6) Pop Shell version (run apt policy pop-shell or provide the latest commit if building locally):

1.1.0~1627383380~21.04~475b49d

(7) Where was Pop Shell installed from:
Part of Pop!_OS installation

(8) Monitor Setup (2 x 1080p, 4K, Primary(Horizontal), Secondary(Vertical), etc):
2 x 1080p externals, both horizontal driven by laptop whose internal display is turned off.

(9) Other Installed/Enabled Extensions:
Only extensions included with Pop!_OS are currently installed

[ids1024]

(yay javascript?)

Not specifically JavaScript, this is the behavior of IEEE floating point as implemented by all modern processors. The calculation has the exact same result in Python. And presumably essentially any language.

I think calculators normally just use rounding to avoid this? That tends to seem more normal, though introduces rounding errors and some other funny behavior. Otherwise using more of a computer algebra system for computation is more exact but more complicated.

Edit: Apparently calculators often use BCD, but rounding floats to fewer digits also works. On actual computers, an arbitrary precision math library like GMP can be used. I guess that wouldn't be practical to use from gnome-shell though.

[ids1024]

I see gnome-calculator uses the GNU mpc/mpfr libraries for calculations.

Since the calculator is a plugin, but currently implemented in gjs, perhaps it could be rewritten in a different language. In Rust the rug crate wraps gmp/mpfr/mpc, while the bigdecimal crate seems to be widely used and would probably work.

[mmstick]

Plugins support IPC so it is possible to ship a Rust binary as a plugin. The only challenging aspect is that any replacement likely isn't a complete drop-in replacement for all of MathJS's features.

[ids1024]

After looking at the math.js docs, it seems it has support for arbitrary precision numbers (which should be similar to how other calculator programs works), and can be configured to use that by default: #1124

So it seems to still be an adequate solution. For now.