You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Calc package shows wrong scientific value for some numbers: e.g. 12e6*2048 shows as 2.4576e+1.
For this particular example, multiplying number with 12e6, it shows wrongly for 12e6*834 though 12e6*4999 it seems, whereas 12e6*833 and 12e6*5000 work fine.
The text was updated successfully, but these errors were encountered:
At a cursory glance it seems that right-most "trailing" zeroes in the power are getting stripped off. Someone will have to look into how the Calc package is handling it. I'll see if I can take a crack at it tonight.
Some repros:
12e6*2048 should give 2.4567e+10 but gives 2.4576e+1.
100000000000e6*2048 should output 2.048e+20 but gives 2.048e+2.
Calc package shows wrong scientific value for some numbers: e.g. 12e6*2048 shows as 2.4576e+1.
For this particular example, multiplying number with 12e6, it shows wrongly for 12e6*834 though 12e6*4999 it seems, whereas 12e6*833 and 12e6*5000 work fine.
The text was updated successfully, but these errors were encountered: