FIX display incorrect

[c3d]

13.72 / 25.4 should display as 0.5402, but currently displays as 5.4016e-1.

Reported by Ben Hagler

[c3d]

This is intentional, according to the following comment
(which I had forgotten)

        // Check if we need to switch to scientific notation in normal mode
        // On the negative exponents, we switch when digits would be lost on
        // display compared to actual digits. This is consistent with how HP
        // calculators do it. e.g 1.234556789 when divided by 10 repeatedly
        // switches to scientific notation at 1.23456789E-5, but 1.23 at
        // 1.23E-11 and 1.2 at 1.2E-12 (on an HP50G with 12 digits).
        // This is not symmetrical. Positive exponents switch at 1E12.
        // Note that the behaviour here is purposely different than HP's
        // when in FIX mode. In FIX 5, for example, 1.2345678E-5 is shown
        // on HP50s as 0.00001, and is shown here as 1.23457E-5, which I believe
        // is more useful.
        // Also, since DB48X can compute on 34 digits, and counting zeroes
        // can be annoying, there is a separate setting for when to switch
        // to scientific notation.

[c3d]

Discussion here: https://www.hpmuseum.org/forum/thread-20653.html

[achan001]

I think in FIX mode, we are not supposed to see "E"

Scientific notation switch really signals FIX conversion failed
(Either overflowed limited display or underflow to 0.000...)

// Also, since DB48X can compute on 34 digits, and counting zeroes
// can be annoying, there is a separate setting for when to switch
// to scientific notation.

To count zeroes, I found that normalized mode is better: 0.1 ≤ |mantissa| < 1
see IEEE Standard for Floating Point Numbers, by V Rajaraman

12345000 --> 0.12345e+008 --> 8 digits integer part
0.0012345 --> 0.12345e-002 --> 2 0's after decimal point.

[c3d]

Solving this by adding a new setting, MinimumSignificantDigits.

This selects the minimum number of significant digits before switching to
scientific notation in FIX mode.

The default value is 0, which is similar to how HP calculators
perform. For example, with 2 FIX, the value 0.055 will display as
0.06, and 0.0055 will display as 0.01.

A higher value will switch to scienfic mode to show at least the given
number of digits. For instance, with 2 FIX, if the value is 1,
then 0.055 will still display as 0.06 but 0.0055 will display as
5.50E-3. If the value is 2, then 0.055 will display as 5.5E-2.

In other words, setting the value to 0 emulates the behaviour of HP
calculators. Seting the value to 34 will switch to scientific
notation if not all digits can be displayed

[c3d]

@achan001

I think in FIX mode, we are not supposed to see "E"

Scientific notation switch really signals FIX conversion failed (Either
overflowed limited display or underflow to 0.000...)

The goal here is to emulate what HP calculators did.
They do show E under specific conditions, which the HP50 reference manual describes as follows:

However, the calculator automatically displays a number in scientific mode if
either of the following is true:
• The number of digits to be displayed exceeds 12.
• A nonzero value rounded to n decimal places otherwise would be displayed as
zero.

The goal is to enable that precise behaviour, while also enabling my preferred behavior, which is to show extra decimals instead of useless zeros.

// Also, since DB48X can compute on 34 digits, and counting zeroes
// can be annoying, there is a separate setting for when to switch
// to scientific notation.

To count zeroes, I found that normalized mode is better: 0.1 ≤ |mantissa| < 1
see IEEE Standard for Floating Point Numbers, by V Rajaraman

The problem of counting zeroes is not a software problem, it is a practical problem for humans.

HP will display 10 or 12 digits depending on the model, so you could end up with 1E-12 shown as .000000000001. If we followed that logic with the 34-digit DB48X, we would display .0000000000000000000000000000000001. I think this becomes useless. This is why there is the StandardExponent setting, which defaults to 9, and will switch to SCI mode beyond 9 digits.
To get an HP48-compatible display, you need to set it to 12.

[c3d]

Illustration of the new MinimumSignificantDigits setting. Consider these numbers:

With Dig 0 and Fix 4, you get the HP behaviour:

With Dig 1, the number in level 4 changes, because we want at least one real significant digit to show, and the 1 there is not significant (it’s rounding):

With Dig 2, the same rule applies for level 3, in order to show two significant digits (5.5 in that case):

With Dig 3 or 4, things do not change. With Dig 5, level 1 changes to display 5 significant digits. Level 2 never changes because all its significant digits are shown at Fix 4. This will be true for all values of Dig above 5 (up to 34).

So you get your usual HP behaviour with Dig 0, and I get the one I want with Dig 34.

[achan001]

I don't quite understand what 0 significant digit (HP behavior) mean.
If DIG is negative, does it then allow underflow? Example, FIX02 (0.001) = 0.00

Can you post the formula, for deciding when the switch (FIX --> SCI) occurs?

[achan001]

This is my attempt at understanding new feature, in terms of ULP
We flip from FIX to SCI, if abs(value) < min(10^Dig/2 ULP)
(unless FIX result exactly matched value already)

FIX04, 1 ULP = 0.0001

Dig=0: flip = 10^0 / 2 = 0.5 ULP
FIX04 (0.000055) = FIX (0.55 ULP) --> FIX04 = 0.0001
FIX04 (0.000550) = FIX (5.50 ULP) --> FIX04 = 0.0006

Dig=1, flip = 10^1 / 2 = 5 ULP
FIX04 (0.000055) = FIX (0.55 ULP) --> SCI04 = 5.5e-5
FIX04 (0.000550) = FIX (5.50 ULP) --> FIX04 = 0.0006

[achan001]

Can you post the formula, for deciding when the switch (FIX --> SCI) occurs?

Just update with my guess of formula, before issue get closed

FIX-N: ULP = 10^(-N)
FLIP = when(Dig==0, 1/2, 10^(Dig-1)) ULP

But, symmetric version may be better, with all FIX notation error ≤ 1/2 ULP
FIX display of 0.01 means 0.01 ± 0.005, not (sometimes) 0.0125 ± 0.0025

FLIP = (10^Dig - 1/2) ULP

Dig<0: FLIP < 0.0 ULP (will not flip to SCI)
Dig=0: FLIP = 0.5 ULP
Dig=1: FLIP = 9.5 ULP
Dig=2: FLIP = 99.5 ULP
...