Display floating point numbers with high precision

[Joe7M]

see: https://www.syntaxbomb.com/smallbasic/rounding-numbers/

The Round() command should be straightforward, but sometimes the program seems to round then add or subtract 0.00000000001 to the number, any idea why?
I compiled this to illustrate. I seem to be getting the same problem with external files. An alternative to round is using an expression like INT(100*number)/100, but it does the same.

FOR n=1 TO 100
a=100*RND
PRINT a, ROUND(a,3)
next n

[Joe7M]

I had a look to the source code of SmallBASIC. It is not a problem of round() as mentioned above, but a problem of the function bestfta_p(...) in fmt.c, which is used to convert a floating point number to a string for printing it to the screen.

It seems, that the function bestfta_p(...) is at the edge of precision. Sometime in line 167 of fmt.c fpart = fround(frac(x), FMT_RND) * FMT_xRND; the rounding error occurs.

One possible solution could be, to remove bestfta_p(...) and just replace it by sprintf(dest,"%.14G", x); . sprintfwill take care of the correct conversion. I made the following table to show what happens with bestfta_p(...)and what is the output of sprintf

                        INPUT      bestfta_p(...)        sprintf(dest,"%.14G", x); 
             0.00000000000001:     10E-15                1E-14
              0.0000000000001:     10E-14                1E-13
               0.000000000001:     10E-13                1E-12
                0.00000000001:     10E-12                1E-11
                 0.0000000001:      1E-10                1E-10
                  0.000000001:      1E-9                 1E-09
                   0.00000001:     0.00000001            1E-08
                    0.0000001:     0.0000001             1E-07
                     0.000001:     0.000001              1E-06
                      0.00001:     0.00001               1E-05
                       0.0001:     0.0001                0.0001
                        0.001:     0.001                 0.001
                         0.01:     0.01                  0.01
                          0.1:     0.1                   0.1
                            1:     1                     1
                           10:     10                    10
                          100:     100                   100
                         1000:     1000                  1000
                        10000:     10000                 10000
                       100000:     100000                100000
                      1000000:     1000000               1000000
                     10000000:     10000000              10000000
                    100000000:     100000000             100000000
                   1000000000:     1000000000            1000000000
                  10000000000:     10000000000           10000000000
                 100000000000:     100000000000          100000000000
                1000000000000:     1000000000000         1000000000000
               10000000000000:     10000000000000        10000000000000
              100000000000000:     100000000000000       1E+14
             1000000000000000:     1E+15                 1E+15
             
                      1e10000:     INF                   INF
             
             0.20361889339983:     0.20361889339983      0.20361889339983
            72.20361889339983:    72.20361889339983     72.2036188934
          1072.20361889339983: 10072.20361889340165  10072.203618893
    round(0.20361889339983,3):     0.204                 0.204
   round(72.20361889339983,3):    72.20399999999999     72.204
round(10072.20361889339983,3): 10072.20399999999972  10072.204
                        0.204:     0.204                 0.204
                       72.204:    72.20399999999999     72.204

for example fmt.c line 216 an following:

/*
 * best float to string (user)
 */
void bestfta(var_num_t x, char *dest) {
  sprintf(dest,"%.14G", x);  
}

/*
 * float to string (user, E mode)
 */
void expfta(var_num_t x, char *dest) {
  sprintf(dest,"%.E", x);
}