Floating point precision in decimal representation

[schlimmchen]

I put some time into understanding what is going on with the issue described by @AndreasBoehm, because it drove me mad. I accumulated quite a bit of info while doing so. I wanted to put it into a comment in the respective PR, but it is simply off-topic, so I will offload it here, for me as a reference and in case someone actually wants to nerd about this with me.

The Issue

Even if i change it to 52,1 it will return back to the shown value.

I would love your input on this. Or anybody's input. ChatGPT was of no use. The thing is this: We previously did this voodoo when serializing into JSON:
https://github.com/helgeerbe/OpenDTU-OnBattery/blob/a87f9fa2cd070e2c30a4c48569eccb8235dc0817/src/WebApi_powerlimiter.cpp#L52

Since I felt that this has to be nonsense, I removed it:
https://github.com/helgeerbe/OpenDTU-OnBattery/blob/0c1909e4dc471da8939e59fcdd34b171419d45ce/src/Configuration.cpp#L98

And now we see this rounding issue.

What drives me mad is that I don't understand why this helps at all! What difference does it make? We take a float, multiply it by 100 and add 0.5, which makes sense. This preserves two decimal places and takes care of rounding. We now cast that to an integer and are left with 100 times the original value, preserving the first two decimal places. Now comes the fun part: We devide this integer by 100.0, which gives us a float again, but this time, it suddenly can accurately represent the value in question. Why? Is the float able to accurately represent the value, or is it not? When does the rounding/accuracy issue occur in the first place? If somebody could explain this, that would be very much appreciated.

This is even "recommended" in the ArduinoJson docs 🤷‍♂️

Two Internet calculators say that 51.3 is not be represented accurately by an IEEE754 (double precision) float value. Okay, I can accept that (and I understand why that is). Then why does the expression using multiplying and rounding and casting yield a value that can represent 51.3? The value has to be transported somehow to the operator=() implementation of that ArduinoJson object, so how does that work?

So, anyways, I will be putting the voodoo back in to "fix" this.

The Deep Dive

TL;DR

%f means six digits after the decimal point. Mystery solved.

Except that we should use %g instead, which I never heard of or seen until now. Which I think is its own mystery.

Findings

The floating-point issue was still not resolved after implementing this lambda (which I was afraid it wouldn't do):

    auto roundedFloat = [](auto val) -> float {
        return static_cast<int>(val * 100 + 0.5) / 100.0;
    };

When changing the return type to auto or double, the voodoo works.

I guess the software implementation for handling doubles or the double -> string function works different than the float -> string function? As far as I can tell neither type can represent 53.1 exactly.

I will leave this here for reference:

auto roundedFloat = [](float val) -> double {
    MessageOutput.printf("input(%.20f)(%.02f)(%f)=0x", val, val, val);
    for(int i = 0; i < 4; ++i) {
        MessageOutput.printf("%02x", *(reinterpret_cast<uint8_t*>(&val) + 3 - i));
    }
    double output = static_cast<int>(val * 100 + 0.5) / 100.0;
    MessageOutput.printf(", output(%.20f)(%.02f)(%f)=0x", output, output, output);
    for(int i = 0; i < 8; ++i) {
        MessageOutput.printf("%02x", *(reinterpret_cast<uint8_t*>(&output) + 7 - i));
    }
    double casted = static_cast<double>(val);
    MessageOutput.printf(", static_cast(%.20f)(%.02f)(%f)=0x", casted, casted, casted);
    for(int i = 0; i < 8; ++i) {
        MessageOutput.printf("%02x", *(reinterpret_cast<uint8_t*>(&casted) + 7 - i));
    }
    MessageOutput.printf("\r\n");
    return static_cast<int>(val * 100 + 0.5) / 100.0;
};
roundedFloat(50.13);
roundedFloat(55.04);
roundedFloat(123.4);
roundedFloat(123.4567891234);

Output is (adjusted for alignment):

input (50.13000106811523437500) (50.13) (50.130001)=0x4248851f, output (50.13000000000000255795) (50.13) (50.130000)=0x404910a3d70a3d71, static_cast(50.13000106811523437500) (50.13)  (50.130001)=0x404910a3e0000000
input (55.04000091552734375000) (55.04) (55.040001)=0x425c28f6, output (55.03999999999999914735) (55.04) (55.040000)=0x404b851eb851eb85, static_cast(55.04000091552734375000) (55.04)  (55.040001)=0x404b851ec0000000
input(123.40000152587890625000)(123.40)(123.400002)=0x42f6cccd, output(123.40000000000000568434)(123.40)(123.400000)=0x405ed9999999999a, static_cast(123.40000152587890625000)(123.40)(123.400002)=0x405ed999a0000000
input(123.45678710937500000000)(123.46)(123.456787)=0x42f6e9e0, output(123.45999999999999374722)(123.46)(123.460000)=0x405edd70a3d70a3d, static_cast(123.45678710937500000000)(123.46)(123.456787)=0x405edd3c00000000

To fiddle with the bits: https://www.h-schmidt.net/FloatConverter/IEEE754.html and https://evanw.github.io/float-toy/ These online-tools seem to know that the float value 0x4248851f is supposed to read 51.3, not 50.130001068115234375. Why?

I would like to transform the floats to strings using an explicit format like %.02f, but then the JSON carries a string, not a number, so that doesn't work.

So, ChatGPT tells me that the inherent decimal precision of a single-precision float is 6 to 7 decimal digits. People on stackoverflow agree.

My best guess is this: The function transforming the float into a string representation for ArduinoJson is overestimating the decimal precision of the float type, and it should have rounded after 7 decimal digits, which would yield the expected values in our cases above. I think we can see that clearly in the examples with 123.x: the default format specifier %f spits out six digits after the decimal point and three before, which would be 9 decimal digits of precision, which the float simply cannot provide.

A similar program "fails" the same way on my host computer (clang, C++14). However, std::cout << static_cast<float>(50.13); does indeed print 50.13 😲 This has to mean that std::operator<<(float) is smart about the actual precision of the decimal, whereas %f is not.

So, I found the reason: As per the manual, %f prints 6 digits after the dot. Period. Makes no sense. I guess it was too hard to standardize this back in the day in a way that would work as expected.

Except that %g does exactly that:

🤯 🤯 🤯

%g is working as expected, on my host and on the ESP32. I wonder when it switches to %e style, which might be cumbersome in its own way, but for the ranges of values we are handling, %g is probably always the better choice.

[schlimmchen]

This is still not it...

This is what 50.13 is serialized into with ArduinoJson 7.1.0

{"voltagefloat":50.13000107,"voltagedouble":50.13}

using this code

    JsonDocument doc;

    float voltagefloat = 50.13;
    double voltagedouble = 50.13;

    doc["voltagefloat"] = voltagefloat;
    doc["voltagedouble"] = voltagedouble;

    serializeJson(doc, Serial);
    Serial.println();

Notice there are even 8 digits after the dot and 10 significant decimal digits in total. So this is not about the usage of printf with the %f specifier. Yet the docs suggest that this should be handled as I expect.

I was about to open an issue against ArduinoJson, but the issue template told me to use the ArduinoJson troubleshooter. So I did, eyes rolling:

So we just have to accept this. I am not happy about that. static_cast<double>(...) does not solve the issue. Maybe the string to float algorithm used in ArduinoJson treats all floating point numbers als doubles, so it assumes the decimal precision to be as high as it is. It might simply not be designed to handle float.

Well, at least I now know that there is no way around this unless I would question the use of ArduinoJson or the use of actual numbers in the JSON (rather than pre-formatted strings).

[spcqike]

Well, how about not using float but int and just re-interpret it back to float wherever needed?

That’s what I do wherever I want to handle a larger number of values to minimize ram usage, as a float uses 4byte and an unit16_t uses 2byte only. I store my voltages (or whatever) as mV in an uint16_t (which represents up to 65535 mV)

[AndreasBoehm]

This issue is pretty wild. For me it also doesn't make much sense what the suggested code from ArduinoJson does..

Should use double instead of float right away? I know that its twice the size but i am wondering if there is any other way to solve that besides using static_cast.

I think that the suggestion from @spcqike is also a really good one.

[schlimmchen]

Thanks for your interest in this, guys!

Should use double instead of float right away? I know that its twice the size but i am wondering if there is any other way to solve that besides using static_cast.

static_cast does not solve it.

There is an actual difference in size between float and double on ESP32 (not on every CPU that is the case), which is less concerning. I am opposed of using double since double is only handled in software, whereas the FPU on ESP32 does handle float.

I can't proof it, and it certainly depends on whether or not your data structure is packed or not, but as the ESP32 is a 32bit CPU, I don't think you actually save 2 Bytes of RAM/stack when you use uint16_t over uint32_t, as alignment (to the CPUs bus width) is a thing. Example: If there is a bool on your stack which occupied only one Byte of RAM, and on top of it is a uint32_t which is not aligned to a 4-Byte boundary because of that bool, the CPU might have difficulty fetching the uint32_t or may need to fetch it in pieces due to misalignment. That might be a false assumption/memory, but AFAIK variables are aligned according to the CPU's bus width for this reason. The performance when calculating with such types afterwards should not be impacted as the CPU crunches a 16bit integer the same way it would crunch a 32bit integer.

Regarding the handling of floats at all: I totally agree and I would like to specify (or at least save) the voltage thresholds as uint32_t and have it be in millivolts (mV) rather than Volts (V). If we did that: Shall the setting in the web UI also be mV? If not, we need to do translation when receiving the settings and when sending them off. Nasty. Also: How do we handle upgrades? Deserialize into uint32_t (I guess that works, at least when using .as<uint32_t>(), then check if the voltage is less than... 1000? Then deserialize again as float, multiply by 1000, ensure proper rounding, then write that to the config struct? Also nasty. I suggest we live with the special handling when serializing the float to JSON, at least until we get more problems that would vanish if we used integers.

[skippermeister]

@schlimmchen
Float und auch Double Zahlen können nicht exakt jede Zahl mit Dezimalstellen darstellen. Das liegt in der Natur der binären Zahlendarstellung mit Mantisse und Exponent. Das ist unter anderem der Grund warum in der Finanzindustrie für Konten keine Gleitkommazahlen (float oder double) genommen werden dürfen. Auch wenn die Darstellung z.B. mit printf als %.2f auf 2 Stellen gerundet wird, ist die interne Darstellung der Gleitkommazahl selten exakt das was visualisiert wird.
Die ArduinoJson Library rundet nicht, sondert wandelt die intern Darstellung der Gleitkommazahl in einen ASCII Zahlenstring. Das ist auch kein Bug, sondern works as designed. In dem obigen Beispiel wird 50.13 als float intern auf die nächst mögliche Zahlendarstellung als 32 Bit float abgebildet. Double hat eine höhere Genauigkeit, aber auch mit Double ist das Problem nicht zu lösen, es verschiebt sich nur. Und wir würden mehr kostbaren RAM Speicher verschwenden.

Eventuell sollten wir uns überlegen intern komplett auf Ganzzahlen umzustellen (riesige Aufgabe). Die Rohdaten aus den Geräten sind eh nur mit 10 oder 100 oder 1000 multiplizierte Ganzzahlen. Und unsere Umrechnung auf float bringt keine Genauigkeit, sondern nur eine Vereinfachung bei Multiplikationen und Divisionen.

Ich habe mal eine Ladegerät für LiIon Akkus entwickelt. Das arbeitet nur mit Ganzzahlen.