dom and OnDemand resolve double types to different results

[DenineLu]

When I parse 0.8825149536132812, dom gets an error in the result, Did I write something wrong? Thanks

code:

padded_string paddedInputJson = R"({"score":0.8825149536132812})"_padded;
simdjson::dom::parser parser_dom;
simdjson::dom::element ele;
ele = parser_dom.parse(paddedInputJson);
std::cout << "     dom:" << ele["score"] << std::endl;

simdjson::ondemand::parser parser_ondemand;
simdjson::ondemand::value val;
val = parser_ondemand.iterate(paddedInputJson);
std::cout << "ondemand:" << val["score"] << std::endl;

result:

     dom:0.8825149536132813
ondemand:0.8825149536132812

[DenineLu]

DOM seems to retain only 16 decimal places, and the 16th place is no longer exact

[lemire]

In binary64, the decimal string 0.8825149536132812 is represented as 115673 * 2**(-17) which is 0.88251495361328125.

You therefore have that 0.8825149536132812 == 0.88251495361328125 == 0.8825149536132813.

[lemire]

Recommended reading:

• Daniel Lemire, Number Parsing at a Gigabyte per Second, Software: Practice and Experience 51 (8), 2021

The number parser in simdjson is state-of-the-art. It has been widely adopted: it is part of the Rust, Go, C# standard libraries. It is part of the GCC C++ standard library, and so forth.

We always welcome bug reports, but you should assume that we are correct: we have not yet encountered a single bug regarding our number parsing.

[lemire]

From the paper above...

You cannot expect all 16-digit numbers to be represented with binary64. You only have 15 digits of accuracy. There is no way in the current C++ standard to have more precision than that, short of rolling your own formats.

[lemire]

Note that this has nothing to do with simdjson. It is just standard C++.

[DenineLu]

In binary64, the decimal string 0.8825149536132812 is represented as 115673 * 2**(-17) which is 0.88251495361328125.

You therefore have that 0.8825149536132812 == 0.88251495361328125 == 0.8825149536132813.

Thank you very much for your answer, now I understand the reason for this! But since my usage scenario needs to get unresolved values, I need to go for the ondemand method.

[lemire]

@DenineLu

Both DOM and ondemand give the same result. It is not possible that you get the double value 0.8825149536132812 in one case and a distinct double value 0.8825149536132813 in the second case.

There are two good reasons for that:

1. The two values are the same. There is no 0.8825149536132812 distinct from 0.8825149536132813 in C++.
2. The number parser is the same. Both DOM and ondemand use the same number parsing functions.

Your code example does not do what you think it does. In one instance (ondemand), you are just printing out the content of the JSON string, and in the other (DOM), you are re-serializing a parsed number.

[lemire]

Your title was... "dom and OnDemand resolve double types to different results"

But it is not what your code does at all...

Try the following code...

  padded_string paddedInputJson = R"({"score":0.8825149536132812})"_padded;
  simdjson::dom::parser parser_dom;
  simdjson::dom::element ele;
  ele = parser_dom.parse(paddedInputJson);
  double score_dom = ele["score"];
  std::cout << "     dom:" << score_dom << std::endl;

  simdjson::ondemand::parser parser_ondemand;
  simdjson::ondemand::value val;
  val = parser_ondemand.iterate(paddedInputJson);
  double  score_ondemand = val["score"];
  std::cout << "ondemand:" << score_ondemand << std::endl;

[DenineLu]

Your title was... "dom and OnDemand resolve double types to different results"

But it is not what your code does at all...

Try the following code...

  padded_string paddedInputJson = R"({"score":0.8825149536132812})"_padded;
  simdjson::dom::parser parser_dom;
  simdjson::dom::element ele;
  ele = parser_dom.parse(paddedInputJson);
  double score_dom = ele["score"];
  std::cout << "     dom:" << score_dom << std::endl;

  simdjson::ondemand::parser parser_ondemand;
  simdjson::ondemand::value val;
  val = parser_ondemand.iterate(paddedInputJson);
  double  score_ondemand = val["score"];
  std::cout << "ondemand:" << score_ondemand << std::endl;

Thank you very much for your guidance, I understand my mistake, thanks!