Very large number gets reformated as invalid JSON

[j-brooke]

As revealed in this issue, when attempting to justify a number list, FracturedJson will sometimes write very large numbers as "Infinity", without quotes, resulting in invalid JSON output.

The numbers for which this happens are ones that overflow a 64-bit float's range - for example, 1e500. This happens when FJ tries to reformat numbers to align them, such as in a compact multiline array or table case.

For instance, start this input:

[
  1e500, 1e-500, 4.0
]

Now reformat, using the settings, DontJustifyNumbers=false, MaxInlineComplexity=0. The result is this:

[
    Infinity,        0,        4
]

That's not valid JSON, which is a pretty bad mistake for a JSON formatter.

The following input creates similar problems:

[
    { "a": 1e500 },
    { "a": 1e-500 },
    { "a": 4.0 }
]

It gets reformatted as:

[
    {"a": Infinity},
    {"a":        0},
    {"a":        4}
]

This problem exists in the C# and JavaScript libraries, as well as the web-base formatter and the Visual Studio Code extension.

[josdejong]

Thanks for opening this issue. Maybe good to add dealing with a value having more than 16 digits, like 9123372036854000123, that will lose the last few digits (at least in JavaScript).

I've had similar issues in my web application and implemented lossless-json to solve this (and there are quite a few similar BigNumber or BigInt libraries).

Some ideas for a solution:

1. Implement a function that keeps the numeric values as a string and still is able to format them. I think it is only appending trailing zeros and a dot when needed, if that is indeed the case it sounds like a nice approach to me.
2. Implement a hook to allow you to format/stringify BigNumber values yourself, similar to say the reviver and reducer arguments of JavaScript's JSON.parse and JSON.stringify that allow you to deal with custom classes for example.

[j-brooke]

Thanks @josdejong . I appreciate the suggestions. There's already some logic that decides whether any given column of stuff contains reformattable numbers. ( Here's the relevant section in C# ). The existing logic gives up trying to reformat and justify the list if any of the numbers have too many digits or if the regular ToString() form is in scientific notation. It didn't occur to me to check for Infinity, but that's easy enough to add.

The underflow case is trickier. If I have a string like 0, 0.0, or 0.00e35, that shouldn't disqualify reformatting into 0.0000 (or whatever precision is needed). But 1e-500 should ideally be recognized as not reformattable. Right now the best idea I can come up with is running the text through a regular expression when the parsed value is zero, but I had to add the extra overhead. Long lists of zeros are probably a common situation in JSON. (And unlike the infinity case, allowing the conversion from 1e-500 to 0 doesn't break the document.)

[josdejong]

Ah, thanks for the pointer. So, would you like to alter the exponent like e35 too, or leave the exponent as is? In the latter case, you can split the value in two parts and format the first part without the exponent.

[j-brooke]

Fixed in versions 3.1.1 of the .NET and JS libraries. Fixed in version 3.1.2 of the VS Code extension.

[josdejong]

Thanks!