Proposal to remove Infinity and NaN

[jordanbtucker]

I might get a lot of flack for this, but I don't think Infinity and NaN belong in JSON5. Let me address my reasons and try to preemptively address some concerns.

Motivation

Number Implementation Agnosticism

JSON had a good reason for not including Infinity and NaN. It took an implementation agnostic approach to numbers as stated in the official spec (PDF) on page ii:

JSON is agnostic about numbers. In any programming language, there can be a variety of number types of various capacities and complements, fixed or floating, binary or decimal. That can make interchange between different programming languages difficult. JSON instead offers only the representation of numbers that humans use: a sequence of digits. All programming languages know how to make sense of digit sequences even if they disagree on internal representations. That is enough to allow interchange.

I agree with JSON on this matter to some extent, and by adding Infinity and NaN, JSON5 becomes more dogmatic and expects all implementations to comply with IEEE-754 floating point values.

Granted, IEEE-754 is almost ubiquitous, and so IEEE-754 is usually the best representation of JSON numbers, as mentioned in RFC 7159.

Since software that implements IEEE 754 … numbers is generally available and widely used, good interoperability can be achieved by implementations that expect no more precision or range than these provide.

Minimal benefit

If you can build and demonstrate a use case for serializing Infinity, then you've probably invalidated my proposal. It seems like Infinity was added just because it's IEEE-754.

It may be easier to build a use case for NaN since it represents an invalid numeric value and so brings with it type information. But JSON is typically produced and consumed with some form of schema in place, and if the schema expects a number and gets a null, would NaN really add any benefit?

Compatibility with JSON

Every feature of JSON5 is compatible with JSON (in the sense that all values represented by JSON5 can be serialized as JSON without losing significant meaning) with the exception of Infinity and NaN.

{ unquoted:  "and you can quote me on that"} // JSON5
{"unquoted": "and you can quote me on that"} // JSON

{ ùńîċõďë:  "¡ celebridad internacional !"}                                  // JSON5
{\u00f9\u0144\u00ee\u010b\u00f5\u010f\u00eb: "¡ celebridad internacional !"} // JSON5
{"ùńîċõďë": "¡ celebridad internacional !"}                                  // JSON

{"single quotes": 'I can use "double quotes" here'}   // JSON5
{"single quotes": "I can use \"double quotes\" here"} // JSON

{"line breaks": "Look, Mom!\
No \\n's!"}                              // JSON5
{"line breaks": "Look, Mom!\nNo \\n's!"} // JSON

{"character escapes": "No \x65\x73\x63\x61\x70\x65 from reality"} // JSON5
{"character escapes": "No escape from reality"}                   // JSON

{"hexadecimal": 0xdecaf} // JSON5
{"hexadecimal": 912559}  // JSON

{"leading decimal point":  .8675309, "and trailing": 8675309. } // JSON5
{"leading decimal point": 0.8675309, "and trailing": 8675309.0} // JSON

{"positive sign": +1} // JSON5
{"positive sign":  1} // JSON

{"trailing comma": "in objects", "and in": ["arrays",],} // JSON5
{"trailing comma": "in objects", "and in": ["arrays" ] } // JSON

{"NaN": NaN,     "Infinity": Infinity}  // JSON5
{"NaN": "null?", "Infinity": "lolwut?"} // JSON

// Comments were never meant to represent data in JSON5, so they don't need an analog in JSON.

The reason I feel that JSON5 needs to maintain backward compatible serialization with JSON is because I think JSON5 should be to JSON what Babel and CoffeeScript are to JavaScript—syntactic sugar. In other words, JSON5 transpiles to JSON.

JSON is a data interchange format, but many products are using it like a configuration file format, and those files often need to be modified by hand. JSON5 alleviates the pain points and fills in the gaps while maintaining compatibility with ES5.

If everyone (or just @aseemk) disagrees with me about whether JSON5 should just be a syntactically sweet version of JSON, then I will concede and continue to maintain this project with support for Infinity and NaN and maybe include native support for Date.

Concerns

What about existing JSON5 files?

v1.0 of the spec would still include support for Infinity and NaN, however they would be deprecated and removed from a future version.

What would this mean for Date and RegExp?

Date and RegExp would never be natively supported by JSON5, however they would still get serialized as strings in stringify.

If JSON5 is just syntactically sweetened JSON, then why not just use CSON?

I really like CSON, and it's especially great for projects written in CoffeeScript, but it's not so great for projects written in JavaScript.

To demonstrate, say you're writing an application that uses Highcharts, a JavaScript based charting library, and you have a chart like this one. Here's what the configuration looks like:

$(function () {
    $('#container').highcharts({
        title: {
            text: 'Monthly Average Temperature',
            x: -20 //center
        },
        subtitle: {
            text: 'Source: WorldClimate.com',
            x: -20
        },
        xAxis: {
            categories: ['Jan', 'Feb', 'Mar', 'Apr', 'May', 'Jun',
                'Jul', 'Aug', 'Sep', 'Oct', 'Nov', 'Dec']
        },
        yAxis: {
            title: {
                text: 'Temperature (°C)'
            },
            plotLines: [{
                value: 0,
                width: 1,
                color: '#808080'
            }]
        },
        tooltip: {
            valueSuffix: '°C'
        },
        legend: {
            layout: 'vertical',
            align: 'right',
            verticalAlign: 'middle',
            borderWidth: 0
        },
        series: [{
            name: 'Tokyo',
            data: [7.0, 6.9, 9.5, 14.5, 18.2, 21.5, 25.2, 26.5, 23.3, 18.3, 13.9, 9.6]
        }, {
            name: 'New York',
            data: [-0.2, 0.8, 5.7, 11.3, 17.0, 22.0, 24.8, 24.1, 20.1, 14.1, 8.6, 2.5]
        }, {
            name: 'Berlin',
            data: [-0.9, 0.6, 3.5, 8.4, 13.5, 17.0, 18.6, 17.9, 14.3, 9.0, 3.9, 1.0]
        }, {
            name: 'London',
            data: [3.9, 4.2, 5.7, 8.5, 11.9, 15.2, 17.0, 16.6, 14.2, 10.3, 6.6, 4.8]
        }]
    });
});

That looks like great candidate for storing your configuration in a .json5 file. Better yet, if you want to make changes to your configuration and quickly test them out in JSFiddle, you can just copy and paste the JSON5—something you wouldn't be able to do with a CSON file unless you transpiled it to JSON or wrote your project in CoffeeScript.

Discussion

Let me know what you think. All questions, comments, concerns, and kudos are welcome.

[csiz]

I disagree with JSON5 as syntactic sugar for JSON. It means forever keeping compatibility with with whatever design decision were made when JSON was standardised. I for one strongly disagreed with Crockford's decision to leave out NaN and comments, which are the main reasons I chose JSON5.

NaN and slightly less so Infinity are incredibly useful for scientific computing. As a use case for Infinity I can think of serialising boundary conditions. As you said IEEE-754 is basically ubiquitous and so is the choice to represent an invalid number as NaN. Python's numpy and matplotlib nicely handle NaN but fail on None (the equivalent of null). Basically every math or plotting library in every language always uses NaN (instead of null) except JavaScript. And the only reason JavaScript defaults to null is to be compatible with JSON...

I"m not the only one to disagree with the elision of NaN & Infinity, the python standard implementation of JSON serialises and interprets NaN by default. See https://docs.python.org/3/library/json.html#basic-usage. Python has a huge user base so decisions in it's standard don't come lightly.

If allow_nan is false (default: True), then it will be a ValueError to serialize out of range float values (nan, inf, -inf) in strict compliance of the JSON specification. If allow_nan is true, their JavaScript equivalents (NaN, Infinity, -Infinity) will be used.

Date support would also be nice 👍

[aseemk]

@jordanbtucker: I'm sorry for missing your original post! But thanks for fantastic and thoughtful discussion to both you guys!

I think your point about this being the sole exception to "everything in JSON5 can be represented in JSON" / "JSON5 is syntactic sugar" is a great argument. But @csiz's pointer to Python supporting NaN and Infinity by default is powerful too.

I've been hesitant to add Date and RegExp because one of JSON5's principles has been (along the lines of what @jordanbtucker is saying) "no new datatypes". But this discussion rightly points out that adding NaN and Infinity does basically extend the Number datatype.

🤔

[aseemk]

Related: json5/json5#136

[d-frey]

If you are looking for a real-world use-case: I use JSON for logging data. Basically my log-file is one JSON object per line, each object containing structured data. When the data for a log message contains variables from my code, I need to be able to write out any value a variable might hold. For double values, this currently means that I turn non-finite values into strings, but it would be more helpful if I could declare my log file structures as JSON5 and simply keep the type the same (number) and only writing out the correct value instead of switching the type for some values.

[gene-pavlovsky]

@d-frey
+1

[jordanbtucker]

Leaving Infinity and NaN as is for now.

[hax]

Note, number values out of the scope will be parse to ±Infinity as original JSON spec. That means JSON5 do not need support Infinity literal, we can use 1e+99999 to represent +Infinity and use -1e+99999 for -Infinity. It also means we do not need to remove ±Infinity to keep compatibility. The only problem is NaN, especially Number(JSON.parse(JSON.stringify(NaN))) => 0 which is weird.

[geyang]

Support for Infinity and NaN is absolutely critical. Suppose you are using the newest and finest:

• graphQL backend in python, for statistics processing with pandas and numpy
• relay modern on the front end

The data series contains Infinity and NaNs all the time. In order to add custom NaN and Infinity support, I would have to add an extra layer in the serialization and deserialization pipeline. And writing custom JSON scalar SERDES is not trivial.

[skoehler]

I was referred to this issue and I would like to point out a discrepancy in the following text, which apparently is from the official JSON spec:

JSON is agnostic about numbers. In any programming language, there can be a variety of number types of various capacities and complements, fixed or floating, binary or decimal. That can make interchange between different programming languages difficult. JSON instead offers only the representation of numbers that humans use: a sequence of digits. All programming languages know how to make sense of digit sequences even if they disagree on internal representations. That is enough to allow interchange.

The interchange of information between programs (and thus machines) has become a major use-case of JSON. It's nice that it's human readable. But when I'm finished writing my programs, I rarely read or write JSON. As such, it is mind boggling why numbers are converted to base 10, if an exchange of the numbers in base 16 would be much more convenient for machines (notions for hexadecimal floats exist in C99 and Java, and it will take a microcontroller some serious effort to convert a float to base 10). When humans must read JSON, then it can be made pretty by some editor/viewer. Your favorite IDE could show JSON in a form that humans appreciate (this may not only include conversion to base 10 but also pretty printing and such).

Furthermore, human beings are perfectly capable of using infinity. So why does the author of the above text dare to claim that we are not?

Also, what is "agnostic" supposed to mean in this context anyway? As far as I know, agnostic means that, regarding a certain question, somebody does not commit to a point of view. However, JSON clearly forces us to use base 10 and it forbids certain values. It is certainly not number agnostic in the sense that it would allow me to provide my own number parser to deal with numbers in base 13, for example. Actually, the author did the exact opposite of what he claims: he limited JSON to some subset of number formats - maybe in some attempt to keep things "simple".

If JSON seriously was not intended to exchange data between programs/machines, then we shouldn't use it as such. However, if that is a valid use-case, then let me tell your that my programs indeed need Infinity and NaN.

Also, if JSON5 is only intended to offer "syntax sugar" on top of JSON, then I agree that JSON5 shouldn't include Infinity and NaN. However, then I neither need JSON5 nor JSON. Then JSON is only good for making your manager leave your office when he asks what kind of data format you use to send information. However, then we need something other than JSON/JSON5 for our programs.

[jordanbtucker]

@skoehler Thanks for your comments, but this issue was settled in September 2017. I pointed you to this issue to explain why Infinity and NaN are not in the README. However, they are part of the spec and that will likely never change.

[nopdotcom]

The issue of what the spec says may be settled, but what toolmakers do with it isn't. It sounds like there would be a demand for JSON5 decoding that does not permit NaN or Inf±. Certainly if you advertise a drop-in API replacement for JSON parsing, it would be disturbing if the range of output from the replacement function was different from the range of the original.

And you do advertise a drop-in API replacement: "The JSON5 API is compatible with the JSON API." Except that JSON5 can produce values with behavior completely unexpected by client code.

As far as I can tell, this means JSON5 as documented is internally contradictory (that is, you can't be API compatible and support returning NaNs), and as a result, there are no conforming implementations.

[ell1e]

Yes, I think the output values changing is the big problem, also see my reasoning here.

While inf+/inf- may not change the stuff coming in (JSON -> internal data), I'm not so sure it doesn't affect the other direction (internal data->JSON): what do common JSON dump implementations do with infinite values? Don't they just error out, which is kind of what I'd expect? So making this an expected two-way value in JSON5 again seems to potentially mess with what can usually come out of a JSON dumping program.

I think having explicit literals for inf+/inf- also promotes them to more expected values, which then might make it a bigger issue if a parser doesn't do them (e.g. because the programming language it is made for doesn't have infinity as a value).

[jordanbtucker]

@nopdotcom This is getting away from the topic of this thread, but I would like to respond. JSON5 is not advertised as a drop-in replacement for JSON. The intention of keeping the API compatible with JSON is to make it easier for developers to transition to using JSON5. They don't need to learn a brand new API because the method signature is the same. Maybe the wording should be changed to "The JSON5 API is backward compatible with the JSON API" since JSON5 documents are backward compatible with JSON documents. Perhaps more clarifications are needed if developers are interpreting the sentence to mean that the two API's are interchangeable.

Secondly, what determines whether a JSON5 library is a conforming implementation is whether it parses and generates documents that conform to the spec, not whether it conforms to any API. The json5 NPM package is a reference implementation that conforms to the spec. Other implementations are not required to follow the same design choices included in the json5 package.

---

On topic, there is clear support for NaN and Infinity, as indicated in this thread, despite the concerns some of us have, including myself. NaN and Infinity are artifacts that made it into spec. Implementers are welcome to create libraries that do not produce documents containing NaN and Infinity, however I advise them to still parse documents containing those values in order to fully conform to the spec. See #24 for my views on that.

I'm not opposed to adding prose to the spec that discourages the use of NaN and Infinity due to interoperability concerns. In fact, I have several modifications planned for v1.1.0 of the spec.

[ell1e]

JSON5 is not advertised as a drop-in replacement for JSON.

That is however, IMHO, a notable downside. I ended up skipping on it entirely, and going with "JSON with comments" instead even though it is so badly underspecced, pretty much only due to this. I just felt like it would be hard for me to do a good enough implementation in a lang with no NaN, and I couldn't enable it by default either for the JSON module to make it immediately available to a wider range of users due to that output difference, and then I decided on not bothering using it at all. Just something to think about, maybe.