Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

encoding/json: handle NaN and Inf #3480

Closed
FlorianUekermann opened this issue Apr 5, 2012 · 12 comments
Closed

encoding/json: handle NaN and Inf #3480

FlorianUekermann opened this issue Apr 5, 2012 · 12 comments

Comments

@FlorianUekermann
Copy link
Contributor

What steps will reproduce the problem?
1. Run http://play.golang.org/p/FN7C1ZUqpw
    js, err := json.Marshal(math.NaN())
    fmt.Print("JSON: \"", string(js), "\" Error: \"",err,"\"")

What is the expected output?
JSON: "null" Error: "nil"

What do you see instead?
JSON: "" Error: "json: unsupported value: NaN"

Which version are you using?
go1

This is a bit weird. I would't mind the error since it might be a reasonable idea to
notify the user that the encoding didn't preserve, although I would prefer no error at
all.

The RFC draft describing JSON (http://tools.ietf.org/html/rfc4627) explicitly forbids
the use of NaN and (-)Inf as numbers, so does the ECMAScript Specification (regardless
of version) but  pretty clear about the handling such values.
Page 220 NOTE 4 of the ECMA-262 ECMAScript Language Specification 5.1 edition (
http://goo.gl/gY9PK ) contains the following sentence:

"""
NOTE 4
...
NaN and Infinity regardless of sign are
represented as the String null.
"""




On a more general note:

The fact that the json package doesn't always return valid JSON that is complete with
respect to the values that are representable always was mystery to me. Especially since
the same page of the ECMAScript specification explicitly explains how this should be
handled:

"""
NOTE 5
Values that do not have a JSON representation (such as undefined and functions) do not
produce a String.
Instead they produce the undefined value. In arrays these values are represented as the
String null. In objects an
unrepresentable value causes the property to be excluded from stringification.
"""

To handle these cases properly without breaking existing code I propose changing the
behavior of json.Marshal in general. It shouldn't stop if it encounters an
unrepresentable value, but simply produce null and emit the errors anyway.

(The case of functions as values of interface variables/fields probably deserves some
discussion, since it would be a bit awkward in a statically typed language to decide if
a field should be left out depending on it's value.)
@FlorianUekermann
Copy link
Contributor Author

Comment 1:

Sorry one of the first sentences is missing a few words, it should read:
The RFC draft describing JSON (http://tools.ietf.org/html/rfc4627) explicitly forbids
the use of NaN and (-)Inf as numbers, so does the ECMAScript Specification (regardless
of version) but _it is_ pretty clear about the handling _of_ such values.

@FlorianUekermann
Copy link
Contributor Author

Comment 2:

Sorry the first two paragraphs are both missing a few words, they should read:
This is a bit weird. I would't mind the error since it might be a reasonable idea to
notify the user that the encoding didn't preserve _all data_, although I would prefer no
error at all.
The RFC draft describing JSON (http://tools.ietf.org/html/rfc4627) explicitly forbids
the use of NaN and (-)Inf as numbers, so does the ECMAScript Specification (regardless
of version) but _it is_ pretty clear about the handling _of_ such values.

@rsc
Copy link
Contributor

rsc commented Sep 12, 2012

Comment 3:

Labels changed: added priority-later, removed priority-triage.

Status changed to Accepted.

@rsc
Copy link
Contributor

rsc commented Sep 12, 2012

Comment 4:

Labels changed: added go1.1maybe.

@gopherbot
Copy link
Contributor

Comment 6 by julius.volz:

One workaround for now is to custom-type your floats and implement the json.Marshaler
interface (json.http://golang.org/pkg/encoding/json/#Marshaler) for that type. E.g.:
==============================================
type jsonFloat float32
func (value jsonFloat) MarshalJSON([]byte, error) {
  ... // return your own float representation here.
}
==============================================
This also works for floats that are parts of structs, etc. But I agree it would be nice
to get this fixed in Go itself.

@robpike
Copy link
Contributor

robpike commented Mar 7, 2013

Comment 7:

Labels changed: removed go1.1maybe.

@rsc
Copy link
Contributor

rsc commented Jul 12, 2013

Comment 8:

Suppose Marshal turns NaN, +Inf, and -Inf into JSON `null`.
What does Unmarshal do?

@gopherbot
Copy link
Contributor

Comment 9 by julius.volz:

Good question. I can think of two options:
a) unmarshalling any "null" back into NaN seems at least somewhat sensible
b) don't support unmarshalling "null" back to a float at all, only support marshalling
(return an error when encountering a "null" value for a float field)
But I'm not sure I can appropriately judge the implications of either option.

@FlorianUekermann
Copy link
Contributor Author

Comment 10:

In that case it should decode to NaN to keep things symmetric and because we should try
to loose as little information as possible. The current implementation maps null to 0
for any number type, which means there is no way to tell "0" and "null" apart in the
result, which isn't perfect anyway. Maybe decoding 'null' should fail for ints.
We could of course also use strings for encoding the three special cases. And just
refuse to decode 'null' for numbers, if we can live with differing a little bit from the
ECMAScript behavior on the encoding side (decoding doesn't happen in ECMAScript anyway,
they basically just evaluate JSON strings).

@rsc
Copy link
Contributor

rsc commented Jul 15, 2013

Comment 11:

You can tell "0" and "null" apart in the input by using *float64 instead of
float64. This is consistent with int64.
Also, "null" does not turn into 0. It makes the field not be written to. It
keeps its previous value.
Russ

@FlorianUekermann
Copy link
Contributor Author

Comment 12:

Oh god, you're right of course. I'm very sorry about the misinformation, stupid of me. I
forgot to try with differently initialized variables.  I also forgot about the more
general rules regarding null when unmarshalling.
Encoding any float value to null would be weirdly asymmetric with the current decoding
behavior I guess. I am not sure this is a good idea anymore. It's hard to match
ecmascript behavior without major changes or being inconsistent about null.

@rsc
Copy link
Contributor

rsc commented Jul 30, 2013

Comment 13:

Status changed to Unfortunate.

Dieterbe added a commit to grafana/metrictank that referenced this issue Dec 3, 2015
otherwise we would see errors like `json: error calling MarshalJSON for
type main.Point: invalid character 'N' looking for beginning of value`

the data can contain NaN values, i verified they were not coming from
or stored in the chunks, but happened in divide(), like when we did 0/0.
Graphite-web has some special cases in divideSeries() but we don't need to, we can just
use NaN in the data arrays and execute our logic/math as usual,
this works fine in Go, we just need to encode them properly.

for the record, in the official json encoder, NaN/null isn't even
supported:
golang/go#3480
@golang golang locked and limited conversation to collaborators Jun 24, 2016
This issue was closed.
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

4 participants