New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

encoding/json: support unmarhaling "1.111111e+06" into int64 #5562

Closed
gopherbot opened this Issue May 27, 2013 · 13 comments

Comments

Projects
None yet
7 participants
@gopherbot

gopherbot commented May 27, 2013

by ben.lubar:

Thanks.

What steps will reproduce the problem?
http://play.golang.org/p/NFwl6Vyh3d

What is the expected output?
true
true

What do you see instead?
true
true
panic: json: cannot unmarshal number 1.111111e+06 into Go value of type int64

Which compiler are you using (5g, 6g, 8g, gccgo)?
irrelevant; library bug (6g)

Which operating system are you using?
irrelevant; library bug (linux/amd64)

Which version are you using?  (run 'go version')
go version devel +ca166884c853 Sat May 25 22:47:36 2013 +0400 linux/amd64
@minux

This comment has been minimized.

Show comment
Hide comment
@minux

minux May 27, 2013

Member

Comment 1:

json doesn't preserve value types, and in fact, all json numbers are treated as floating
point numbers,
so if you marshal a floating point value, don't expect to correctly unmarshal it back as
an integer.

Status changed to WorkingAsIntended.

Member

minux commented May 27, 2013

Comment 1:

json doesn't preserve value types, and in fact, all json numbers are treated as floating
point numbers,
so if you marshal a floating point value, don't expect to correctly unmarshal it back as
an integer.

Status changed to WorkingAsIntended.

@gopherbot

This comment has been minimized.

Show comment
Hide comment
@gopherbot

gopherbot May 28, 2013

Comment 2 by ben.lubar:

http://play.golang.org/p/OAeg4VrwUw
If all json values are floating-point, it is a bug to handle some values two different
ways depending on encoding. For example, if I have "foo" in some JSON, I don't want it
being handled differently than "\u0066\u006f\u006f". Decoding either of the equivalent
values into any type should yield the same result.

gopherbot commented May 28, 2013

Comment 2 by ben.lubar:

http://play.golang.org/p/OAeg4VrwUw
If all json values are floating-point, it is a bug to handle some values two different
ways depending on encoding. For example, if I have "foo" in some JSON, I don't want it
being handled differently than "\u0066\u006f\u006f". Decoding either of the equivalent
values into any type should yield the same result.
@minux

This comment has been minimized.

Show comment
Hide comment
@minux

minux May 28, 2013

Member

Comment 3:

because json only supports floating point, the default number type for json is float64.
as a special case, if encoding/json marshals a integer, encoding/json could latter
unmarshal
the result into an integer.
Member

minux commented May 28, 2013

Comment 3:

because json only supports floating point, the default number type for json is float64.
as a special case, if encoding/json marshals a integer, encoding/json could latter
unmarshal
the result into an integer.
@remyoudompheng

This comment has been minimized.

Show comment
Hide comment
@remyoudompheng

remyoudompheng May 28, 2013

Contributor

Comment 4:

it may round results or modify values if they are not representable (like type
conversion), but I think it should really not be an error.
Contributor

remyoudompheng commented May 28, 2013

Comment 4:

it may round results or modify values if they are not representable (like type
conversion), but I think it should really not be an error.
@minux

This comment has been minimized.

Show comment
Hide comment
@minux

minux May 28, 2013

Member

Comment 5:

ok.
what should we do if we ask encoding/json to unmarshal "1.111111e+05" into an int64?
error or truncate/round?
ps: encoding/xml is consistent on this matter with encoding/json.

Status changed to Thinking.

Member

minux commented May 28, 2013

Comment 5:

ok.
what should we do if we ask encoding/json to unmarshal "1.111111e+05" into an int64?
error or truncate/round?
ps: encoding/xml is consistent on this matter with encoding/json.

Status changed to Thinking.

@rsc

This comment has been minimized.

Show comment
Hide comment
@rsc

rsc Jun 3, 2013

Contributor

Comment 6:

I think it's fine the way it is. If you have a number that looks like 1.11e5 and you are
trying to stuff it into an int64, you've probably screwed up somewhere, and it is
reasonable for package json to tell you.
Contributor

rsc commented Jun 3, 2013

Comment 6:

I think it's fine the way it is. If you have a number that looks like 1.11e5 and you are
trying to stuff it into an int64, you've probably screwed up somewhere, and it is
reasonable for package json to tell you.
@rsc

This comment has been minimized.

Show comment
Hide comment
@rsc

rsc Jul 12, 2013

Contributor

Comment 7:

After a month of thought, I still believe this is working as intended.

Status changed to WorkingAsIntended.

Contributor

rsc commented Jul 12, 2013

Comment 7:

After a month of thought, I still believe this is working as intended.

Status changed to WorkingAsIntended.

@rsc

This comment has been minimized.

Show comment
Hide comment
@rsc

rsc Oct 29, 2013

Contributor

Comment 8:

Issue #6657 has been merged into this issue.

Contributor

rsc commented Oct 29, 2013

Comment 8:

Issue #6657 has been merged into this issue.

@gopherbot

This comment has been minimized.

Show comment
Hide comment
@gopherbot

gopherbot Aug 15, 2014

Comment 9 by voidlogic7:

I think the current position on this is unreasonable. If the callers struct indicates an
integer value and:
var valFromJSON float64
if valFromJSON == float64(int(valFromJSON)) { ...
The scientific number should be accepted as the integer value requested, if not the
current error should apply.
Even if it is known that the value will be an integer, a Go system may receive data from
system that will use scientific notion whenever the value is smaller in that notation
(to minimize JSON size). Ex: "2000000" (7 bytes) vs "2e+6" (4 bytes).

gopherbot commented Aug 15, 2014

Comment 9 by voidlogic7:

I think the current position on this is unreasonable. If the callers struct indicates an
integer value and:
var valFromJSON float64
if valFromJSON == float64(int(valFromJSON)) { ...
The scientific number should be accepted as the integer value requested, if not the
current error should apply.
Even if it is known that the value will be an integer, a Go system may receive data from
system that will use scientific notion whenever the value is smaller in that notation
(to minimize JSON size). Ex: "2000000" (7 bytes) vs "2e+6" (4 bytes).
@rsc

This comment has been minimized.

Show comment
Hide comment
@rsc

rsc Sep 15, 2014

Contributor

Comment 10:

Issue #8460 has been merged into this issue.

Contributor

rsc commented Sep 15, 2014

Comment 10:

Issue #8460 has been merged into this issue.

@orian

This comment has been minimized.

Show comment
Hide comment
@orian

orian Feb 22, 2015

Just for all future Go users:
http://play.golang.org/p/-gyDtk5UG8

If one applies Unmarshal + Marshal, the produced JSON message will be have different representation of numbers than original message. IMO it's a bug (I acknowledge "Working-as-intended" status).

orian commented Feb 22, 2015

Just for all future Go users:
http://play.golang.org/p/-gyDtk5UG8

If one applies Unmarshal + Marshal, the produced JSON message will be have different representation of numbers than original message. IMO it's a bug (I acknowledge "Working-as-intended" status).

@rogpeppe

This comment has been minimized.

Show comment
Hide comment
@rogpeppe

rogpeppe Feb 22, 2015

Contributor

FWIW I think this is unfortunate. JSON numbers are floats, and I think we should at least try to convert them to integers if we're unmarshaling into an integer type. If the JSON is produced by some other language, it may very well encode larger integers in exponential notation in the expectation that they can be round-tripped back into an integer. Luckily the usual Javascript JSON stringify function does not produce e-notation for integers in the int32 range, otherwise this would be a much bigger issue than it is.

If it were up to me, I'd treat unmarshaling into an integer type as the same as converting a float64 to an integer type, int64(f).

As an mitigating change, perhaps we could change %g to avoid e-notation for numbers in the range -0x80000000 to 0x7fffffff.

@orian You can use UseNumber to round trip JSON without changes like that. http://play.golang.org/p/YcpPyC6v3s

Contributor

rogpeppe commented Feb 22, 2015

FWIW I think this is unfortunate. JSON numbers are floats, and I think we should at least try to convert them to integers if we're unmarshaling into an integer type. If the JSON is produced by some other language, it may very well encode larger integers in exponential notation in the expectation that they can be round-tripped back into an integer. Luckily the usual Javascript JSON stringify function does not produce e-notation for integers in the int32 range, otherwise this would be a much bigger issue than it is.

If it were up to me, I'd treat unmarshaling into an integer type as the same as converting a float64 to an integer type, int64(f).

As an mitigating change, perhaps we could change %g to avoid e-notation for numbers in the range -0x80000000 to 0x7fffffff.

@orian You can use UseNumber to round trip JSON without changes like that. http://play.golang.org/p/YcpPyC6v3s

@felixge

This comment has been minimized.

Show comment
Hide comment
@felixge

felixge Sep 23, 2015

@rogpeppe 👍 . Given that 1e6 === 1000000 in JS/JSON, it seems very unreasonable to handle them different in json.Unmarshal.

felixge commented Sep 23, 2015

@rogpeppe 👍 . Given that 1e6 === 1000000 in JS/JSON, it seems very unreasonable to handle them different in json.Unmarshal.

This issue was closed.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.