Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

encoding/json: JSON number serialization differs from ES6/V8 #14135

Closed
cyberphone opened this issue Jan 28, 2016 · 12 comments

Comments

Projects
None yet
8 participants
@cyberphone
Copy link

commented Jan 28, 2016

The following program (written by a complete go n00b)

package main

import "fmt"
import "encoding/json"

func main() {
    mapD := map[string]float64{"apple": 5e-6, "lettuce": 0.0}
    mapB, _ := json.Marshal(mapD)
    fmt.Println(string(mapB))
}

returns

{"apple":5e-06,"lettuce":0}

it should rather return

{"apple":0.000005,"lettuce":0}

Rationale for this request:
microsoft/ChakraCore#149

@ianlancetaylor ianlancetaylor changed the title JSON number serialization differs from ES6/V8 encoding/json: JSON number serialization differs from ES6/V8 Jan 28, 2016

@ianlancetaylor ianlancetaylor added this to the Go1.7 milestone Jan 28, 2016

@musgravejw

This comment has been minimized.

Copy link

commented Jan 29, 2016

@gopherbot

This comment has been minimized.

Copy link

commented Jan 29, 2016

CL https://golang.org/cl/19092 mentions this issue.

@rsc

This comment has been minimized.

Copy link
Contributor

commented Jan 29, 2016

I spent a little while looking at this. CL 19092 suggests just changing the format to %f, but that doesn't match at least Chrome's console.log either. It matches for 5e-6, but not for larger or smaller numbers.

By manual binary search it appears that Chrome's algorithm is:

abs := math.Abs(x)
if abs == 0 || 1e-6 <= x && x < 1e21 {
   use %f
} else {
   use %e
}

This is kind of a peculiar thing to do, but if all the Javascript engines agree then I don't see significant harm to using the same algorithm for Go. However, a much stronger argument would be if the ECMA-262 spec actually documents and mandates this apparent rule. Does it?

@cyberphone

This comment has been minimized.

Copy link
Author

commented Jan 29, 2016

ECMA-262 does indeed mandates this formatting but unfortunately offers two solutions for the last (unprecise) digit. V8 selected the more advanced ("correct") version.

@rsc

This comment has been minimized.

Copy link
Contributor

commented Jan 29, 2016

Thanks @cyberphone. I have no problem with producing the correct last digit, since Go already does. However, on closer examination it looks like maybe JS doesn't get later digits correct: 999999999999999868928 is an exact float64 value. Go prints it correctly as 999999999999999868928 (%f) or as 9.999999999999999e+20 (%g/%e), while Chrome prints it as 999999999999999900000. Does ECMA-262 mandate this too? Can you point to a page in the PDF that I can read? Thanks.

@cyberphone

This comment has been minimized.

Copy link
Author

commented Jan 29, 2016

@rsc If I understand this correctly 999999999999999868928 is an exact float64 value but according to the authorities on this matter (excludes me...) it is still wrong showing it because the precision is only 17 digits.

7.1.12.1 in http://www.ecma-international.org/ecma-262/6.0/ECMA-262.pdf is the reference.

@rsc

This comment has been minimized.

Copy link
Contributor

commented Jan 29, 2016

@cyberphone

This comment has been minimized.

Copy link
Author

commented Jan 29, 2016

Yes :-) A possibility could be integrating the V8 algorithm code in go "as is" and make it available through a method in "strconv" like FormatFloatES6. This is essentially how I did it in my JSON Java library.

@RalphCorderoy

This comment has been minimized.

Copy link

commented Jan 30, 2016

The issue has been raised on json/encoding. ECMA-262's ToString http://www.ecma-international.org/ecma-262/6.0/index.html#sec-tostring-applied-to-the-number-type describes a behaviour (two to hedge their bets) of Javascript. But it's http://www.ecma-international.org/publications/standards/Ecma-404.htm that covers JSON and that makes clear on page ii that it seeks to get away from a particular programming language's representation and move towards a human representation. Code that depends on 1e42 being represented a particular way in JSON seems broken to me. A library that deliberately discards bits so the conversion can't be reversed, more so.

@cyberphone

This comment has been minimized.

Copy link
Author

commented Jan 30, 2016

@RalphCorderoy The sole purpose of this proposal is creating a foundation for "normalized" JSON objects that can safely traverse through different systems even if they carry a digital signature. ES6 also specifies ordered properties although it is not a part of JSON.

For digitally signed JSON/JavaScript, the following samples illustrate the two principal alternatives (JCS is just an example) that developers have to select from. They should be fully comparable with respect to security since they use the same algorithms (not easy to see since the IETF scheme dresses everything in Base64URL).

JSON Cleartext Signature (JCS): https://cyberphone.github.io/openkeystore/resources/docs/jcs.html#ECMAScript_Compatibility_Mode

var signedObject = {
      // The data
      statement: "Hello signed world!",
      otherProperties: [2000, true],
      // The signature
      signature: {
          algorithm: "ES256",
          publicKey: {
              type: "EC",
              curve: "P-256",
              x: "vlYxD4dtFJOp1_8_QUcieWCW-4KrLMmFL2rpkY1bQDs",
              y: "fxEF70yJenP3SPHM9hv-EnvhG6nXr3_S-fDqoj-F6yM"
          },
          value: "2H__TkcV28QpGWPkyVbR1CW0I8L4xA...CTyYfmAippJXqdzgNAonnFPVCSI5A6novMQ"
      }
};

JSON Web Signature (JWS): https://tools.ietf.org/rfc/rfc7515.txt

var signedObject = {
      payload: "eyJpc3MiOiJqb2UiLA0KICJleHAiOjEz...eGFtcGxlLmNvbS9pc19yb290Ijp0cnVlfQ",
      protected: "eyJhbGciOiJFUzI1NiJ9",
      signature: "DtEhU3ljbEg8L38VWAfUAqOyKA...gfTjDxw5djxLa8IS lSApmWQxfKTUJqPP3-Kg6NU1Q"
};

JCS was created for preserving the human representation of JSON.

@joeshaw

This comment has been minimized.

Copy link
Contributor

commented Feb 25, 2016

Related: #6384

@gopherbot

This comment has been minimized.

Copy link

commented Oct 5, 2016

CL https://golang.org/cl/30371 mentions this issue.

@gopherbot gopherbot closed this in 92b3e36 Oct 5, 2016

@golang golang locked and limited conversation to collaborators Oct 5, 2017

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
You can’t perform that action at this time.