jq mangles floats

[travisbrady]

I often use jq to filter json before piping to other programs and those other programs die when they hit examples like this:

$ echo '{"id" : 1241000000}' | jq '.'
{
  "id": 1.241e+09
}

[wtlangford]

The exponential form for floats is valid JSON. I'd mark this as a bug on
the other programs, personally.

On Mon, Nov 24, 2014 at 5:49 PM, travisbrady notifications@github.com
wrote:

I often use jq to filter json before piping to other programs and those
other programs die when they hit examples like this:

$ echo '{"id" : 1241000000}' | jq '.'
{
"id": 1.241e+09
}

—
Reply to this email directly or view it on GitHub
#627.

[travisbrady]

@wtlangford so you think jq should reformat that? Why not just leave it as is?

@stedolan the reason I noticed this is that I'm also an OCamler and Yojson/Atdgen explodes on this sort of thing.

This just looks incorrect irrespective of any other libraries/tools you might be using:

$ echo '{"ids" : [1240999999,1241000000,1241000001]}' | jq '.'
{
  "ids": [
    1240999999,
    1.241e+09,
    1241000001
  ]
}

[wtlangford]

Well, I'm not sure about whether we should reformat. The issue is due to
how we handle numbers internally. All numbers are casted to floating point
representations, so we can't differentiate between floats and integers at
print time. The inconsistency there is definitely interesting, though, as I
don't know why only one would be rendered in that way. Probably because
that's the smallest form for the number.

Regardless of what we do here, I'd argue that the other programs /should/
accept it.

That being said, I'm not a fan of the exponential form and wouldn't be sad
if it left. @nicowilliams, I think you're more familiar with the printing
section. What say you?

On Tue, Nov 25, 2014, 18:10 travisbrady notifications@github.com wrote:

@wtlangford https://github.com/wtlangford so you think jq should
reformat that? Why not just leave it as is?

@stedolan https://github.com/stedolan the reason I noticed this is that
I'm also an OCamler and Yojson/Atdgen explodes on this sort of thing.

This just looks incorrect irrespective of any other libraries/tools you
might be using:

$ echo '{"ids" : [1240999999,1241000000,1241000001]}' | jq '.'
{
"ids": [
1240999999,
1.241e+09,
1241000001
]
}

—
Reply to this email directly or view it on GitHub
#627 (comment).

[nicowilliams]

It's not that jq reformats numbers.

It's that jq parses them as IEEE754 doubles, then jq formats those on
output. On output there's no information left from parse-time about the
number's original form (assuming it hasn't been transformed by arithmetic
operations).

There's been a lot of talk about fixing this, but it's difficult. There's
plenty of ideas about this, and if we're going to much with this we might
as well also add more precision (bignums). It's something we currently
lack contributor energy for (but feel free to contribute!).

[nicowilliams]

One of the ideas bandied about is to allow for number formatting options to
be specified on the command line, but that seems likely to be
insufficient. Another option would be to use the unused type values we
have to encode a few bits of information about the number's form at parse
time (scientific notation? engineering? no exponent?) so that the
formatter can use it to attempt to recreate the original.

FYI, JSON processors are allowed to do these sorts of transformations.
JSON numbers have no canonical form (heck, neither does JSON in general).

[nicowilliams]

See issues like #218.

[dtolnay]

Closing as a duplicate of #218.