encoding/json: Unmarshal leaves ±Inf in the destination on float overflow in Go 1.27

[IBlackVoid]

Go version

go1.27.0

What did you do

Unmarshal a number that overflows the destination float, then inspect the destination.

package main

import (
	"encoding/json"
	"fmt"
	"math"
)

func main() {
	var f float64
	err := json.Unmarshal([]byte("1e1000"), &f)
	fmt.Printf("err=%v value=%v isInf=%v\n", err != nil, f, math.IsInf(f, 0))

	var v any
	json.Unmarshal([]byte(`{"x":1e1000}`), &v)
	out, err := json.Marshal(v)
	fmt.Printf("re-marshal: %q err=%v\n", out, err)
}

What did you see happen

go1.27.0:

err=true value=+Inf isInf=true
re-marshal: "" err=json: unsupported value: +Inf

go1.27.0 with GOEXPERIMENT=nojsonv2:

err=true value=0 isInf=false
re-marshal: "{\"x\":null}" err=<nil>

Both report the overflow. They differ in what they leave behind: the new implementation stores the clamped ±Inf that strconv.ParseFloat returns alongside ErrRange, the previous one left the destination alone.

What did you expect to see

0, as before, and a value that encoding/json can still marshal.

Unmarshal documents that it decodes as much as it can when it encounters an error, so the residue is observable rather than undefined, and encoding/json/v2 documents this specific value as unreachable (arshal.go):

It fails if it overflows the representation of the Go float type. Since JSON lacks a native representation for a NaN or ±Inf, such values cannot be the result of decoding.

The practical consequence is the second half of the reproducer: Marshal rejects ±Inf, so decoding, logging the error, and re-encoding the partially populated value used to succeed and now fails.

Worth noting before this is treated as a plain oversight: v2/arshal_test.go asserts the current behaviour on purpose (Floats/Float32/Overflow and Floats/Float64/Overflow expect math.Inf(-1)). So the code, its test, and the package documentation do not agree, and I am not sure which of the three is meant to give.

---

Found while comparing pre-1.27 and 1.27 encoding/json behaviour with a differential harness (https://github.com/IBlackVoid/jsonparity).

[prattmic]

cc @dsnet @neild @ChrisHines

[gabyhelp]

Related Issues

• encoding/json/v2: unexpected support for Inf in map[string]any value #74797 (closed)
• encoding/json: Extremely minor variation between Go 1.26 and Go 1.27 around json.RawMessage #81050
• encoding/json: Error reporting difference between jsonv1 in experimental mode #80641 (closed)
• encoding/json: quoted null rejected for string fields with the ,string option in Go 1.27 #81061
• encoding/json: unmarshaling into a function value returns a syntax error #79948 (closed)
• encoding/json: changes in gotip(1.27) without any changes #79770 (closed)
• encoding/json: unmarshal regression for extra data error under goexperiment.jsonv2 #74614 (closed)
• proposal: encoding/json: Unmarshal overflowing numbers into math/big types #24502 (closed)
• encoding/json/v2: missing io.ErrUnexpectedEOF from v1 Decoder.Token #80999 (closed)
• json.Number doesnt supoprt NaN / INF #46872 (closed)

_{(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)}

[dsnet]

Here's a simpler example that demonstrates regressions.

There are two separate (but related) regressions here:

1. When unmarshaling 1e1000 into a concrete float, we now assign +Inf, when it used to avoid mutating the value.
2. When unmarshaling 1e1000 into an empty any interface, we now assign +Inf, when it used to assign nil.

Of note, the v1 docs said,

If a JSON value is not appropriate for a given target type, or if a JSON number overflows the target type, Unmarshal skips that field and completes the unmarshaling as best it can.

v2 already has that behavior for int and uint, and we should consistently do that for float as well.

However, I think the 2nd regression is more nuanced. The prior behavior disagrees with the docs. Noticed that I pre-populated the "x" field and it still got assigned to nil, which isn't really "skipping that field". Similar to the concrete float case, we should avoid mutating whatever was previously there.

[gopherbot]

Change https://go.dev/cl/820580 mentions this issue: encoding/json/v2: avoid setting with errors parsing float

[dsnet]

https://go.dev/cl/820580 fixes case 1 and halfway fixes cases 2.

In particular for case 2, we still populate the map entry with a concrete float64, but leave it as zero. This is a justifiable position since the JSON value is legitimately a number, it just couldn't semantically be parsed into the resulting float64.

The prior v1 behavior was strange in that it assigned nil to the entire interface value and I'm not sure how to replicate that behavior.

[dsnet]

@gopherbot Please backport to 1.27. This is a regression in encoding/json causing it to store Inf when a parsing a value that overflows.

[gopherbot]

Backport issue(s) opened: #81084 (for 1.27).

Remember to create the cherry-pick CL(s) as soon as the patch is submitted to master, according to https://go.dev/wiki/MinorReleases.

[IBlackVoid]

I built CL 820580 and ran a pre-populated 3.14 through every destination shape, against the old implementation and the new one.

destination	GOEXPERIMENT=nojsonv2	go1.27.0	CL 820580
*float64	3.14	+Inf	3.14
*any	3.14	+Inf	float64(0)
struct field float64	3.14	+Inf	3.14
struct field any	3.14	+Inf	float64(0)
map[string]float64	0	+Inf	0
map[string]any	nil	+Inf	float64(0)
[]float64 element	3.14	+Inf	3.14
[]any element	3.14	+Inf	float64(0)

On the nil, it is not a behaviour, it is the scratch value. v1 decodes every map element into a single mapElem that it zeroes each iteration (decode.go:694 and :696) and then stores with SetMapIndex whether the decode succeeded or not (decode.go:812). It never has the existing entry in hand at all. The zero of any is nil and the zero of float64 is 0, which is the entire difference between the last two map rows. So there is nothing to replicate, and float64(0) is already what v1 does for a typed float map. Same mechanism as #42508.

The concrete float rows come out exact after the CL. The four any rows still move, and v1 left the old value in three of them.

That gap does not show up in the tests, because the two surfaces disagree. Under the CL:

json/v2   *any           -> 3.14 (float64)
json v1   *any           -> 0    (float64)

json/v2   *any(float32)  -> 3.14 (float32)
json v1   *any(float32)  -> 0    (float64)

v2 merges into the concrete type already in the interface, so Floats/Any/ExistingFloat64/Overflow and Floats/Any/ExistingFloat32/Overflow pass. The encoding/json case added to TestPrefilled uses addr(any(float64(0))) for both ptr and out, so it passes either way. Swapping that fixture for math.Pi, like the float64 case just above it, fails. The failure prints addresses rather than values, since %v on a *any gives the pointer.

Last thing worth having in the record. Whatever is left behind has to survive a round trip:

nojsonv2   Marshal after the failed decode -> {"x":3.14}
go1.27.0   Marshal after the failed decode -> json: unsupported value: +Inf

The doc you quoted invites callers to carry on after this error, and on 1.27 they carry on holding a value Marshal refuses. The CL fixes that in every row, {"x":0} encodes fine.

[dsnet]

The logic resulting in things still being set to zero in 3rd column is due to

go/src/encoding/json/v2/arshal_default.go

Line 1887 in c69908e

va.SetZero()

If you remove that line, it fixes some many things , but breaks a number of other test cases in v1.

Commenting that out produces:

destination	CL 820580 without va.SetZero()
*float64	3.14
*any	3.14
struct field float64	3.14
struct field any	3.14
map[string]float64	0
map[string]any	0
[]float64 element	3.14
[]any element	3.14

The map[string]any case is still different.

[gopherbot]

Change https://go.dev/cl/820740 mentions this issue: encoding/json/v2: leave the destination alone when a value fails to convert

[prattmic]

Are we happy with the semantics in https://go.dev/cl/820580? If so, we should close this issue. Or do we want to try to be more consistent about not touching the existing fields as @IBlackVoid suggests? I don't have a strong opinion. The latter sounds nicer, but diverges from v1 behavior IIUC.