encoding/json/v2: performance regression when umarshalling in []any

[gaaf]

Unmarshaling into a []any with preset typed elements becomes 25% slower with v1 when run under the jsonv2 experiment. In addition, it does 50% more allocations. Switching to purely v2 is slightly faster but triples the amount of allocations.

My use case for this scenario is parsing the array variant of JSON-RPC 2.0 params fields, which may be encoded as a JSON object or array.

Go version 1.26.0.

goos: linux
goarch: amd64
cpu: AMD Ryzen 5 5600G with Radeon Graphics         
         │    v1.txt    │            v1-wrap-v2.txt            │               v2.txt               │
         │    sec/op    │    sec/op     vs base                │   sec/op     vs base               │
Array-12   1064.5n ± 1%   1335.0n ± 1%  +25.41% (p=0.000 n=10)   959.3n ± 1%  -9.88% (p=0.000 n=10)

         │   v1.txt    │           v1-wrap-v2.txt           │               v2.txt                │
         │    B/op     │    B/op     vs base                │    B/op      vs base                │
Array-12   176.00 ± 0%   48.00 ± 0%  -72.73% (p=0.000 n=10)   224.00 ± 0%  +27.27% (p=0.000 n=10)

         │   v1.txt   │           v1-wrap-v2.txt           │                v2.txt                │
         │ allocs/op  │ allocs/op   vs base                │  allocs/op   vs base                 │
Array-12   4.000 ± 0%   6.000 ± 0%  +50.00% (p=0.000 n=10)   11.000 ± 0%  +175.00% (p=0.000 n=10)

Test case:

package jsonrpc2_test

import (
	"encoding/json"
	// json "encoding/json/v2"
	"reflect"
	"testing"
)

type Params struct {
	Name   string
	Count  uint64
	Index  int32
	Price  float64
	Active bool
	Tags   []string
}

var JSONArray = []byte(
	`[
		"example",
		1844674407370955161,
		42,
		1234.567,
		true,
		["a","b","c"]
	]`,
)

func BenchmarkArray(b *testing.B) {
	paramsType := reflect.TypeOf((*Params)(nil)).Elem()
	fieldCount := paramsType.NumField()
	// Set slice element's types to Params element's types
	args := make([]any, fieldCount)
	for i := 0; i < fieldCount; i++ {
		args[i] = reflect.New(paramsType.Field(i).Type).Interface()
	}

	for i := 0; i < b.N; i++ {
		json.Unmarshal(JSONArray, &args)
	}
}

For purely v2 testing, the json import line is changed to json "encoding/json/v2".

[gabyhelp]

Related Issues

• encoding/json: skyrocketing memory allocation in encoding when `jsonv2` experiment enabled #75026 (closed)
• encoding/json: Unmarshal to a typed struct uses 50% more bytes in go 1.21 (and current versions) vs go 1.20 #69828 (closed)
• encoding/json: performance regression marshaling small values for goexperiment.jsonv2 #74617
• encoding/json: Unmarshal behavior changed to match its documented behavior about not reusing slice elements #39427 (closed)
• encoding/json: do not fill d.errorContext until deemed necessary #37571 (closed)
• encoding/json: GOEXPERIMENT=jsonv2 changes behavior of Decoder.InputOffset after More #75468 (closed)
• encoding/json: map[string]interface performance #35585 (closed)

Related Code Changes

• encoding/json/v2, reflect: free intermediate memory when unmarshalling slices
• encoding/json: fix decoding of types with '[]byte' as underlying type

_{(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)}

[gaaf]

None of @gabyhelp's references seem relevant.

Should have referenced #71497.

[prattmic]

cc @dsnet

[Dorbmon]

This is likely intended. Under v1 semantics, existing non-nil interface elements are merged and decoded into their original *T values one by one. Under default v2 semantics, each reused slice element is zeroed before decoding; for []any this turns elements into nil interfaces, erasing dynamic type information, so decoding falls back to generic any values.

[gaaf]

Indeed, behavior did change in purely v2. Type information is wiped from the resulting []any and replaced with the "default" type for the JSON value. I should not have cut the test part from the example, as that would have shown that immediately.

I seem to have missed this change was "intended". The replacement to get the correct types seems to be to add an Unmarshaler, but this shows an even worse performance regression. Did I miss a way to get similar performance with correct (preset) types?

goos: linux
goarch: amd64
cpu: AMD Ryzen 5 5600G with Radeon Graphics         
         │   v1.txt    │           v1-wrap-v2.txt            │          v2-unmarshaler.txt          │
         │   sec/op    │   sec/op     vs base                │   sec/op     vs base                 │
Array-12   1.065µ ± 1%   1.335µ ± 1%  +25.41% (p=0.000 n=10)   3.639µ ± 1%  +241.80% (p=0.000 n=10)

         │   v1.txt    │           v1-wrap-v2.txt           │          v2-unmarshaler.txt           │
         │    B/op     │    B/op     vs base                │     B/op      vs base                 │
Array-12   176.00 ± 0%   48.00 ± 0%  -72.73% (p=0.000 n=10)   1935.00 ± 0%  +999.43% (p=0.000 n=10)

         │   v1.txt   │           v1-wrap-v2.txt           │          v2-unmarshaler.txt          │
         │ allocs/op  │ allocs/op   vs base                │  allocs/op   vs base                 │
Array-12   4.000 ± 0%   6.000 ± 0%  +50.00% (p=0.000 n=10)   40.000 ± 0%  +900.00% (p=0.000 n=10)

Test code for v1 and v1-wrap-v2 are still the same as in original post.

Test code only for json/v2, using an Unmarshaler

package jsonrpc2_test

// Tests are only for purely json/v2

import (
	"encoding/json/jsontext"
	json "encoding/json/v2"
	"reflect"
	"testing"

	"github.com/stretchr/testify/assert"
)

type Params struct {
	Name   string
	Count  uint64
	Index  int32
	Price  float64
	Active bool
	Tags   []string
}

var JSONArray = []byte(
	`[
		"example",
		1844674407370955161,
		42,
		1234.567,
		true,
		["a","b","c"]
	]`,
)

func TestArray(t *testing.T) {
	paramsType := reflect.TypeFor[Params]()
	fieldCount := paramsType.NumField()
	// Build slice of Params element's types
	args := make([]any, fieldCount)

	n := 0
	err := json.Unmarshal(JSONArray, &args,
		// Intercept every attempt to unmarshal into the any type.
		json.WithUnmarshalers(
			json.UnmarshalFromFunc(func(dec *jsontext.Decoder, val *any) error {
				// Provide a concrete Go type to unmarshal into.
				*val = reflect.New(paramsType.Field(n).Type).Interface()
				n++
				// Return SkipFunc to fallback on default unmarshal behavior.
				return json.SkipFunc
			}),
		))
	assert.NoError(t, err)

	var s string = "example"
	assert.IsType(t, &s, args[0])
	assert.Equal(t, &s, args[0])

	var u uint64 = 1844674407370955161
	assert.IsType(t, &u, args[1])
	assert.Equal(t, &u, args[1])

	var i int32 = 42
	assert.IsType(t, &i, args[2])
	assert.Equal(t, &i, args[2])

	var f float64 = 1234.567
	assert.IsType(t, &f, args[3])
	assert.Equal(t, &f, args[3])

	var b bool = true
	assert.IsType(t, &b, args[4])
	assert.Equal(t, &b, args[4])

	var ss = []string{"a", "b", "c"}
	assert.IsType(t, &ss, args[5])
	assert.Equal(t, &ss, args[5])
}

func BenchmarkArray(b *testing.B) {
	paramsType := reflect.TypeFor[Params]()
	fieldCount := paramsType.NumField()
	// Build slice of Params element's types
	args := make([]any, fieldCount)
	for i := range fieldCount {
		args[i] = reflect.New(paramsType.Field(i).Type).Interface()
	}

	for i := 0; i < b.N; i++ {
		n := 0
		err := json.Unmarshal(JSONArray, &args,
			// Intercept every attempt to unmarshal into the any type.
			json.WithUnmarshalers(
				json.UnmarshalFromFunc(func(dec *jsontext.Decoder, val *any) error {
					// Provide a concrete Go type to unmarshal into.
					*val = reflect.New(paramsType.Field(n).Type).Interface()
					n++
					// Return SkipFunc to fallback on default unmarshal behavior.
					return json.SkipFunc
				}),
			))
		if err != nil {
			b.Fatal(err)
		}
	}
}

[dsnet]

The fact that this worked in v1 was relying on behavior that goes contrary to the documented semantics of even v1. In particular, Unmarshal says:

To unmarshal a JSON array into a slice, Unmarshal resets the slice length to zero and then appends each element to the slice.

Unfortunately, the behavior of v1 was internally inconsistent about how it actually handled unmarshaling into a non-zero value. In v2, we aimed to have consistent behavior regarding merging, which is documented as:

In general, unmarshaling follows merge semantics (similar to RFC 7396) where the decoded Go value replaces the destination value for any JSON kind other than an object. For JSON objects, the input object is merged into the destination value where matching object members recursively apply merge semantics.

Regarding performance: The increase in allocations in v2 is an artifact of how we implemented the fix for bugs like #22967, #27722, or #33993 where custom methods were not called because it was declared on a pointer receiver, but only the value receiver was available. To fix that problem, we shallow copy aggressively to ensure that every value is always addressable. Since the underlying interface values are not addressable, that results in a shallow copy. In the future, we hope to make the implementation avoid shallow copying by default and lazily do it on demand.

[dsnet]

To support this particular use case, I recommend making use of the new jsontext.Decoder, which allows you to parse the input in a purely streaming manner directly into the destination struct. For example:

func (p *Params) UnmarshalJSONFrom(d *jsontext.Decoder) error {
	// Consume the begin JSON array delimiter.
	switch tok, err := d.ReadToken(); {
	case err != nil:
		return err
	case tok.Kind() != '[':
		return fmt.Errorf("got %v, expecting '['", tok.Kind())
	}

	// Unmarshal each JSON array element into the Go struct
	// matching the array element order with the struct field order.
	v := reflect.ValueOf(p).Elem()
	for i := range v.NumField() {
		fv := v.Field(i).Addr()
		if fv.CanInterface() { // ignore unexported fields
			if err := jsonv2.UnmarshalDecode(d, fv.Interface()); err != nil {
				return err
			}
		}
	}

	// Consume the end JSON array delimiter.
	switch tok, err := d.ReadToken(); {
	case err != nil:
		return err
	case tok.Kind() != ']':
		return fmt.Errorf("got %v, expecting ']'", tok.Kind())
	}

	return nil
}

Performance:

BenchmarkSliceV1
BenchmarkSliceV1-32       	 1864593	       639.7 ns/op	     176 B/op	       4 allocs/op
BenchmarkSliceV1in2
BenchmarkSliceV1in2-32    	 1000000	      1017 ns/op	     176 B/op	       7 allocs/op
BenchmarkSliceV2
BenchmarkSliceV2-32       	 2381050	       509.2 ns/op	     224 B/op	      11 allocs/op
BenchmarkStructV2
BenchmarkStructV2-32      	 3095935	       385.7 ns/op	       0 B/op	       0 allocs/op

This is around 1.7x faster and has zero allocations.

[gaaf]

Thanks for the UnmarshalJSONFrom hint. My implementation using a Decoder did almost the same but performance differs a lot.

For my jsonrpc2 lib, I really don't want to have the user attach a (complex) UnmarshalJSONFrom to every params struct in the API, just to parse an array input. My current (jsonv1) implementation lets the user register an RPC method, where the register function is using generics and reflect to create an optimized json unmarshaler for the RPC's parameters struct (both from object and array syntax).

I'm really wondering why using UnmarshalJSONFrom on a struct receiver (BenchmarkStructV2) is so much more efficient than using a local v2 Decoder (BenchmarkArrayDecoderv2), which is basically the same code:

// Array-style parser using v2's jsontext.Decoder streaming
func makeParserArrayDecoderV2() Parser {
	return func(in []byte) *Params {
		d := jsontext.NewDecoder(bytes.NewReader(in))

		// Consume the begin JSON array delimiter.
		switch tok, err := d.ReadToken(); {
		case err != nil:
			return nil
		case tok.Kind() != '[':
			return nil
		}

		// Unmarshal each JSON array element into the Go struct
		// matching the array element order with the struct field order.
		p := new(Params)
		v := reflect.ValueOf(p).Elem()
		for i := range v.NumField() {
			fv := v.Field(i).Addr()
			if fv.CanInterface() {
				if err := jsonv2.UnmarshalDecode(d, fv.Interface()); err != nil {
					return nil
				}
			}
		}

		// Consume the end JSON array delimiter.
		switch tok, err := d.ReadToken(); {
		case err != nil:
			return nil
		case tok.Kind() != ']':
			return nil
		}

		return p
	}
}

BenchmarkStructV2
BenchmarkStructV2-12                     1723464               696.9 ns/op             0 B/op          0 allocs/op
BenchmarkArrayDecoderV1
BenchmarkArrayDecoderV1-12                480448              2281 ns/op            1544 B/op         31 allocs/op
BenchmarkArrayDecoderV1in2
BenchmarkArrayDecoderV1in2-12             618867              1817 ns/op             696 B/op         10 allocs/op
BenchmarkArrayDecoderV2
BenchmarkArrayDecoderV2-12                679052              1690 ns/op            5520 B/op         11 allocs/op
BenchmarkArrayUnmarshalerV2
BenchmarkArrayUnmarshalerV2-12            423493              2744 ns/op            1517 B/op         30 allocs/op

Playground

[dsnet]

I'm really wondering why using UnmarshalJSONFrom on a struct receiver (BenchmarkStructV2) is so much more efficient than using a local v2 Decoder (BenchmarkArrayDecoderv2), which is basically the same code:

It's because the latter would need to re-allocate the Decoder with all the buffers and state needed parse JSON. The UnmarshalJSONFrom method avoids this since it reuses the Decoder is already owned by the "json" package and pooled across usages.

My current (jsonv1) implementation lets the user register an RPC method, where the register function is using generics and reflect to create an optimized json unmarshaler for the RPC's parameters struct (both from object and array syntax).

In json/v2, there are two ways to customize behavior:

• type-specified customization via methods
• caller-specified customization via options

As the author of a general-purpose JSON RPC library, I believe it's the later form of customization that you would want. In json/v2, this is provided via the MarshalToFunc and UnmarshalFromFunc.

For example, you could do something like:

var (
	unmarshalersMu     sync.Mutex
	unmarshalers       []*json.Unmarshalers // unmarshaler for each registered type
	joinedUnmarshalers atomic.Pointer[json.Unmarshalers] // nil means this needs to be reconstructed on demand
)

// RegisterType registers a particular struct type to optionally use a JSON array representation
// in addition to it's default representation as a JSON object.
// In the JSON array representation, each exported Go struct fields corresponds
// in-order with a JSON array element.
func RegisterType[T any]() {
	t := reflect.TypeFor[T]()
	if t.Kind() != reflect.Struct {
		panic(fmt.Sprintf("%T must be a struct", t))
	}

	unmarshalersMu.Lock()
	defer unmarshalersMu.Unlock()
	joinedUnmarshalers.Store(nil) // reset this so that it is reconstructed on next load
	unmarshalers = append(unmarshalers, json.UnmarshalFromFunc(func(d *jsontext.Decoder, p *T) error {
		// If the next kind is not a JSON array, the fallback on default JSON representation.
		if d.PeekKind() != jsontext.KindBeginArray {
			return errors.ErrUnsupported
		}

		// Consume the begin JSON array delimiter.
		switch tok, err := d.ReadToken(); {
		case err != nil:
			return err
		case tok.Kind() != jsontext.KindBeginArray:
			return fmt.Errorf("got %v, expecting %v", tok.Kind(), jsontext.KindBeginArray)
		}

		// Unmarshal each JSON array element into the Go struct
		// matching the array element order with the struct field order.
		v := reflect.ValueOf(p).Elem()
		for i := range v.NumField() {
			fv := v.Field(i).Addr()
			if fv.CanInterface() {
				if err := json.UnmarshalDecode(d, fv.Interface()); err != nil {
					return nil
				}
			}
		}

		// Consume the end JSON array delimiter.
		switch tok, err := d.ReadToken(); {
		case err != nil:
			return err
		case tok.Kind() != jsontext.KindEndArray:
			return fmt.Errorf("got %v, expecting %v", tok.Kind(), jsontext.KindEndArray)
		}

		return nil
	}))
}

func loadUnmarshalers() json.Options {
	// Fast-path: Return the joined unmarshalers as-is if there were no mutations.
	if u := joinedUnmarshalers.Load(); u != nil {
		return json.WithUnmarshalers(u)
	}

	// Slow-path: Reconstruct the joined unmarshalers if there were new registrations.
	unmarshalersMu.Lock()
	defer unmarshalersMu.Unlock()
	u := json.JoinUnmarshalers(unmarshalers...)
	joinedUnmarshalers.Store(u)
	return json.WithUnmarshalers(u)
}

Whenever the JSON RPC library calls json.Unmarshal, it also passes in the options constructed by loadUnmarshalers.

[dsnet]

This issue contains two related issues:

1. The change in behavior of how unmarshal behaves when merging into an non-zero value. The v1 documentation ironically already specifies the behavior that v2 happens to implement. It just so happens that v1 has internally inconsistent behavior that made it possible to accidentally depend on the current behavior. Since this is a bug in v1, we'll keep the behavior in v2.
2. In the v2 implementation, unmarshaling into an interface value is more allocation heavy compared to v1. This is an implementation detail of how v2 operates and we should aim to optimize this in the future.