Strict key matching ignore Unicode zero characters

[maciej-kisiel]

Overview

The library will match key\u0000 with key when decoding structs. This poses security threats in protocols that require exact matching of JSON object keys.

Reproduction

package main

import (
	"testing"

	"github.com/segmentio/encoding/json"
)

func TestUnmarshalNullCharacter(t *testing.T) {
	type Target struct {
		Field string `json:"field"`
	}

	jsonText := `{"field\u0000": "value"}`
	var got Target
	if err := json.Unmarshal([]byte(jsonText), &got); err != nil {
		t.Fatalf("Unmarshal failed: %v", err)
	}
	if got.Field != "" {
		t.Errorf("Field was set!")
	}
}

Additional context

Here is some root cause analysis generated by my coding agent (may not be entirely correct):

Root Cause Analysis

The issue stems from an interaction between segmentio/encoding/json and its underlying assembly-optimized key lookup package, segmentio/asm/keyset.

1. Zero-Padding in Comparison: The keyset.Lookup function (implemented in assembly, e.g., keyset_amd64.s) uses SIMD (AVX) instructions to compare keys. To handle keys shorter than 16 bytes, it loads the key and masks out (zeroes) the trailing bytes beyond the key's length.
2. Ambiguous Nulls: The keys stored in the keyset are also effectively zero-padded to 16 bytes.
   • If you have a struct field named "field" (length 5), it is compared as field followed by 11 null bytes.
   • If the input JSON key is "field\u0000" (length 6), the parser decodes this to the byte slice field\x00.
   • When Lookup processes field\x00, it masks out bytes after index 6. The result in the register is field + \x00 + 10 null bytes.
3. False Positive Match: To the assembly comparison logic, the zero-padded "field" (len 5) and the zero-padded "field\x00" (len 6) look identical (field\0\0...). The Lookup function returns the index for "field", leading to the incorrect match.

[maciej-kisiel]

If the fix is accepted, I would urge you to create a new patch release with it, so that upstream users can use the patched version.