feat: update and add mode handling (#2)

Makefile

@@ -18,7 +18,7 @@ TMPDIR ?= /tmp
 # Setup default go-make installation flags.
 INSTALL_FLAGS ?= -mod=readonly -buildvcs=auto
 # Setup go-make version to use desired build and config scripts.
-GOMAKE_DEP ?= github.com/tkrop/go-make@v0.4.17
+GOMAKE_DEP ?= github.com/tkrop/go-make@v0.4.18
 # Request targets from go-make show-targets target.
 TARGETS := $(shell command -v $(GOBIN)/go-make >/dev/null || \
 	$(GO) install $(INSTALL_FLAGS) $(GOMAKE_DEP) >&2 && \

README.md

@@ -54,42 +54,41 @@ by the great work of Dave Chaney [pkg/json][json-pkg] and his article about
 [Building a high-performance JSON parser][json-hp].
 
 **Warning:** This work is not meant as a drop in replacement for the default
-[encoding/json][json-enc] parser, and even if it provides a compatible
-interface it produces many essential but also subtle differences.
+[encoding/json][json-enc] parser, and even if it currently provides a visibly
+compatible interface it produces many essential but also subtle differences.
 
 I have undertaken this journey in a trial to make it fit for fast parsing of
 very short default configuration scripts and tags out of pure curiosity and
 the lack of working alternatives in `go`. While the parser is now nearly
 production ready, I am not sure yet where this journey ends.
 
-It currently supports the full [JSON5][json5] specification with braces,
+Currently, the parser supports the [JSON5][json5] specification with braces,
 brackets, commas, colons, and all other [JSON5 features][json5-features],
 including:
 
-* Support for unquoted keys in objects.
-* Support for escape sequences in strings, including `\n`, `\t`, `\\`, etc.
-* Support for single-line (`//`) and multi-line (`/* ... */`) comments.
-* Support for integer, decimal, and hexadecimal numbers (e.g., `0x1E`).
-* Support for Unicode escape sequences in strings (e.g., `\u{1F600}`,
-  `\U0X1F4A9`).
+* Single and double quoted keys in objects.
+* Escape sequences in strings, including `\n`, `\t`, `\\`, etc.
+* Single-line (`//`) and multi-line (`/* ... */`) comments.
+* Integer, decimal, and hexadecimal numbers (e.g., `0x1E`).
+* Unicode escape sequences in strings (e.g., `\u{1F600}`, `\U0X1F4A9`).
 
-Besides, the parser also supports following extra features:
+Besides, the parser also supports the following extra _relaxed_ and
+_extended_ features:
 
-* Support for unquoted string values in objects and arrays, that are not
-  reserved keywords (`true`, `false`, `null`, `NaN`, `Infinity`), and do not
-  contain any leading or trailing whitespace or special characters.
-* Support for complex numbers in values (e.g., `1+2i`).
+* Unquoted string keys and values in objects and arrays, that are not reserved
+  keywords (`true`, `false`, `null`, `NaN`, `Infinity`), and do not contain any
+  leading or trailing whitespace or special characters.
+* Complex numbers in values (e.g., `1+2i`).
 
-**Note:** Currently, the parser is not capable to switch between the strict
-[JSON][json]/[JSON5][json5] parsing modes, and the relaxed/extended
-[JSON5][json5] parsing modes, but this feature will be available in the
-near future (see [Future plans](#future-plans)).
-
-At the moment I'm just more focusing on performance and correctness of the
-`Scanner`, `Parser`, and `Decoder`, as well as on a first design of the public
-interfaces. I must admit that I currently lack the vision of how to expose the
-different parsing modes and features I have in mind in a clean way.
+Based on these additional features, the input can end up looking more like a
+[`yaml`][yaml] document instead of a strict [JSON5][json5] document, but the
+parser will still interpret all elements correctly. While the _relaxed_ and
+_extended_ parsing is the default, these features can be disabled, so that the
+parser will only accept strict [JSON5][json5] or [JSON][json] documents.
 
+**Note:** The _relaxed_ and _extended_ `JSON5` parsing mode is absolutely
+forgiving and never fails, but may produce invalid token series for the
+`Decoder`.
 
 
 <!--
@@ -137,31 +136,36 @@ components:
 * The `Scanner` abstraction allows to efficiently scan the input data provided
   by the `Reader` into tokens. The `Scanner` is coming in two main flavors with
   and without tracking of line and character position, as well as in multiple
-  sub flavors for strict `JSON`, strict `JSON5`, and relaxed/extended `JSON5`
-  parsing modes.
+  sub flavors for strict, relaxed, and extended `JSON5` and strict `JSON`
+  parsing (strict `JSON` is not implemented yet).
 
 * The `Printer` abstraction allows to consume a stream of tokens directly as
   provided by the `Scanner` back into a identical output byte stream providing
-  a valid `JSON`, `JSON5`, or relaxed/extended `JSON5` document with proper
-  indentation and comments.
+  a valid strict, relaxed, or extended `JSON5` document with proper indentation
+  and comments (not implemented yet).
+
+* The `Decoder` abstraction allows to directly decode the stream of tokens into
+  Go objects using reflection. The `Decoder` comes in two flavors supporting a
+  _native_ and a _precise_ type decoding using primitive or precise composite
+  Go types.
 
-* The `Decoder` abstraction allows to directly decode the stream of tokens from
-  the `Scanner` into Go objects using reflection. The `Decoder` comes in two
-  flavors supporting a _native_ and a _precise_ type decoding.
+  **Note:** Contrary to the `encoding/json` package, the `Decoder` requires by
+  default exact names. You can enable case-insensitive matching by setting the
+  `CaseIgnore` mode.
 
 * The `Encoder` abstraction allows to encode Go objects into a stream of tokens
   that can be consumed by the `Printer` to produce a valid `JSON`, `JSON5`, or
-  relaxed/extended `JSON5` document.
+  relaxed/extended `JSON5` document (not implemented yet).
 
 * The `Filter` abstraction allows to dynamically filter a stream of tokens
   provided by the `Scanner` according to a specified filter function. The
   default filters allow to skip comments, whitespace, and other tokens that
   are not matching a specific `JSON`, `JSON5`, or relaxed/extended `JSON5`
-  document standard.
+  document standard (not implemented yet).
 
 * The `Parser` abstraction allows to validate the stream of tokens provided by
   the `Scanner` and - if requested - to build an abstract syntax tree, that can
-  be used for analysis and processing.
+  be used for analysis and processing (not implemented yet).
 
 
 ## Future plans
@@ -172,41 +176,37 @@ The following features are planned for the future, but not yet implemented:
   they can be used independently and reused in other projects, and advance
   tests to public interface testing.
 
-* Create different `Scanner` implementations for strict `JSON`, strict `JSON5`,
-  as well as extended and relaxed `JSON5` parsing modes - with and without
-  tracking of character and line numbers. For strict `JSON` and `JSON5` parsing
-  modes, the scanner needs to define a clear memory and failure model.
-
-  *Note: The relaxed and extended `JSON5` parsing mode is absolutely forgiving
-  and never fails, but may produce invalid token series for the `Decoder`.
-
 * Create a non-releasing `Reader`, that allows to access the underlying data
   without releasing the buffer to enable permanent zero-copy decoding of the
-  data.
+  data (partially done).
 
 * Create specialized `Decoder` implementations for different parsing modes and
   different default value types, i.e. big.Int and big.Float vs int64 and
-  float64.
+  float64 (_precise_ vs _native_).
 
-* Create an extended and relaxed JSON5 `Emitter` supporting exactly the token
-  parsing events of the relaxed and extended JSON5 `Scanner` to output the
-  unchanged or patched JSON data without ever creating a decoded object.
+* Create an _extended_ and _relaxed_ [JSON5][json5]/[JSON][json] `Emitter`
+  supporting the token parsing events of the _relaxed_ and _extended_
+  [JSON5][json5] `Scanner` to output a patched or unchanged [JSON5][json5]
+  data without ever creating a decoded object.
 
 * Create a `Parser` that can parse JSON5 data into an abstract syntax tree,
   allowing for more advanced manipulation and analysis of the JSON5 structure.
 
-* Create an `Encoder` that can encode Go objects into minimal relaxed JSON5
-  data, that can be decoded by the `Decoder` without loss of any information.
+* Create an `Encoder` that can encode Go objects into minimal _relaxed_ and
+  _extended_ [JSON5][json5] output, that can be decoded by the `Decoder`
+  without loss of information.
 
-* Create JSONPatch support to allow for efficient, on-the-fly patching of
-  JSON data while scanning, parsing, or decoding it.
+* Create [JSONPatch][json-patch] support to allow for efficient, on-the-fly
+  patching of JSON data while scanning, parsing, or decoding it.
 
 
 Open questions:
 
 * Should we eliminate defensive error handling in the `Decoder` that can not
   happen due to the `Scanner` implementation and just panic in these cases?
 
+[json-patch]: <https://datatracker.ietf.org/doc/html/rfc6902>
+
 
 ## Building
 

go.mod

@@ -4,10 +4,10 @@ go 1.26.5
 
 require (
 	github.com/adhocore/jsonc v0.10.0
-	github.com/pkg/json v0.0.0-20231102085813-703c45db51c1
+	github.com/pkg/json v0.0.0-20260504031610-00e60cb5517f
 	github.com/stretchr/testify v1.11.1
 	github.com/titanous/json5 v1.0.0
-	github.com/tkrop/go-testing v0.3.0
+	github.com/tkrop/go-testing v0.4.1
 	github.com/yosuke-furukawa/json5 v0.1.1
 	gopkg.in/yaml.v3 v3.0.1
 )

go.sum

@@ -10,8 +10,8 @@ github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
 github.com/kylelemons/godebug v1.1.0 h1:RPNrshWIDI6G2gRW9EHilWtl7Z6Sb1BR0xunSBf0SNc=
 github.com/kylelemons/godebug v1.1.0/go.mod h1:9/0rRGxNHcop5bhtWyNeEfOS8JIWk580+fNqagV/RAw=
 github.com/pkg/diff v0.0.0-20210226163009-20ebb0f2a09e/go.mod h1:pJLUxLENpZxwdsKMEsNbx1VGcRFpLqf3715MtcvvzbA=
-github.com/pkg/json v0.0.0-20231102085813-703c45db51c1 h1:KwfHlwChGylq0YD3KHikudAw6yxjIyuolysHpd3v8gw=
-github.com/pkg/json v0.0.0-20231102085813-703c45db51c1/go.mod h1:fa/tC+3e7pkKgpdE8QBs/D9HksUG15xmlVL19qnufRM=
+github.com/pkg/json v0.0.0-20260504031610-00e60cb5517f h1:J1XTpfOjYWdhsXGtLE1fmER8gHPRgXJfmePGEOW4uHQ=
+github.com/pkg/json v0.0.0-20260504031610-00e60cb5517f/go.mod h1:MAh3BZfSarzjOyDooVDqqjll1xxRpWLiZbjqGUutTgo=
 github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 h1:Jamvg5psRIccs7FGNTlIRMkT8wgtp5eCXdBlqhYGL6U=
 github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
 github.com/robertkrimen/otto v0.2.1 h1:FVP0PJ0AHIjC+N4pKCG9yCDz6LHNPCwi/GKID5pGGF0=
@@ -22,14 +22,14 @@ github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu
 github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U=
 github.com/titanous/json5 v1.0.0 h1:hJf8Su1d9NuI/ffpxgxQfxh/UiBFZX7bMPid0rIL/7s=
 github.com/titanous/json5 v1.0.0/go.mod h1:7JH1M8/LHKc6cyP5o5g3CSaRj+mBrIimTxzpvmckH8c=
-github.com/tkrop/go-testing v0.3.0 h1:UuwPZ27iQvaN9fwCCIcbO4Jcg5RWX14+wybXtHlREGg=
-github.com/tkrop/go-testing v0.3.0/go.mod h1:0gbXXV55KYBvwjiDrLEKwnBmeVA2Hwx4Yj0V0086DLY=
+github.com/tkrop/go-testing v0.4.1 h1:HStzcE31TpkT4Pqjbd8qV37Ew56Kbg0ugUymwMdUtJo=
+github.com/tkrop/go-testing v0.4.1/go.mod h1:LnFQzchgll5iIoqw8pJiEGyynAEZK/a7a+6wSw8uGiw=
 github.com/yosuke-furukawa/json5 v0.1.1 h1:0F9mNwTvOuDNH243hoPqvf+dxa5QsKnZzU20uNsh3ZI=
 github.com/yosuke-furukawa/json5 v0.1.1/go.mod h1:sw49aWDqNdRJ6DYUtIQiaA3xyj2IL9tjeNYmX2ixwcU=
 go.uber.org/mock v0.6.0 h1:hyF9dfmbgIX5EfOdasqLsWD6xqpNZlXblLB/Dbnwv3Y=
 go.uber.org/mock v0.6.0/go.mod h1:KiVJ4BqZJaMj4svdfmHM0AUx4NJYO8ZNpPnZn1Z+BBU=
-golang.org/x/text v0.30.0 h1:yznKA/E9zq54KzlzBEAWn1NXSQ8DIp/NYMy88xJjl4k=
-golang.org/x/text v0.30.0/go.mod h1:yDdHFIX9t+tORqspjENWgzaCVXgk0yYnYuSZ8UzzBVM=
+golang.org/x/text v0.41.0 h1:vz/seA0lnX87Othu2f/0L24RcgrXD9/YFTSuGjj3rH8=
+golang.org/x/text v0.41.0/go.mod h1:jvf1O8ajNzZqhSrQBPbutR/EB83Cc0CFrezNQIwbb5M=
 gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
 gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk=
 gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q=

json/common.go

@@ -1,5 +1,117 @@
 package json
 
+// Mode defines the decoding mode for the decoder/scanner/reader.
+type Mode int
+
+// Mode values are used to configure the behavior of the decoder/scanner/reader.
+const (
+	// Evict forces the decoder/scanner/reader to evict processed data from the
+	// buffer as soon as possible.
+	Evict Mode = 0x0000
+	// Retain forces the decoder/scanner/reader to retain all data in the
+	// buffer, while activating decoder optimizations based on the buffering.
+	Retain Mode = 0x0001
+
+	// StrictJson forces the decoder/scanner/reader to strictly follow the
+	// JSON specification and throw errors on any non-JSON-standard input (not
+	// implemented).
+	StrictJson Mode = 0x0002
+
+	// Strict forces the decoder/scanner/reader to strictly follow the JSON5
+	// specification and throw errors on any non-standard input.
+	Strict Mode = 0x0000
+	// Relaxed allows the decoder/scanner/reader to accept a more relaxed
+	// JSON5 input missing quotes around object keys without throwing errors.
+	Relaxed Mode = 0x0010
+	// Extended allows the decoder/scanner/reader to accept non-standard
+	// JSON5 input using extensions, e.g. parsing of complex numbers.
+	Extended Mode = 0x0020
+
+	// Native forces the decoder/scanner/reader to use native Go types for
+	// numbers.
+	Native Mode = 0x0000
+	// Precise forces the decoder/scanner/reader to use arbitrary precision
+	// types for numbers.
+	Precise Mode = 0x0100
+
+	// CaseIgnore forces the decoder/scanner/reader to ignore case when
+	// comparing object keys.
+	CaseIgnore Mode = 0x1000
+
+	// CaseDelimSkip forces the decoder/scanner/reader to skip hyphens and
+	// underscores when comparing object keys.
+	CaseDelimSkip Mode = 0x2000 // FIXME: Not implemented yet.
+)
+
+// Basic JSON token types for scanning.
+const (
+	// ObjectStart `{` - the object start.
+	ObjectStart byte = '{'
+	// ObjectEnd `}` - the object end.
+	ObjectEnd byte = '}'
+	// ArrayStart `[` - the array start.
+	ArrayStart byte = '['
+	// ArrayEnd `]` - the array end.
+	ArrayEnd byte = ']'
+	// Comma `,` - the literal comma.
+	Comma byte = ','
+	// Colon `:` - the literal colon.
+	Colon byte = ':'
+	// Space ` ` - a white space token.
+	Space byte = ' '
+	// Comment `/` - a single-line comment token (`//`).
+	Comment byte = '/'
+	// CommentMulti `*` - a multi-line comment token (`/*...*/`).
+	CommentMulti byte = '*'
+	// Escape `\` - a backslash escape token.
+	Escape byte = '\\'
+	// Quote `'` - a single quote string token.
+	Quote byte = '\''
+	// String `"` - a double quote string token.
+	String byte = '"'
+	// True `t` - the literal true token.
+	True byte = 't'
+	// False `f` - the literal false token.
+	False byte = 'f'
+	// Null `n` - the literal null token.
+	Null byte = 'n'
+)
+
+// Special JSON token types for scanning.
+const (
+	// NaN `N` - the literal NaN token.
+	NaN byte = 'N'
+	// Infinity `I` - the literal Infinity token.
+	Infinity byte = 'I'
+	// Integer `i` - the literal integer token.
+	Integer byte = 'i'
+	// Decimal `d` - the literal decimal token.
+	Decimal byte = 'd'
+	// Complex `c` - the literal complex token.
+	Complex byte = 'c'
+	// HexaDecimal `x` - the literal hexadecimal token.
+	HexaDecimal byte = 'x'
+	// EOF `\0` - the end of a token stream.
+	EOF byte = 0x00
+)
+
+// ScanContext describes the parser position expected by Scanner.Next methods.
+type ScanContext byte
+
+// ScanContext values are used to indicate the expected context of the next token
+// to be scanned by the Scanner. The context is used to determine the valid
+// token types and parsing rules for the next token.
+const (
+	// RootValue scans a value in root position.
+	RootValue ScanContext = iota
+	// ObjectKey scans an object property key.
+	ObjectKey
+	// ObjectValue scans an object property value.
+	ObjectValue
+	// ArrayValue scans an array element value.
+	ArrayValue
+)
+
 // hexmap maps every ASCII byte to its 4-bit hex value. Entries outside
 // '0'-'9', 'A'-'F', 'a'-'f' are 0xFF (invalid sentinel). Valid values are
 // `0x00`-`0x0F`; the OR of any four values can therefore be compared to be