Clarification of JSON key escape decoding and name delivery timing, separately for parser_callback_t and json_sax::key

[bartygroch-gif]

I’m seeking clarification of the documented contract for object key names in nlohmann/json. The material I reviewed did not establish how JSON escape decoding relates to the name exposed by each interface.

Please address these interfaces separately:

• parser_callback_t when the event is key;
• json_sax::key.

For each interface, could you point to existing documentation or clarify the declared contract regarding the following?

1. Does the exposed name preserve its source spelling, or contain the characters obtained after decoding JSON escape sequences?
2. Is that decoding complete before the callback or key method is invoked?
3. Do different valid JSON spellings representing the same name produce the same interpreted string at that interface?
4. Is validation of Unicode escapes in the name complete before the name is exposed, and can an invalid name trigger the interface’s key notification?
5. Which versions, configurations and input formats does the answer cover?

Please distinguish JSON escape decoding from input encoding, stored-string encoding, additional Unicode normalization and changes to the letter case of names. I am not assuming that escape decoding includes normalization or case conversion. The case of hexadecimal digits in an escape is also distinct from the letter case of the name itself.

The purpose is to understand the name available for comparison during parsing. This question does not assume that the library detects or rejects duplicate names.

[nlohmann]

Thanks for the precise question. The current documentation doesn't spell this out (json_sax::key only says "An object key was read"), so this answer comes from the implementation. We will update the documentation to state this contract soon.

This applies to JSON text input on current develop. parser_callback_t (event key) and json_sax::key behave the same, because both receive the string token produced by the lexer.

1. Decoded, not source spelling. The key contains the characters obtained after decoding all JSON escape sequences (\n, \", \/, \uXXXX, surrogate pairs), stored as UTF-8.
2. Decoding is complete before the notification. The parser calls key() / the callback only after the string token was scanned successfully.
3. Same name, same string. Different valid spellings of the same name ("a", "a", "a" with upper- or lowercase hex digits, "\/" vs. "/") yield the identical string. The case of hex digits in an escape has no effect.
4. Validation happens first. A malformed \u escape, a lone low surrogate, a high surrogate not followed by a low surrogate, or invalid raw UTF-8 produces a lexer error. In that case key() is not called and the callback never sees a key event for that name. \u0000 is valid and yields an embedded NUL character.
5. No normalization or case conversion. Keys are the decoded code points as-is; comparison is byte-wise.

Caveats:

• key() is invoked before the following : has been checked, so a name can be delivered and the parse can still fail afterwards (e.g. a missing separator).
• Duplicate names are delivered once per occurrence. The DOM builder keeps the last value; with SAX or a callback, handling duplicates is up to you.
• This covers JSON text only. The binary readers (CBOR, MessagePack, UBJSON, BSON, BON8) have no escape sequences; their keys are the raw bytes from the format. I did not verify how each one treats invalid UTF-8.
• I did not check older 3.x releases.

---

Written by Claude Code on behalf of @nlohmann.

[bartygroch-gif]

Thank you for the explanation. Could you clarify the scope and status of the described behavior for JSON text input?

1. Which commit ID or tagged release does “current develop” refer to in your answer?
2. Which releases were checked, and are there any relevant configuration limitations? If a release or configuration was not checked, please indicate that.
3. For parser_callback_t (the key event) and json_sax::key separately, are the described behaviors intended as guarantees of the public API contract, or observations of the implementation at the identified revision?
4. If documentation explicitly stating these guarantees is already available, could you link to it?

This follow-up is limited to JSON text input and seeks clarification of the version and configuration scope and the status of the described behavior.

[nlohmann]

Thanks for the follow-up, and sorry for the vague "current develop". Answers in order:

1. Revision. My first answer was based on develop at 54f6b3e, which is the base of the PR below. It is unreleased (3.13.0).
2. Releases checked. I ran the same key inputs (escapes, é vs é, invalid escapes, lone surrogates, control characters, ill-formed UTF-8, a missing :, duplicate names) against the single headers of 3.0.0 through 3.12.0 with g++ (C++17) and default settings. The tags I built were 3.0.0, 3.1.0–3.8.0, and 3.9.0 onwards, but not every patch release.
   • The key event of a parser_callback_t behaves identically from 3.0.0 on.
   • json_sax::key behaves identically from 3.2.0 on (the SAX interface was added in 3.2.0).
   • Binary-format keys behave identically in all releases that support the format: CBOR, MessagePack and UBJSON, plus BSON from 3.4.0.
   • BON8 is not in any release yet.
   • Not checked: releases before 3.0.0, other compilers, and non-default configurations.
3. Guarantee or observation. As of now it is an observation of the implementation at the revision above. PR Document the contract of object keys passed to json_sax::key and key callbacks #5956 turns it into documented behavior and adds a test case that covers both interfaces, so once it is merged it is part of the documented contract.
4. Documentation. Document the contract of object keys passed to json_sax::key and key callbacks #5956 adds the contract to api/json_sax/key.md and api/basic_json/parser_callback_t.md, with a pointer from features/parsing/sax_interface.md. It also has a version-history note covering the points in 2.

One correction to scope: my first answer excluded binary formats, but the PR documents them too. For CBOR, MessagePack, UBJSON and BSON, keys are passed through as stored bytes without a UTF-8 check. BON8 rejects ill-formed names.