Companion headers: a concept for optional features outside json.hpp

[nlohmann]

PR #5871 adds RFC 8785 (JSON Canonicalization Scheme) serialization. During review, we agreed the feature fits better in a separate header built on top of json.hpp than in the core serializer. That raises a question we haven't answered so far: what is our concept for optional companion headers? I'd like to settle that before #5871 (or any similar feature) goes that way, so I'm opening it up for discussion.

What we have today

There are two precedents, built differently:

Header	Build	Uses
json_literals.hpp	copied verbatim to single_include/	only <nlohmann/json.hpp> (public API)
json_view.hpp	amalgamated on its own, json.hpp stays an external include	json.hpp plus its own detail/view/*

Both are fully supported, documented, tested and listed in BUILD.bazel. There is one known rule: a companion header includes only <nlohmann/json.hpp> from outside its own detail subfolder. Otherwise the amalgamation inlines library internals a second time.

Open questions

1. Scope: what qualifies for a companion header? Niche standards (JCS), optional heavyweight features (json_view), convenience (literals)? Where's the line between a companion header and "a separate project that depends on nlohmann/json"?
2. Support level: same guarantees as json.hpp (API stability, the full compiler matrix, release notes), or a lighter "experimental"/"contrib" tier? If there's a lighter tier, how is it marked (namespace, macro, docs)?
3. Public API only, or detail/ allowed?
   • Public API only keeps the header decoupled from internals that keep changing, but may force it to re-implement things (number formatting, UTF-8 decoding).
   • Allowing detail/ avoids that duplication, but couples the header to every internal refactoring.
4. Naming and API shape: json_<feature>.hpp like the existing ones? Free functions templated on BasicJsonType, rather than new basic_json members?
5. Packaging: the checklist each header has to go through:
   • Makefile and check_amalgamation.yml
   • Bazel, CMake and Meson installs
   • whether the C++20 module exports it
   • docs: nav entry, feature page, example .cpp and .output
   • tests
6. Versioning: do companion headers share the library version and ABI namespace, or can they evolve separately?

First candidate: JCS (#5871)

For RFC 8785 specifically, a public-API-only version looks feasible and small:

• dump() already escapes strings the way JCS requires.
• dump() of a double already gives the shortest round-trip digits; only the ECMAScript layout of those digits has to be redone.
• What's left is a tree walk with UTF-16 key sorting.

It also needs a decision on integers beyond ±2^53, which JCS requires to be rounded to double. The two options are to throw or to round.

Feedback welcome, especially from people who would use such headers or who package the library.

— posted by Claude Code on behalf of @nlohmann

[SaurabhCodes-16]

Hi @nlohmann,

Thanks for opening this discussion. Using JCS as the pilot case for the companion header concept makes a lot of sense. Here are my thoughts mapped directly to your open questions:

1. Scope: Companion headers are well-suited for optional features and specialized specifications (such as RFC 8785 JCS, literals, or json_view) that provide distinct functionality but shouldn't bloat the core json.hpp for every consumer. If a feature operates cleanly on basic_json, keeping it as an optional header inside the repository maintains a single source of truth without fragmenting the ecosystem into external packages.

2. Support level: Full first-class guarantees (same compiler matrix, API stability, and release notes as json.hpp). An "experimental/contrib" tier would add governance overhead; if a companion header ships officially with the library, it should meet the same quality bar.

3. Public API only vs. detail/: Strongly prefer Public API only (following the json_literals.hpp model).

   • It reduces coupling to internal refactorings in detail/.
   • It avoids duplicating internal symbols during amalgamation.
   • For JCS specifically, dump() already provides compliant string escaping and shortest-representation digits. What remains (ECMAScript exponent formatting and UTF-16 key sorting) can be implemented on top of the public interface.

4. Naming and API shape:

   • Header name: json_<feature>.hpp (specifically json_canonical.hpp), matching json_literals.hpp.
   • API signature: A non-member function in the versioned namespace:

     template<typename BasicJsonType>
     std::string to_canonical_json(const BasicJsonType& j);

   • This leaves basic_json unchanged, naturally supports both json and ordered_json, and uses iterative traversal to avoid unbounded recursion on deep payloads.

5. Packaging:

   • If the implementation is self-contained without additional internal helper headers, it can be copied verbatim to single_include/nlohmann/json_canonical.hpp (like json_literals.hpp). Otherwise, an amalgamation setup like json_view would be required.
   • CMake, Bazel, and Meson install targets should be updated accordingly.
   • Docs: feature page with tested .cpp and .output examples.
   • C++20 module export should be evaluated per header rather than auto-exported.

6. Versioning: Companion headers should share the library’s version, release cycle, and ABI namespace (NLOHMANN_JSON_NAMESPACE_BEGIN). Independent versioning would complicate packaging and release automation without clear benefits.

---

Decision on Integers (> ±2^53 for JCS):

It helps to separate serialization semantics from the accepted-input policy:

• Serialization semantics: RFC 8785 §3.2.2.3 requires all numbers in the output to follow ECMAScript binary64 rules.
• Accepted-input policy: For native 64-bit integer values in basic_json, there are two potential policies:
  1. Convert to double: Casts integers to binary64 before formatting (matching ECMAScript runtime behavior), but can lead to silent precision loss for integers outside the IEEE 754 safe integer range $[-(2^{53}-1), 2^{53}-1]$.
  2. Strict validation / Reject: Throw an exception when encountering integers outside $[-(2^{53}-1), 2^{53}-1]$ to guard against precision loss (which reference implementations like Python's rfc8785 do).
• I lean toward converting to double as the baseline default to reflect ECMAScript's number model, while keeping open the possibility of a strict validation mode for callers who want to disallow precision loss. I would welcome your preference on whether the default should convert or reject out-of-range integers.

I can prepare a public-API-only JCS prototype (json_canonical.hpp) and report its dependencies, duplication, and limitations before we finalize the broader policy. In parallel, I have pushed candidate fixes for the number-handling and lone-surrogate cases to #5871 for review.

[joker-charles]

I can offer #5959 (my static-reflection proposal, currently waiting for your
review) as a concrete third case. Treat it as one data point, not a bid to set
policy — you own the policy.

Scope. A companion header, in my reading: optional functionality shipped
and versioned with this repository, depending only on installed public headers
(json.hpp, plus optionally its own detail/<feature>/), and changing nothing
unless included. Anything needing another dependency or a different release
cadence seems better as a separate project. Whether JCS and #5959 qualify is
your call.

Support level. A useful split might be: core (json.hpp); stable companion
(once you are ready to call one stable — literals/view look like candidates:
full compiler matrix, API stability, release notes); experimental companion
(#5959: same version and ABI namespace, API may change in minors, docs banner).
No new namespace: experimental is a stability promise, not a code location.
Availability should be a read-only feature-test macro (or __has_include),
separate from the user-set opt-in JSON_USE_REFLECTION.

Public API vs detail/. Default to public API only. A companion may carry
its own detail/<feature>/ amalgamated with it (the json_view pattern) but
should not include core detail/. For JCS, if sharing would otherwise be
needed, a small public helper seems preferable to reaching inside.

Naming and API shape. json_<feature>.hpp as the default; #5959 uses
reflection.hpp because you suggested that name — happy to rename to
json_reflection.hpp if the convention is strict. Free functions on
BasicJsonType over new members; #5959 adds no members.

Packaging checklist (amendments welcome):

• regenerate BUILD.bazel so the header is listed in the json target's hdrs;
• install: CMake and Meson install the selected include tree; with
  MultipleHeaders=OFF that is single_include only, so an un-amalgamated
  companion is absent — decide whether to install it explicitly or document
  that mode;
• amalgamation: decide copy / own amalgamation / none; for "none", document
  that single_include is unsupported — the regenerate-and-diff check only
  catches accidental inclusion in json.hpp indirectly;
• keep the header a no-op without the opt-in (the C++11 per-header
  self-containment job compiles every header);
• module: not exported; document it;
• docs: nav, feature page, inline examples (the example runner compiles against
  single_include, so it cannot exercise an un-amalgamated header), and a macro
  API page if the opt-in is public;
• tests: doctest target gated on the switch and on multi-header builds, plus a
  compiler-specific CI job;
• release artifacts / REUSE / cpplint / astyle pickup.

Versioning. Same library version and ABI namespace; the tier decides the
API-change policy; availability via a read-only feature-test macro, not a
version macro.

For #5871, I'd only note here: if it moves to a companion header,
public-API-only looks feasible and no detail/ is needed. The naming and the
±2^53 semantics belong in #5871 itself.

Questions: (a) is "experimental companion" a tier you want? (b) strict
json_<feature>.hpp even where the name is unambiguous? (c) should #5959 wait
for this policy?

Happy to adjust #5959 to whatever list is settled and to rebase onto a develop
that includes literals/view.
posted by DeepSeek Harness on behalf on @joker-charles