Release v1.0.1

Jsonifier v1.0.1 Release Notes

🏛️ Cathedral-Grade Rewrite: Structural Indices, Bool-Propagating Control Flow, and Full UTF-8 Validation

This release touches nearly every layer of the library — parsing, serialization, SIMD backends, containers, and error handling all got rebuilt around a few unifying architectural decisions. It's the largest single-release change set in the project's history.

---

🚀 Highlights

Structural indices replace raw pointers throughout. The structural index array (used by the parser, serializer, minifier, and prettifier alike) now stores uint32_t offsets into the source buffer instead of raw 8-byte pointers. This halves the structural index array's memory footprint and improves cache behavior across every major operation — parsing, minifying, prettifying, and serializing.

Parsing, validation, minification, and prettification all now return bool. Every core operation propagates failure immediately via [[likely]]/[[unlikely]]-guarded early returns instead of the old pattern of pushing errors to a side-channel and continuing to parse garbage. This is both a correctness improvement (no more wandering through a broken buffer after the first failure) and a performance one (failed parses bail immediately).

Full UTF-8 validation, on by default. A new Lemire/simdjson-derived UTF-8 validator ships with both a bulk validateUtf8() path and a streaming register-based validator that carries state across chunk boundaries — built for validating strings incrementally during parsing rather than requiring the whole buffer up front. parse_options::validateUtf8 defaults to true. Backed by an extensive new test suite covering the full Markus Kuhn UTF-8 stress corpus, alignment-sweep fuzzing, and page-boundary overrun checks via mmap/mprotect.

New SVE2 backend for ARM. Jsonifier now has a full SIMD backend for ARM's SVE2 instruction set, covering both low-level primitives and the higher-level structural/whitespace/string stage1 detection layer. ⚠️ This backend is experimental and not yet fully correct — a handful of parsing cases still misbehave under SVE2. Treat it as a preview; NEON remains the recommended path on ARM for production use until this shakes out.

AVX-512 gating tightened. AVX-512 codepaths now require F + BW + VBMI2 support to be detected together, rather than just F — a real safety fix for hardware that reports partial AVX-512 support.

PCLMULQDQ detection added, plumbed through CMake and the runtime CPU-feature detector as a new instruction-set flag (JSONIFIER_CLMUL).

---

🧠 Parser

• Object member dispatch got a tighter known-order fast path: for minified + known-key-order parsing, key matching now fuses comma + quote + key + quote + colon into a single compile-time literal match instead of checking each punctuation character separately.
• Vector parsing now parses into a reusable thread-local scratch buffer (reusing existing capacity where possible) and only moves the result into the caller's container once the full parse succeeds — avoids reallocation churn and ensures partial failures never leave the destination half-written.
• Tuple parsing unrolled via fold-expression-based functor_runner instead of deep recursive template instantiation — better compile times and codegen on larger tuples.
• The hash-map fallback path's anti-hash-state cache now only updates on a genuinely successful match, fixing a subtle bug where a failed parse could poison the cache for subsequent lookups.
• New parse_options fields: maxDepth (default 1024, guards against runaway/malicious nesting) and nullTerminated.
• The old fully-separate parse_partial_impl template hierarchy has been merged into the main parse_impl specializations, cutting a large amount of duplicated logic.

✍️ Serialization

• New branchless character-escaping via direct table lookup, replacing the old per-character switch statement.
• Fixed a real big-endian correctness bug in bool serialization ("true"/"false" packed-integer constants are now computed with explicit endian-aware byte ordering instead of a little-endian-only hardcoded value).
• indent_table was substantially simplified: a single padded memcpy (with memset overflow handling beyond 8 levels of depth) replaces the old per-depth offset-table lookup, removing a template parameter from every serializer specialization that used it.
• Tuple serialization unrolled via the same functor_runner pattern as parsing.
• Single-element raw arrays now skip loop setup entirely via if constexpr.

🧬 SIMD / CPU Detection

• CPU feature detection was rewritten to emit structured, greppable output (HAS_X=0/1, CPU_MASK=, SVE2_VL_BITS=) instead of an 8-bit exit-code bitmask, and the CMake side now supports proper cross-compilation via a JSONIFIER_CPU_INSTRUCTIONS override.
• opTest semantics were corrected (previously inverted) — this was fixed at every call site, including anyBitsSetAnywhere.
• New cross-backend operations: opSubs, opPermute, opAlignR, opSrLi, opPrev (cross-register byte-boundary carry, critical for the UTF-8 validator), isAscii, orAll.
• BMI/LZCNT/POPCNT primitives rewritten to branch per compiler and architecture, always falling back to std::countr_zero/countl_zero/popcount as a safety net.
• Constexpr evaluation budgets were raised substantially across MSVC, Clang, and GCC to accommodate the heavier compile-time table generation elsewhere in the codebase.

📦 Containers

• Fixed a real bug in jsonifier::internal::array::end() — it previously returned the same iterator as begin(), meaning naive iteration never terminated correctly.
• Fixed array::at() to actually throw std::runtime_error on out-of-bounds access — it previously constructed the exception and silently discarded it without throwing.
• array gained compile-time-constant indexing (bounds-checked at compile time), element-wise operator==/!=, and deduction guides.
• Allocator rewritten around a self-describing allocation header stored ahead of the returned pointer, replacing size-recomputation-based deallocation. Added a Windows large-page fallback path and a non-AVX aligned-malloc path.

🧪 Testing

• Massive expansion of the unit test suite: nearly every test category now runs across all 8 combinations of partial × knownOrder × nullTerminated.
• New integration of the external JSONTestSuite conformance corpus (150+ cases).
• New dedicated test files for error construction/reporting, SIMD intrinsics correctness, and UTF-8 validation (including the full Markus Kuhn stress suite, alignment sweeps, and mmap-based page-boundary fault testing).
• Fixed a bug in the test runner's main.cpp where a thrown test exception would still result in a 0 (success) process exit code.

⚠️ Breaking Changes

• jsonifier_core's template parameter changed from a boolean (doWeUseInitialBuffer) to a uint64_t initialBufferSize (default 1MB) — buffer size is now directly configurable rather than toggled on/off.
• The standalone compareJson/json_comparator API has been removed. If you were using struct/document comparison, hold off upgrading until we confirm whether/where this lands.
• parse_status / validate_status / minify_status / prettify_status enums renamed and renumbered (parse_statuses, etc.), with several new members reflecting new failure modes (exceeded_max_depth, illegal_control_character, unexpected_token, and others).
• Several public headers were renamed from PascalCase to snake_case (Allocator.hpp → allocator.hpp, Config.hpp → config.hpp, Error.hpp → error.hpp, and many more) — update any direct includes accordingly.

---

Release v1.0.0

Jsonifier v1.0.0

A high-performance, RFC8259-compliant C++23 library for validating, parsing, serializing, prettifying, and minifying JSON — built on SIMD instructions and compile-time reflection for key lookups.

This release marks the first stable, fully-packaged, fully-tested cut of Jsonifier. Below is what landed.

---

Build System & CMake

• Modern CMake (3.28+) build targeting C++23, structured as an INTERFACE library with a jsonifier::Jsonifier alias.
• Modular CMake setup split into library_setup.cmake and installation_setup.cmake for clean separation of build and install logic.
• Comprehensive compile-definition matrix exposing arch (X64/ARM64), platform (Windows/Linux/Mac), and compiler (Clang/MSVC/GCC) as compile-time constants, plus configurable JSONIFIER_INLINE/JSONIFIER_CLANG_INLINE/JSONIFIER_LIFETIME_BOUND attribute macros that adapt per-compiler and per-config.
• Full CMakePresets.json covering Windows Release/Debug variants with Benchmarks, Tests, and ASAN permutations.
• Install/export target support via JsonifierConfig.cmake.in for find_package(Jsonifier CONFIG REQUIRED) consumption.

CPU Architecture Detection

• Automatic runtime architecture detection (JsonifierDetectArchitecture.cmake + main.cpp feature detector) probing for LZCNT, POPCNT, BMI1, NEON, AVX, AVX2, and AVX-512.
• Auto-generation of JsonifierCPUInstructions.hpp with the detected instruction-set bitmask and helper macros.
• Manual override available through JSONIFIER_CPU_FLAGS for fine-grained control.

Continuous Integration

• Cross-platform unit-tests.yml matrix: Ubuntu (Clang/GCC), macOS (Clang/GCC), and Windows (MSVC), running on every push and PR.
• Every build runs with ASAN + UBSAN enabled for memory-safety and undefined-behavior detection across all platforms.
• Automated vcpkg release pipeline (Construct-Vcpkg-Info.yml + PHP make_vcpkg.php/Vcpkg.php) that does a two-pass build to capture the correct SHA512 and open a downstream vcpkg PR on tagged release.
• Workflow-run cleanup automation (DeleteRuns.yml).

Packaging & Distribution

• vcpkg port (Vcpkg/ports/jsonifier) with full version history in the registry, supporting Windows/Linux/macOS x64.

Documentation

A complete documentation set landed under Documentation/, covering installation (vcpkg / FetchContent / source), reflection-based structure registration, parsing & serialization usage, validation, error handling, prettifying, minifying, partial reading, minified-JSON optimization, custom parse/serialize specialization, parsing arbitrary raw_json_data, runtime key exclusion, and CPU architecture selection.

Project Standards

• MIT licensed.
• Enforced code style via .clang-format (tabs, 180-column limit, C++11 braced-list style).
• RFC8259 compliance as a baseline guarantee.

---

Core Engine

Library Entry Points

The umbrella <jsonifier> header and Index.hpp aggregate the full public surface — parsing, serialization, prettifying, minifying, validation, the hash map, raw JSON data, string/SIMD utilities — behind a single include.

Parsing Engine

Full parse_impl and partial-read parse_partial_impl specializations spanning the entire type lattice: jsonifier objects, maps, vectors, raw arrays, tuples, strings, chars, enums, numbers, bools, null, variant, optional, and shared/unique/raw pointers, plus raw-JSON passthrough and skip. Object parsing uses a thread-local antiHashStates table to fast-path known-order keys, falling back to a compile-time hash map on miss, with a dispatch layer that inlines a fold for small member counts (≤6) and switches to a function-pointer jump table beyond that. Minified and prettified inputs each get their own code path so whitespace handling costs nothing when it isn't needed. The parser surfaces precise, source-located errors and supports parse-into-existing, parse-and-return, and parse-many variants.

Validation

A standalone structural validator walks objects, arrays, strings, numbers, bools, and null against RFC8259, emitting specific validate_status codes (missing colon, missing comma/closing brace, invalid number/string/bool/null, etc.) rather than a bare pass/fail.

Serialization

The serializer estimates output size at compile time via getPaddingSize and refines it at runtime with computeRuntimeSize, so the buffer is sized in as few reallocations as possible. Prettify indentation is emitted through a precomputed indent_table blitter that memcpys whole indent runs (with an overflow path for deep nesting) instead of looping per-space, and small fixed tokens go through packed char_blitter writes. A branchless bool serializer packs true/false via integer arithmetic and a single 5-byte store.

Minify & Prettify

Both operate directly on the SIMD structural index, classifying each structural via a 256-entry lookup table — the minifier back-tracks whitespace per token, and the prettifier maintains an explicit depth/state stack with customizable indent size and character.

---

Test Suite & Validation

Test Harness & Build Integration

• Dedicated jsonifier-unit-tests target wired into CMake, pulling in the rt-ut unit-test framework and benchmarksuite via FetchContent.
• Per-compiler sanitizer and warning configuration: Clang builds run under the full -Weverything/-Wpedantic/-Werror gauntlet (with noise-floor exclusions like -Wno-padded, -Wno-c++98-compat), GCC and MSVC get their own tuned -Wall/-Wextra/-Werror and /Wall//W4//WX matrices, all with ASAN/UBSAN plumbing.
• Smart sanitizer fallbacks: auto-disables on GCC/macOS where unsupported, warns and disables UBSAN under MSVC, and auto-detects the Homebrew GCC runtime path for libstdc++ rpath linking.
• BASE_PATH compile definition so tests can locate their JSON fixtures at runtime.

Real-World Parse/Serialize Coverage

Round-trip parse → serialize validation across a large set of representative real-world payloads, each with full reflection mappings: Apache Builds, Canada (GeoJSON), CITM Catalog, Discord, GitHub Events, Google Maps, Instruments, Marine IK, Mesh, Random, and Twitter (including dedicated partial-read variants for Twitter and the ABC structs). Each runs in both minified and prettified modes, and across both knownOrder = true and false paths.

Conformance Testing

RFC8259 conformance suite running 60+ fail cases and 27 pass cases from the jsonchecker corpus, with each test asserting the specific expected parse_status (e.g. Missing_Array_End, Invalid_Number_Value, Unfinished_Input, Invalid_String_Characters) rather than just pass/fail — so regressions in error classification get caught, not just error presence.

Type & Primitive Validation

• Float: 64 hand-picked edge cases including denormals, subnormal boundaries, e±308 extremes, the 9223372036854775808 rounding cliffs, and long-digit mantissa stress strings, each checked against an exact expected double.
• Integer / Unsigned: bounds testing right up to INT64_MIN/UINT64_MAX, overflow rejection (18446744073709551616), plus malformed-exponent and type-mismatch fail cases.
• String: 35 pass cases covering escape sequences, surrogate pairs (\uD834\uDD1E), control chars, and multi-codepoint emoji/ZWJ sequences, 35 expected decoded outputs, and 26 fail cases for malformed escapes, lone/inverted surrogates, and truncated input.

Robustness Testing

• Bounds/truncation: serializes a seed object, then byte-by-byte truncates from the tail, asserting that every truncated prefix fails validation — for both minified and prettified forms.
• Round-trip integrity: 27 round-trip files including unique_ptr members and move-only types (Obj3 with deleted copy).

Inline Type-Coverage Tests (main.cpp)

A broad battery of ~75 inline assertions covering partial-read mode, renamed keys (makeJsonEntity), std::optional present/absent, enums (as integers, in arrays, as map keys), nested structs, shared_ptr, tuples, nested maps, vectors of vectors, escaped keys, large numbers, Unicode, and special characters — each as both a standalone reflection test and a partial-read variant, with boundary-length padding sweeps (0–80 chars) to flush out off-by-one and SIMD-tail bugs.

Release v0.9.98

Hey everyone, just a new release with the following changes being primary:

• Removed dependence upon a compile-time script for CPU architecture selection.
• Implemented partial reading/writing.
• Added newly optimized float parsing algorithm.
• Added a new tuple implementation to significantly reduce compile time.

Full Changelog: v0.9.97...v0.9.98

Release v0.9.97

Hey everyone, just a new release with the following changes being primary:

• Added likely/unlikely to a bunch of if/else statements.
• Swapped std::copy_n for std::memcpy in a few places.
• Moved a couple masks for the Neon functions out into global scope.
• Replaced jsonifier_internal::const_iterator with jsonifier_internal::iterator.
• Moved the vector of prettifier states out of the function scope to the prettifier classes' scope.
• Separated jsonifier_internal::string_literal out into its own header.
• Moved a bunch of string-utility functions out into the derailleur class.
• Replaced the join function with combineLiterals.
• Modified the logic of the parse_impl classes.
• Implemented a compile-time jump table to improve parsing performance significantly.
• Replaced the float/double parsing/serializing methods with more conformant version.
• Modified the serializing code to no longer rely on hash-maps for serializing members.
• Marked a bunch of additional functions as noexcept.
• Added an ALWAYS_INLINE macro in addition to the INLINE one.

Release v0.9.96

Hey everyone, just a new release with the following changes being primary:

• Added support for directly prettifying the Json data as it's written.
• Fixed a string parsing issue with sizes.
• Added the ARM-NEON implementation for various string operations.
• Added new simd-based hashmap implementation to improve key find-performance.
• Added a new minimal-char-based hashmap implementation to improve key find-performance.

Release v0.9.95

Hey everyone, just a new release with the following changes being primary:

• Added support for Arm-Neon 128-bit SIMD intrinsics.
• Modified some lines to correct cleanliness.
• Updated the raw_json_data class to remove superfluous functions.
• Updated some of the error_codes.
• Modified the parser, validator, serializer, and minifier classes to reduce function call overhead.
• Refactored the usage of ascii_classes to instead utilize the json_structural_type enumeration.
• Modified the CPU-detection script.
• Added a new compare function to accelerate string comparisons.
• Added MSVC, CLANG and GNU macros.

Release v0.9.94

Hey everyone, just a new release with the following changes being primary:

• Implemented a prettify function that utilizes simd instructions.
• Implemented a minify function that utilizes simd instructions.
• Implemented a validate function that utilizes simd instructions.
• Implemented a function for parsing/serializing into std::variant types.
• Implemented a fix to a buffer-overflow.

Release v0.9.93

Hey everyone, just a new release with the following changes being primary:

• Refactored the simd_base classes' functions to use perfect forwarding.
• Updated some types.
• Fixed an issue related to parsing/serializing raw array types.
• Implemented a jsonifier_value_t for parsing/serializing single values.
• Updated the parsing and serializing string functions to utilize a fuller range of simd-types.

Release v0.9.92

Hey everyone, just a new release with the following changes being primary:

• Refactored the number-parsing functions to use lookup tables instead of subtraction operations.
• Refactored the simd_string_reader class.
• Refactored some functions with fold-expressions for manual loop unrolliing.
• Refactored the toChars and parseNumber functions.
• Removed some superfluous typedefs.
• Testing new simd logic for string parsing.
• Renamed some types to snake_case.
• Refactored the jsonifier_core, parser, and serializer classes to utilize the CRTP.
• Refactored the simd_structural_iterator and serialization_iterator classes to remain contained.

Release v0.9.91

Hey everyone, just a new release with the following changes being primary:

• Removed store and storeu functions from being class-contained to free-standing.
• Removed unnecessary branch statements from serialize_impl functions.
• Improved some of the preprocessor macros for setting architecture.
• Replaced a function that uses intrinsics with a non-intrinsic (but more rapid) version.
• Renamed parsestring to parseString,.
• Fixed a heap-buffer-overflow issue on two of the comparison functions.
• Added more implementation to the raw_json_data class.
• Added a dedicated concepts namespace.
• Added some string-to-number functions.
• Split the parseNumber function into multiple specializations to improve performance.
• Refactored the jsonifier_object_t and jsonifier_array_t related functions.
• Refactored the AVX-types into a statically-operating class instead of a containing one, to improve performance.
• Renamed a bunch of things.
• Added the structural_index_vector and buffer_string classes for parsing performance.
• Improved error output - for debugging the indices.
• Implemented new serialization logic for strings.
• New subtraction logic.
• Reorganized some of the headers.