Skip to content

1.4.0

Latest

Choose a tag to compare

@maltzsama maltzsama released this 09 Jul 09:11

Performance & Robustness

This release doubles encode throughput and makes decode ~50% faster — all while keeping the wire format
byte-identical to 1.3.0. Every coder path now beats Foundation JSON on the reference shape.

⚡ Major Performance Improvements

🚀 Streaming Encoder

The encoder no longer builds a full MsgPackEncodedValue tree and walks it twice (byteSize then writeValue).
Instead, it writes MessagePack directly into one growable buffer. Small headers (< 16 or < 65536 entries) are
patched in place; larger containers trigger a single memmove to grow the header.

  • Encode: 2.01× faster than 1.3.0 (4.619 → 2.297 µs/op)
  • Zero heap allocations per leaf field — one _MsgPackEncoder instance is reused for every nested value
  • Output is byte-identical to 1.3.0 (minimal size-class headers preserved)

🔍 Decoder Rewrite

Span-carrying value IR. The scanned IR was an enum whose indirect case raw(from:count:_) boxed every
value on the heap (~20 boxes per document). It is now a struct { kind, from, count } — the byte span travels
inline and only container arrays allocate.

Byte-compared map keys. Neither the lazy (LazyMapCursor) nor eager (EagerMapCursor) keyed container
allocates a String per on-wire key or builds a key dictionary up front. Keys are matched by raw UTF-8
comparison against a sequential cursor — declaration-order decoding matches immediately. The eager cursor
builds a keep-first String index only when lookups go out of order.

Lazy coding paths. Nested values carried codingPath + [key] arrays built eagerly on every dispatch.
Paths are now (basePath, tail) pairs appended only when an error context or codingPath read needs them.

  • Decode eager: 1.52× faster (2.828 → 1.855 µs/op)
  • Decode lazyScan: 1.43× faster (3.166 → 2.221 µs/op)
  • All paths now beat Foundation JSON

📦 Zero-Copy Batch APIs

New APIs for NyaruDB2's batch paths eliminate per-record Data allocations:

// Append to a caller-supplied buffer (capacity reused across calls, buffer untouched on error)
encoder.encode(record, into: &batch)

// Decode in place from a slice — no Data per record
let value = try decoder.decode(T.self, from: UnsafeRawBufferPointer(start: ptr, count: len))

📊 Benchmarks

Benchmark harness: swift run -c release bench. Reference shape HarnessUser, 10,000 docs,
best-of-15, Apple Silicon, Swift 6.2.4.

Operation v1.3.0 (µs/op) current (µs/op) Foundation JSON (µs/op) current vs v1.3.0
encode 4.619 2.297 2.971 2.01×
decode (eager) 2.828 1.855 2.866 1.52×
decode (lazyScan) 3.166 2.221 2.866 1.43×

Average payload: msgpack 115.8 B, JSON 145.0 B.

String-heavy variant (5 strings): encode 2.34×, decode eager 1.69×.
Scalar-only 7× Int struct on lazyScan: 1.87×.

🛡️ Hardening

🔒 Scanner & Decoder Safety

  • Bounds checks on all reads — guards against truncated and malformed payloads
  • Claimed-length validation — huge claimed lengths (4 GB+) fail fast instead of exhausting memory
  • Max nesting depth guard — prevents stack overflow from deeply nested data
  • UTF-8 validation — invalid UTF-8 now throws DecodingError instead of returning garbage
  • Integer overflow guards — values outside the target type's range now throw
  • Timestamp edge cases — negative nanoseconds and values ≥ 1e9 properly validated

🧪 Testing & Compatibility

  • Cross-version compatibility gate — golden 1.3.0 fixtures, live round-trips in both directions,
    map-key declaration order verified
  • Robustness suite — 75 tests covering truncated inputs, deep nesting, nil into non-optional,
    invalid UTF-8, integer overflow, timestamp edge cases, and data slices
  • Vendored 1.3.0 baseline at Sources/MsgpackBaselineV1 for reproducible bench comparisons

Compatibility

No public API breakage and no wire-format change. Drop-in upgrade from 1.3.0.