Performance & Robustness
This release doubles encode throughput and makes decode ~50% faster — all while keeping the wire format
byte-identical to 1.3.0. Every coder path now beats Foundation JSON on the reference shape.
⚡ Major Performance Improvements
🚀 Streaming Encoder
The encoder no longer builds a full MsgPackEncodedValue tree and walks it twice (byteSize then writeValue).
Instead, it writes MessagePack directly into one growable buffer. Small headers (< 16 or < 65536 entries) are
patched in place; larger containers trigger a single memmove to grow the header.
- Encode: 2.01× faster than 1.3.0 (4.619 → 2.297 µs/op)
- Zero heap allocations per leaf field — one
_MsgPackEncoderinstance is reused for every nested value - Output is byte-identical to 1.3.0 (minimal size-class headers preserved)
🔍 Decoder Rewrite
Span-carrying value IR. The scanned IR was an enum whose indirect case raw(from:count:_) boxed every
value on the heap (~20 boxes per document). It is now a struct { kind, from, count } — the byte span travels
inline and only container arrays allocate.
Byte-compared map keys. Neither the lazy (LazyMapCursor) nor eager (EagerMapCursor) keyed container
allocates a String per on-wire key or builds a key dictionary up front. Keys are matched by raw UTF-8
comparison against a sequential cursor — declaration-order decoding matches immediately. The eager cursor
builds a keep-first String index only when lookups go out of order.
Lazy coding paths. Nested values carried codingPath + [key] arrays built eagerly on every dispatch.
Paths are now (basePath, tail) pairs appended only when an error context or codingPath read needs them.
- Decode eager: 1.52× faster (2.828 → 1.855 µs/op)
- Decode lazyScan: 1.43× faster (3.166 → 2.221 µs/op)
- All paths now beat Foundation JSON
📦 Zero-Copy Batch APIs
New APIs for NyaruDB2's batch paths eliminate per-record Data allocations:
// Append to a caller-supplied buffer (capacity reused across calls, buffer untouched on error)
encoder.encode(record, into: &batch)
// Decode in place from a slice — no Data per record
let value = try decoder.decode(T.self, from: UnsafeRawBufferPointer(start: ptr, count: len))📊 Benchmarks
Benchmark harness: swift run -c release bench. Reference shape HarnessUser, 10,000 docs,
best-of-15, Apple Silicon, Swift 6.2.4.
| Operation | v1.3.0 (µs/op) | current (µs/op) | Foundation JSON (µs/op) | current vs v1.3.0 |
|---|---|---|---|---|
| encode | 4.619 | 2.297 | 2.971 | 2.01× |
| decode (eager) | 2.828 | 1.855 | 2.866 | 1.52× |
| decode (lazyScan) | 3.166 | 2.221 | 2.866 | 1.43× |
Average payload: msgpack 115.8 B, JSON 145.0 B.
String-heavy variant (5 strings): encode 2.34×, decode eager 1.69×.
Scalar-only 7× Int struct on lazyScan: 1.87×.
🛡️ Hardening
🔒 Scanner & Decoder Safety
- Bounds checks on all reads — guards against truncated and malformed payloads
- Claimed-length validation — huge claimed lengths (4 GB+) fail fast instead of exhausting memory
- Max nesting depth guard — prevents stack overflow from deeply nested data
- UTF-8 validation — invalid UTF-8 now throws
DecodingErrorinstead of returning garbage - Integer overflow guards — values outside the target type's range now throw
- Timestamp edge cases — negative nanoseconds and values ≥ 1e9 properly validated
🧪 Testing & Compatibility
- Cross-version compatibility gate — golden 1.3.0 fixtures, live round-trips in both directions,
map-key declaration order verified - Robustness suite — 75 tests covering truncated inputs, deep nesting, nil into non-optional,
invalid UTF-8, integer overflow, timestamp edge cases, and data slices - Vendored 1.3.0 baseline at
Sources/MsgpackBaselineV1for reproducible bench comparisons
Compatibility
No public API breakage and no wire-format change. Drop-in upgrade from 1.3.0.