v0.14.0

Trust and tooling release: industry-style test suite, debug observability, and a
measured Linux Kemal re-cut. Collector throughput unchanged; Kemal post-GC RSS
now measured (not estimated).

Performance

• Linux Kemal (same-host median-of-3, wrk -c 100 -d 30, scrub on): /json ~89% of Boehm @ ~0.79× post-GC RSS; / ~89% @ ~0.78×. Session bench/log/linux/2026-07-29-035426/. See docs/PERF.md (Linux). Fat-app (acikturkiye) not re-cut — still ~93% thr / ~2.65× RSS est. (ACIKTURKIYE.md).

Added

• Debug invariant checker (GCRY_DEBUG_INVARIANTS=1): validates heap invariants at runtime -- live_objects counter accuracy, freelist cycle/consistency checks, chunk index integrity, and block overlap detection. Hooks into malloc, free, and collect. Diagnostics use write(2) / no managed-heap alloc (not a claim that GC is async-signal-safe). -Dgcry_invariant_abort for core dumps. Exposed Heap#each_chunk, #freelist_for, #nursery_freelist_for for the checker. CI runs invariants on every PR. (spec/invariant_spec.cr, make invariants, CI Debug invariants step.)
• Coverage infrastructure: spec/all_specs.cr entrypoint for kcov (DWARF-based line/branch coverage). ci/coverage.sh wrapper runs kcov + crystal tool unreachable + crystal tool macro_code_coverage. make coverage / coverage-kcov / coverage-unreachable / coverage-macro targets. CI coverage job builds the spec binary, installs kcov from Debian, and uploads the report. (ci/coverage.sh, Makefile, .github/workflows/ci.yml)
• Memory safety CI: make asan builds and runs specs with AddressSanitizer (-Dasan). make valgrind-samples runs samples under Valgrind memcheck (--leak-check=full). CI asan and valgrind jobs on every PR. (Makefile, .github/workflows/ci.yml)
• Deterministic replay fuzzing: bench/fuzz.cr rewritten with --seed=, --seconds=, --log=, and --replay= flags. Fuzz logs every operation to a replayable log file (opcode + args). Replay mode reads the log and replays the exact sequence of heap operations. Op 9 (spawn + Channel) excluded from logs as non-deterministic Crystal runtime. CI runs fuzz + replay on every PR. (make fuzz-replay FUZZ_LOG=path, CI Fuzz with log + replay step.)
• Property-based testing: bench/property_test.cr -- random alloc/free/collect sequences with deep heap invariant verification: live_objects counter accuracy (reported == walked count), heap_size == sum of chunk mapped_bytes, freelist consistency, and per-node live? assertion. 100k iterations in ~8s. (make property-test, CI Property test step.)
• Layout property test: bench/layout_property_test.cr -- 5 self-contained sub-tests verifying precise scan offset correctness, conservative fallback, leaf layout (scan_cap=0), noscan offset keep-alive semantics, and scan_cap limiting. Runs 10k iterations in ~2.5s. (make layout-property-test, CI Layout property test step.)
• MT property test: bench/mt_property_test.cr -- concurrent allocation via fiber workers (2, 4, 8) with periodic collect; verifies no objects lost under concurrent alloc, live_objects counter accuracy after TLAB flush, and parallel mark (workers=2) produces the same live set as serial mark (workers=1). 500 iterations × 3 worker counts in ~2.4s. (make mt-property-test, CI MT property test step.)
• 24-hour soak test: bench/soak.cr -- sustained load with alloc storm (~1000 obj/s), periodic collect (1 Hz), fiber spawn (10 Hz), finalizer load (100 obj/s), WeakRef via disappearing links (10 Hz). Hourly telemetry: heap size, free bytes, live objects, pause p50/p99, RSS. Post-soak RSS check (< 10% growth) and drain verification. Weekly CI cron (Monday 06:00 UTC). (make soak, CI soak job.)
• Alloc pattern fuzzing: bench/pattern_fuzz.cr -- 3 allocation distributions (Zipfian power-law, bimodal small+large, stride array-growth) each checked against baseline uniform-random. Verifies pause p99 < 8-10x baseline and RSS growth < 10%. 200 phases × 5000 objects per phase. (make pattern-fuzz, CI Alloc pattern fuzz step.)
• Thread storm test: bench/thread_storm.cr -- 3 phases: thread spawn storm (OS threads doing alloc/free/collect in batches), rapid thread create/destroy (250 short-lived threads), Crystal Signal.trap deferred alloc (event-loop mutator path; GC is not async-signal-safe — see POLICY.md). 1000+ iterations total, 0 errors. (make thread-storm, CI Thread storm step.)
• OOM scenarios: bench/oom_test.cr -- 3 phases: bounded heap (low gc_threshold, 500 iterations, no crash), mmap failure (graceful OutOfMemoryError), finalizer under OOM (no crash under pressure). (make oom-test, CI OOM test step.)
• Bug-fix test policy: CONTRIBUTING.md with "bug fix must include test" rule, .github/PULL_REQUEST_TEMPLATE.md with reproducing test checkbox, and spec/regression/ directory with 4 regression tests (live_objects dormant chunk, hash_layout entries_size, scan_cap alloc_size mismatch, signal_stack false root). (spec/regression/, CI regression jobs.)
• API misuse test suite: spec/api_misuse_spec.cr -- tests covering GC.free(null), GC.realloc(null, 0), GC.malloc(0), GC.malloc_atomic(0), Gcry.add_root(null), Gcry.register_disappearing_link(null, ...), collect inside finalizer (no deadlock), Crystal Signal.trap deferred alloc (Linux; not async-signal-safe), add_root with large pointer, alternating malloc/free. (make spec, CI spec step.)
• Fork reinit test: bench/fork_reinit.cr -- standalone LibC.fork + after_fork_child_reinit + alloc in child + parent continues allocating after collect. 3 assertions, all pass. (make fork-test, CI Fork reinit test step.)
• Finalizer complex scenarios: bench/finalizer_complex.cr -- 7 phases: finalizer chain, finalizer calling GC.collect, finalizer adding root (resurrection), finalizer + disappearing links interaction, finalizer under heavy allocation pressure (500 objects), finalizer creating 1000 objects, and many disappearing links (200). 8/8 assertions pass. (make finalizer-complex, CI Finalizer complex scenarios step.)
• Perf regression alerting: bench/perf_smoke.sh rewritten with variance protocol -- 5 wrk runs per path, min/max discarded, median reported, noise ratio computed (IQR/median). same-host variance protocol (N wrk runs, min/max discard, median, noise ratio); gate is gcry /json % of Boehm only. Absolute RPS is not compared across hosts. Per-run JSON under bench/log/ uploaded as CI artifact. (bench/perf_smoke.sh, CI perf smoke job.)
• Microbenchmark suite: bench/micro/run_all.cr -- 6-phase suite measuring alloc latency (10 size classes, p50/p99/max), free latency, collect latency (5000 obj, p50/p99/max), TLAB refill cost, STW suspend/resume latency, and GC lock overhead. Runs in < 10s. (make microbench, CI Microbenchmark suite step.)
• Pause time budget: bench/pause_budget.cr -- major p99/max budgets scaled to live set, incremental collect_a_little slice budget (STW-aware), minor vs major pause ratio. (make pause-budget, CI Pause budget step.)
• RSS leak detection: bench/rss_leak.cr -- cyclic alloc/free/collect; gate is intra-run RSS growth only (late-half vs early-half <10%). RSS/heap ratio is informational. Writes gitignored bench/trend.json. (make rss-leak, CI RSS leak detection step.)
• Darwin platform parity tests (Phase 6.1): spec/platform_darwin_spec.cr asserts soft-dirty/mprotect stubs return unsupported, pthread_get_stackaddr_np stack bounds contain the current SP, and host-page-aligned MADV_FREE_REUSABLE reclaim works. process_spec Darwin section exercises Mach thread_suspend/resume STW round-trip + SP clamp under -Dgc_none. Windows process-GC gap documented in docs/INTEGRATION.md (crystal#15173 HeapAlloc stub ≠ gcry port).
• Compiler GC contract (Phase 6.3): bench/compiler_gc_contract.cr mirrors Crystal spec/std/gc_spec.cr (stats/prof_stats/enable) plus malloc/realloc/collect, disable/enable, and runtime @crystal_type_id vs crystal_instance_type_id. CI also runs crystal tool hierarchy / unreachable on gcry sources. (make compiler-gc-contract)
• Kemal E2E (Phase 6.4): bench/kemal_e2e.sh hits every endpoint (/, /json, /gc-collect, /gc-stats, /metrics) before and after concurrent wrk load. CI runs 60s; full 10-min DoD via KEMAL_E2E_DURATION=600 make kemal-e2e.
• GC trace log (Phase 7.1): GCRY_TRACE=1 emits NDJSON events (alloc/free sampled, collect_start/collect_end, finalizer, barrier_arm) via Gcry::Trace. Reentrancy guard avoids malloc recursion. (make trace-smoke, spec/trace_dump_spec.cr)
• Heap dump (Phase 7.2): Gcry.dump_heap(io) / dump_heap_addresses / heap_dump_gone/new for live-object NDJSON and leak diffs. Dump count matches live_objects.
• Mutation harness (Phase 7.3): bench/mutations/run.sh — 10 hand-crafted sed mutants; kill suite scores 10/10. Feasibility notes in docs/MUTATION.md.

Fixed

• Gcry::Trace under -Dgc_none: do not require "json" or write via abstract IO — both pulled JSON/OpenSSL into the GC bootstrap and broke process builds. Trace now emits NDJSON with a stack buffer + LibC.write to a raw fd.

• Darwin release_physical_pages spec: do not assert immediate zero-fill after MADV_FREE_REUSABLE (kernel may keep contents until reclaim). Assert aligned success + still-mapped only.

• Nursery HTTP::Headers regression: moved from process_spec to standalone bench/nursery_headers.cr — Spec + process GC + nursery was flaky on CI (SEGV during Spec reporting).

• Process parallel mark: moved from process_spec to bench/parallel_mark_process.cr for the same Spec+process-GC flake; CI retries process_spec up to 3 times.

• live_objects counter drift on dormant chunks: the counter was not updated when a fully-free chunk was marked DORMANT during sweep, causing the invariant checker to flag a mismatch (actual=6502, reported=1). Discovered by the new invariant checker. Covered by spec/regression/1_live_objects_dormant.cr.

• after_fork_child_reinit stability: LibC.fork + reinit + alloc in child, parent continues after collect. Covered by bench/fork_reinit.cr.

Changed

• Signal policy clarity: GC is not async-signal-safe (POLICY.md). Crystal Signal.trap is deferred (event loop); tests/docs no longer claim handler-safe GC.malloc.