Skip to content

Add AVX2 escape kernels with runtime dispatch - #45

Merged
lpgauth merged 1 commit into
mainfrom
avx2-escape
Jul 1, 2026
Merged

Add AVX2 escape kernels with runtime dispatch#45
lpgauth merged 1 commit into
mainfrom
avx2-escape

Conversation

@lpgauth

@lpgauth lpgauth commented Jul 1, 2026

Copy link
Copy Markdown
Owner

Summary

  • Add AVX2 (32-byte lane) variants of both string-escaping kernels in escape.rs: the escape-only path used for atom keys and the fused UTF-8-validate + escape path used for binary strings. They mirror the SSE2 structure exactly — classify control/"/\ bytes with a vector compare, bulk-store clean chunks, handle the first flagged byte via QUOTE_TAB, and fall into byte-by-byte validation on non-ASCII chunks.
  • Dispatch selects AVX2 at runtime via is_x86_feature_detected!("avx2") (cached by std_detect after the first call), since precompiled binaries target baseline x86-64. SSE2 remains the fallback and also handles the sub-32-byte tail after the AVX2 main loop. The aarch64/NEON and scalar paths are untouched.
  • Verified with a standalone harness that cross-checks every SIMD kernel against the scalar reference on 24,362 cases per arch: escapable bytes at every position around the 16/32/64-byte boundaries, multi-byte UTF-8 sequences sliding across lane boundaries, invalid-UTF-8 rejection, canary checks for writes past the reported length, and 20k fuzzed inputs. Runs: native aarch64 (NEON), x86_64 under Rosetta (SSE2 path), and x86_64 under QEMU -cpu max (AVX2 path actually executed) — all kernels agree with scalar everywhere.

x86_64 escaping previously used only baseline SSE2 (16-byte lanes).
Add AVX2 variants of the escape-only and fused validate+escape
kernels that scan 32 bytes per iteration, selected at runtime via
is_x86_feature_detected! (cached by std_detect) since precompiled
binaries target baseline x86-64. SSE2 remains the fallback and
handles the sub-32-byte tail.

Verified against the scalar reference on 24k boundary-adversarial
and fuzzed cases per arch (native NEON, Rosetta SSE2, QEMU AVX2).
@lpgauth
lpgauth merged commit a7fd0d9 into main Jul 1, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant