🌐 Website · 📚 Documentation
Pure-Go SIMD drop-ins for the standard library — byte-identical fast paths
for encoding/base64, encoding/base32, encoding/hex, encoding/ascii85,
unicode/utf8, hash/adler32, hash/crc32, hash/crc64, strconv,
encoding/json.Valid, ASCII case folding, and the SIMD-amenable primitives
under and around them (match-finding, bit-packing, popcount, byte histograms,
Stream VByte integer codec, XXH3 hashing, float reductions for vector search,
dense []uint64 set ops, INT8 quantized dot products, and Myers bit-parallel
edit distance) — 21 repositories in all, plus a proof-of-concept 7th
(wasm-SIMD) target in matchlen-wasm.
Every hot loop is real SIMD assembly generated by
go-asmgen across all six of Go's
64-bit SIMD targets — amd64, arm64, riscv64, loong64, ppc64le (VSX) and
s390x (vector facility, big-endian) — on plain go build,
CGO_ENABLED=0, stable Go, no GOEXPERIMENT. Each package is a drop-in:
same signatures, byte-identical output (and, where applicable,
byte-identical errors) to the stdlib package it accelerates, with the short
tail delegated back to the standard library so results match exactly.
Six SIMD targets, validated on seven architectures. A single
go-asmgen builder (v0.5.0) over a shared ABI0 layout emits the
kernels for all six targets; cmd/asm encodes them. As of 2026-07,
ppc64le (real POWER9), riscv64 (real SpacemiT X60, RVV 1.0) and loong64 (real
Loongson 3A5000) are natively measured on the GCC Compile Farm — not
just qemu-validated — and the portable scalar fallback is additionally
build+test-validated on a seventh arch, ppc64 (big-endian), on real POWER9.
Most repos are now natively measured on real IBM z15 (VXE2, -count=6) —
the vector-facility kernels post clear wins: bitpack ~34×, streamvbyte
decode ~20×, hex encode ~18×, matchlen ~9.2×, crc64 ~9.1× —
and every kernel remains bit-exact on that big-endian target. Two repos still
carry the older qemu-validated-for-correctness, native throughput pending
framing pending a follow-up z15 run: jsonvalidate (whitespace skip was
routed back to scalar there) and ascii85 — see each repo's own README for
its current status. Headline numbers come from native measurement and are
never extrapolated.
SIMD acceleration stays on six targets (ppc64 BE has no VSX build tag, so it
runs the generic path); proven correctness now spans seven architectures.
Two results from the six-arch port are worth singling out:
- base32 gets real SIMD on ppc64le and s390x where arm64 could not. The
per-char 5-bit extraction needs a register-variable vector shift and an
integer vector multiply — neither of which Go's arm64 assembler exposes, so
the NEON port was shelved. POWER's
VSRH(per-lane variable right shift) and IBM Z'sVMLHH(integer vector multiply-high) are exactly those primitives, so both run the full amd64 spread-extract kernel. - s390x is big-endian, and every kernel is bit-exact there. The whole suite is validated byte-for-byte on a big-endian target — a genuine cross-endian correctness check, not just a recompile.
These are benchmarked, not hand-waved. The table below reports the honest headline for each repo — the wins and the parities and the cases where SIMD genuinely does not help. A multi-table scalar loop that beats the vector unit, or a checksum that lands ~7% behind the best competitor, is reported as such. The credibility is the honesty.
| Repo | Accelerates | Honest headline | CI · Coverage |
|---|---|---|---|
| base64 | encoding/base64 |
encode ~17× stdlib, beats emmansun/base64 ~5–6% (cycle-model-guided); SIMD decode too — beats emmansun decode on amd64 (~1.3×); arm64 encode now ties emmansun via VLD3/VST4 |
|
| base32 | encoding/base32 |
encode ~7.9× stdlib (AVX2); real SIMD on ppc64le VSRH + s390x VMLHH where arm64 can't; now SIMD decode too + arm64 NEON encode on Go 1.27 (VUMULL); no prior pure-Go SIMD base32 exists |
|
| hex | encoding/hex |
beats tmthrgd/go-hex both ways — encode 20.4×, decode 6.24× stdlib (1.37× over tmthrgd) |
|
| utf8 | unicode/utf8 |
Valid ~19× stdlib, edges stuartcarnie/go-simd ~3.5% (Lemire/Keiser) |
|
| adler32 | hash/adler32 |
~12–14× stdlib, but ~7% behind mhr3/adler32-simd — near-parity, honest |
|
| strconv | strconv (base-10 parse + ParseFloat) |
ParseInt/Atoi ~2.2–2.7× on long inputs; added SIMD ParseFloat (Eisel-Lemire mantissa fold, ~1.1–1.36×); never regresses (delegates short/uncertain) |
|
| matchlen | LZ match-finder primitive | ~10× scalar (arm64 NEON); AVX2 ~2.08× SSE2 — real SIMD on all 6 arches | |
| bitpack | FastPFOR / simdcomp bit-packing | ~21–32× scalar — textbook SIMD-friendly, byte-exact vs Lemire's simdcomp | |
| popcount | Hamming weight over []byte |
size-dependent, honest: ~9–12× scalar in-cache, converges to memory-bound out-of-cache; 4-way POPCNTQ beats AVX2 |
|
| histogram | byte-value histogram | multi-table scalar — measured, SIMD doesn't help (scatter, no scatter instruction on AVX2/NEON) | |
| crc64 | hash/crc64 |
first pure-Go SIMD crc64 (stdlib + all Go libs are scalar); CLMUL folding, arm64 ~4.2× stdlib; all 6 arches (riscv64/loong64 scalar — no CLMUL exposed) | |
| crc32 | hash/crc32 |
arm64-only, deliberately: the stdlib IEEE path is already hardware-assisted on amd64/ppc64le/s390x, so only arm64's serial CRC32X fallback is worth beating — an 8-lane PMULL/PMULL2 fold does that; every other arch and every non-IEEE polynomial just calls hash/crc32 |
|
| ascii85 | encoding/ascii85 |
first SIMD ascii85, SIMD encode and decode; real SIMD on all 6 arches incl. an arm64 NEON kernel on Go 1.27 (VUMULL/VMUL; scalar on stable Go ≤ 1.26), still ~1.9× stdlib |
|
| floats | Dot/Sum/Min/Max/Distance/CosineSimilarity (float32 + float64) |
only pure-Go lib with the full vector-search API on all 6 arches (gonum/vek/kelindar are amd64-centric or partial); arm64 ~2× scalar | |
| xxhash | XXH3-64 | first Go XXH3 on all 6 SIMD arches (zeebo/cespare are amd64+arm64 only), bit-exact vs official vectors. Honest: zeebo faster on arm64 (~22 vs ~15 GB/s) — value is breadth, not peak | |
| streamvbyte | Stream VByte uint32 codec |
first Go port on all 6 arches (others amd64-only); full codec — SIMD encode and decode, byte-identical to lemire/streamvbyte (C-interop verified); decode arm64 ~10× scalar | |
| ascii | ASCII case (ToUpper/ToLower/EqualFold, bytes/strings) |
branchless vector case-fold, byte-identical to stdlib; real SIMD on all 6 arches incl. arm64 (sign-bit predicate, no multiply); arm64 ~4.9× stdlib (ToUpper) |
|
| levenshtein | edit distance (Distance) |
Myers bit-parallel edit distance, 62–142× vs agnivade/levenshtein on long inputs. Honest: pure-Go bit-parallel, not vector-SIMD — the word-serial column update has no lane parallelism (documented); endian-clean on all 6 arches |
|
| bitset | bulk []uint64 set ops |
SIMD And/Or/AndNot/Xor + fused Count/IntersectionCount; amd64 Count ~3–3.9×; honest: logical ops are bandwidth-bound (converge to scalar out-of-cache), popcount wins across the board |
|
| int8dot | int8 quantized dot (Dot/DotUint8/DotU8S8) |
INT8 multiply-accumulate for ML embeddings, bit-exact vs scalar; amd64 AVX2 ~4×; arm64 NEON kernel on Go 1.27 (VSMULL/VUMULL; scalar on stable Go ≤ 1.26) |
|
| jsonvalidate | encoding/json.Valid |
byte-identical verdict to encoding/json.Valid; only SIMD JSON validator on all 6 arches (simdjson-go is amd64-only); SIMD string/space scans, ~1.4× on string-heavy JSON (honest: number-/structure-heavy stays scalar) |
base64, base32, hex, ascii85, ascii, utf8, adler32, crc32, crc64, strconv, jsonvalidate are drop-in fast paths matching a named stdlib operation (byte- and, where applicable, error-identical) — crc32 is the odd one out, accelerating arm64 only since the stdlib IEEE path is already hardware-assisted everywhere else. matchlen, bitpack, popcount, histogram, streamvbyte, xxhash, floats, bitset, int8dot, levenshtein accelerate primitives the standard library exposes only per-word or not at all, with an oracle as the reference. Across the org, ppc64le, riscv64 and loong64 are now natively measured on real silicon (GCC Compile Farm — POWER9, SpacemiT X60 RVV 1.0, Loongson 3A5000); most repos are now natively measured on real IBM z15 too (
jsonvalidateandascii85still carry the older qemu-validated-for-correctness framing, native z15 throughput pending for those two); and the suite additionally builds and passes its differential + fuzz tests on a seventh arch, ppc64 big-endian, on real POWER9 (seematchlen-wasmabove for the separate wasm-SIMD 7th-target proof-of-concept — not to be confused with this seventh validated architecture). The five newest repos before crc32 — ascii, levenshtein, bitset, int8dot, jsonvalidate — rounded the suite out to 20; crc32 makes 21. jsonvalidate is the only SIMD JSON validator covering all six arches (byte-identical toencoding/json.Valid); levenshtein is honestly pure-Go bit-parallel, not vector-SIMD (its word-serial Myers column has no lane parallelism, documented), yet still 62–142× vsagnivade; bitset and int8dot are vector primitives (int8dot's arm64 NEON kernel needs Go 1.27, scalar below); and ascii runs real SIMD case-folding on arm64 too, via a multiply-free sign-bit predicate — ~4.9× stdlib.
A kernel is not "done" because it compiles — it is done when it is proven correct and measured honestly. Every repo follows the same pipeline:
- Check existing work first. Before writing a kernel, survey the field
(
emmansun,tmthrgd,mhr3,stuartcarnie,barakmich, Lemire's references) so the comparison is against the real state of the art, not a strawman. - Generate, don't hand-encode. The
.sis emitted by go-asmgenv0.5.0— one builder over a shared ABI0 layout for all six 64-bit targets (amd64 SSE2/AVX2, arm64 NEON, loong64 LSX/LASX, riscv64 RVV, ppc64le VSX, s390x vector facility — the last big-endian).cmd/asmdoes the encoding. - Model the cycles.
llvm-mcagives a port-level cycle model of the inner loop before benchmarking — this is what guided base64 pastemmansun(shuffle-port bound, fixed with a-432-byte load) and adler32 to ~99% of the vector-ALU ceiling. - Validate on real hardware (and qemu where there is no runner).
Differential tests + fuzzing against the stdlib/oracle reference run on a
real-AVX2 x86-64 host (Rosetta hides AVX2, so it is never trusted for
amd64), native arm64, and riscv64 / loong64 / ppc64le / s390x under
qemu-user(debian:trixie,QEMU_CPU=power9/qemu). Correctness is additionally confirmed on real silicon via the GCC Compile Farm (ppc64le on real POWER9, riscv64 on a SpacemiT X60 RVV 1.0 board, loong64 on a real Loongson 3A5000 — all now with native throughput numbers, see above) and ppc64 big-endian on a POWER9 host (a seventh validated arch: the generic fallback path proven bit-exact big-endian, beyond s390x's vector kernel). s390x now runs on real IBM z15 (VXE2) hardware too for most repos, with native throughput numbers (jsonvalidateandascii85still qemu-validated for correctness only, pending a follow-up z15 run). The headline benchmark numbers come from native CI (amd64/arm64), the cfarm hosts (ppc64le / riscv64 / loong64) and the z15 host (s390x), never from emulation, and are not extrapolated. The riscv64 figures are from a single in-order low-power core (the only widely-available RVV 1.0 silicon) — an out-of-order RVV core would likely lift the byte-shuffle kernels off parity too; reported honestly as-is. - 100% statement coverage, enforced as a CI gate on every repo and every
arch job — including every dispatch branch (AVX2 / SSE / POPCNT / scalar
fallback), driven directly by force tests. (The generated
.skernels are not measured bygo test -cover; they are covered by the differential + fuzz suites.)
See the validation toolkit page for the full methodology.
- Drop-in or nothing. Same signatures, byte-identical output and errors. The SIMD path handles whole aligned blocks; the tail delegates to the stdlib, so a caller can swap the import and nothing else.
- Generated, multi-arch assembly. No hand-written
.s, no per-arch encoder — go-asmgen emits Plan 9 text for all six 64-bit targets (incl. ppc64le VSX and big-endian s390x) andcmd/asmencodes it. - Pure Go,
CGO_ENABLED=0, stable Go. cgo wrappers of C libraries (aklomp/base64, simdutf, …) are faster still but need a C toolchain — out of scope; this is the pure-Go tier. - Honest numbers. Wins, parities, and non-SIMD outcomes are all reported.
- Website (landing): https://go-simd.github.io — built with Hugo.
- Documentation: https://go-simd.github.io/docs/ — MkDocs Material, versioned with mike; source in go-simd/docs.
- Assembly generator: go-asmgen.
