Repository navigation
Releases
Releases · go-while/go-yenc-simd
v0.1.2: NEON kernel for arm64, AVX fix, faster loops
- Fix: crash on CPUs with AVX but not AVX2 (Intel Sandy Bridge and Ivy Bridge; AMD Bulldozer, Piledriver, Steamroller and Jaguar). With
GOEXPERIMENT=simd, the AVX kernel in v0.1.0 and v0.1.1 used an AVX2 instruction (VPBROADCASTB), so decoding died with SIGILL on these CPUs. A new test,TestKernelISA, disassembles every kernel and checks each instruction against the CPU features that the kernel's run-time check requires. - NEON kernel for arm64 with
GOEXPERIMENT=simd, 64 bytes per step. On GitHub's runners, random data decodes at 6.0–8.0 GB/s on Apple M1 (plain Go: 1.9–2.7 GB/s) and 5.4 GB/s on Neoverse N2 (plain Go: 2.6–2.7 GB/s). The README has the full table. - Faster AVX2, AVX and plain Go loops. The Go compiler had left a NOP in the hot loops for each inlined helper call. Without them, in CPU cycles per byte on an AMD EPYC 9B45: AVX2 −17.5% on data that needs no escapes and about −7% on random data; AVX −7% on data that needs no escapes and −10% with an escape in every second byte; plain Go −7% on random data and −9% on data that needs no escapes. The AVX-512 kernel is unchanged.
- Plain Go on other CPUs: the 16-bytes-per-step loop now loads and stores 8-byte words directly, so it runs only where unaligned little-endian access is fast (amd64, arm64, 386, ppc64le, wasm). On riscv64, mips, 32-bit arm and big-endian CPUs the plain Go decoder now goes a byte at a time (likely slower there than v0.1.1; not measured).
- yencbench prints the CPU name on arm64 Linux, macOS, the BSDs and Windows, not only on x86 Linux:
GOEXPERIMENT=simd go run github.com/go-while/go-yenc-simd/cmd/yencbench@v0.1.2 - Tests:
TestBoundariesputs every string of up to four special bytes at every offset of plain data (about 2.8 million inputs per kernel) and compares each kernel with the state machine. The plain Go decoder is now compared with the byte-by-byte state machine instead of with itself. CI tests the NEON kernel on GitHub's arm64 runners (macOS and Linux), with a 60-second fuzz run and the race detector.
Full changes: v0.1.1...v0.1.2
v0.1.1: faster plain Go path on arm64
- The plain Go decoder (used on every platform without
GOEXPERIMENT=simd, including Apple Silicon) keeps its SWAR constants in registers on non-amd64 architectures instead of rebuilding them in every iteration. On the GitHub macOS arm64 runner, random data decodes at ~2.3 GB/s (v0.1.0: ~1.5 GB/s) and data without escapes at ~5 GB/s. amd64 code is unchanged. - CI workflow: vet, tests and
yencbenchon Linux, macOS (Apple Silicon) and Windows, with and withoutGOEXPERIMENT=simdon x86-64.
Full changes: v0.1.0...v0.1.1
v0.1.0: first release
First release of go-yenc-simd, a yEnc decoder for Usenet articles in pure Go.
What's in it
Decode(article)andNewReader(r)decode whole articles (headers skipped, dot-stuffing removed, size and CRC32 checked);DecodeIncrementalis the low-level step, a drop-in for rapidyenc'srapidyenc_decode_incremental, with the same state machine byte for byte (504 golden outputs from the C decoder).- With
GOEXPERIMENT=simdon amd64 (Go 1.27): AVX-512 VBMI2, AVX2 or AVX kernels, chosen at run time. Everywhere else a plain Go path that decodes 16 bytes per step. cmd/yencbenchand packageyencbenchto measure the decoder on your machine, for every kernel the CPU can run:
GOEXPERIMENT=simd go run github.com/go-while/go-yenc-simd/cmd/yencbench@v0.1.0
Speed
750 KB article, MB/s of input, median of pinned runs; rapidyenc is the C reference on the same core.
| Intel i7-10710U (AVX2), 2 GHz | AMD EPYC 9B45 (Zen 5, AVX-512) | |
|---|---|---|
| DecodeIncremental, random data | 2,915 | 24,727 (AVX-512), 15,085 (AVX2) |
| same, data that needs no escapes | 4,837 | 25,964 / 22,667 |
| same, plain Go (no GOEXPERIMENT=simd) | 947 | 3,439 |
| Decode (whole article, CRC checked) | 1,808 | 10,408 |
| Reader on a 64 KiB bufio.Reader | 1,954 | 12,552 |
| rapidyenc, random data | 3,775 | 17,263 (VBMI2), 18,795 (AVX2) |
Notes
dstandsrcofDecodeIncrementalmust not overlap (in-place decoding is not supported).GOEXPERIMENT=simdis a Go experiment outside the Go 1 compatibility promise; without it the package builds and works the same, only slower.- Verified with the golden table, random and fuzz comparisons of every kernel against the state machine,
-raceand checkptr builds, and differential tests against the previous implementation.