Skip to content

Releases

v0.1.2: NEON kernel for arm64, AVX fix, faster loops

Choose a tag to compare

@go-while go-while released this 23 Oct 02:17
  • Fix: crash on CPUs with AVX but not AVX2 (Intel Sandy Bridge and Ivy Bridge; AMD Bulldozer, Piledriver, Steamroller and Jaguar). With GOEXPERIMENT=simd, the AVX kernel in v0.1.0 and v0.1.1 used an AVX2 instruction (VPBROADCASTB), so decoding died with SIGILL on these CPUs. A new test, TestKernelISA, disassembles every kernel and checks each instruction against the CPU features that the kernel's run-time check requires.
  • NEON kernel for arm64 with GOEXPERIMENT=simd, 64 bytes per step. On GitHub's runners, random data decodes at 6.0–8.0 GB/s on Apple M1 (plain Go: 1.9–2.7 GB/s) and 5.4 GB/s on Neoverse N2 (plain Go: 2.6–2.7 GB/s). The README has the full table.
  • Faster AVX2, AVX and plain Go loops. The Go compiler had left a NOP in the hot loops for each inlined helper call. Without them, in CPU cycles per byte on an AMD EPYC 9B45: AVX2 −17.5% on data that needs no escapes and about −7% on random data; AVX −7% on data that needs no escapes and −10% with an escape in every second byte; plain Go −7% on random data and −9% on data that needs no escapes. The AVX-512 kernel is unchanged.
  • Plain Go on other CPUs: the 16-bytes-per-step loop now loads and stores 8-byte words directly, so it runs only where unaligned little-endian access is fast (amd64, arm64, 386, ppc64le, wasm). On riscv64, mips, 32-bit arm and big-endian CPUs the plain Go decoder now goes a byte at a time (likely slower there than v0.1.1; not measured).
  • yencbench prints the CPU name on arm64 Linux, macOS, the BSDs and Windows, not only on x86 Linux:
    GOEXPERIMENT=simd go run github.com/go-while/go-yenc-simd/cmd/yencbench@v0.1.2
  • Tests: TestBoundaries puts every string of up to four special bytes at every offset of plain data (about 2.8 million inputs per kernel) and compares each kernel with the state machine. The plain Go decoder is now compared with the byte-by-byte state machine instead of with itself. CI tests the NEON kernel on GitHub's arm64 runners (macOS and Linux), with a 60-second fuzz run and the race detector.

Full changes: v0.1.1...v0.1.2

v0.1.1: faster plain Go path on arm64

Choose a tag to compare

@go-while go-while released this 19 Oct 11:49
  • The plain Go decoder (used on every platform without GOEXPERIMENT=simd, including Apple Silicon) keeps its SWAR constants in registers on non-amd64 architectures instead of rebuilding them in every iteration. On the GitHub macOS arm64 runner, random data decodes at ~2.3 GB/s (v0.1.0: ~1.5 GB/s) and data without escapes at ~5 GB/s. amd64 code is unchanged.
  • CI workflow: vet, tests and yencbench on Linux, macOS (Apple Silicon) and Windows, with and without GOEXPERIMENT=simd on x86-64.

Full changes: v0.1.0...v0.1.1

v0.1.0: first release

Choose a tag to compare

@go-while go-while released this 19 Oct 10:49

First release of go-yenc-simd, a yEnc decoder for Usenet articles in pure Go.

What's in it

  • Decode(article) and NewReader(r) decode whole articles (headers skipped, dot-stuffing removed, size and CRC32 checked); DecodeIncremental is the low-level step, a drop-in for rapidyenc's rapidyenc_decode_incremental, with the same state machine byte for byte (504 golden outputs from the C decoder).
  • With GOEXPERIMENT=simd on amd64 (Go 1.27): AVX-512 VBMI2, AVX2 or AVX kernels, chosen at run time. Everywhere else a plain Go path that decodes 16 bytes per step.
  • cmd/yencbench and package yencbench to measure the decoder on your machine, for every kernel the CPU can run:
    GOEXPERIMENT=simd go run github.com/go-while/go-yenc-simd/cmd/yencbench@v0.1.0

Speed

750 KB article, MB/s of input, median of pinned runs; rapidyenc is the C reference on the same core.

Intel i7-10710U (AVX2), 2 GHz AMD EPYC 9B45 (Zen 5, AVX-512)
DecodeIncremental, random data 2,915 24,727 (AVX-512), 15,085 (AVX2)
same, data that needs no escapes 4,837 25,964 / 22,667
same, plain Go (no GOEXPERIMENT=simd) 947 3,439
Decode (whole article, CRC checked) 1,808 10,408
Reader on a 64 KiB bufio.Reader 1,954 12,552
rapidyenc, random data 3,775 17,263 (VBMI2), 18,795 (AVX2)

Notes

  • dst and src of DecodeIncremental must not overlap (in-place decoding is not supported).
  • GOEXPERIMENT=simd is a Go experiment outside the Go 1 compatibility promise; without it the package builds and works the same, only slower.
  • Verified with the golden table, random and fuzz comparisons of every kernel against the state machine, -race and checkptr builds, and differential tests against the previous implementation.