Skip to content

v0.1.0: first release

Choose a tag to compare

@go-while go-while released this 19 Oct 10:49
· 10 commits to main since this release

First release of go-yenc-simd, a yEnc decoder for Usenet articles in pure Go.

What's in it

  • Decode(article) and NewReader(r) decode whole articles (headers skipped, dot-stuffing removed, size and CRC32 checked); DecodeIncremental is the low-level step, a drop-in for rapidyenc's rapidyenc_decode_incremental, with the same state machine byte for byte (504 golden outputs from the C decoder).
  • With GOEXPERIMENT=simd on amd64 (Go 1.27): AVX-512 VBMI2, AVX2 or AVX kernels, chosen at run time. Everywhere else a plain Go path that decodes 16 bytes per step.
  • cmd/yencbench and package yencbench to measure the decoder on your machine, for every kernel the CPU can run:
    GOEXPERIMENT=simd go run github.com/go-while/go-yenc-simd/cmd/yencbench@v0.1.0

Speed

750 KB article, MB/s of input, median of pinned runs; rapidyenc is the C reference on the same core.

Intel i7-10710U (AVX2), 2 GHz AMD EPYC 9B45 (Zen 5, AVX-512)
DecodeIncremental, random data 2,915 24,727 (AVX-512), 15,085 (AVX2)
same, data that needs no escapes 4,837 25,964 / 22,667
same, plain Go (no GOEXPERIMENT=simd) 947 3,439
Decode (whole article, CRC checked) 1,808 10,408
Reader on a 64 KiB bufio.Reader 1,954 12,552
rapidyenc, random data 3,775 17,263 (VBMI2), 18,795 (AVX2)

Notes

  • dst and src of DecodeIncremental must not overlap (in-place decoding is not supported).
  • GOEXPERIMENT=simd is a Go experiment outside the Go 1 compatibility promise; without it the package builds and works the same, only slower.
  • Verified with the golden table, random and fuzz comparisons of every kernel against the state machine, -race and checkptr builds, and differential tests against the previous implementation.