Skip to content

v0.3.0: Add hex base32 and percent codecs (#2)

Latest

Choose a tag to compare

@dans-stuff dans-stuff released this 03 Oct 22:32
91c27f2
* base64: 512/256/128-bit cascade, fix panic on malformed padding, drop NEON asm

Encode and Decode consume whole blocks with the widest available vectors and
hand the untouched remainder, padding included, to encoding/base64, adding the
consumed offset to CorruptInputError. This fixes a panic on SIMD builds:
StdEncoding.DecodeString("QQ=") stripped the padding but sized the buffer for
the padded input.

Removes the func-var dispatch, the SIMDENC_NO_* switches and the hand-written
NEON assembly (arm64 now uses encoding/base64). Adds differential tests against
encoding/base64 that run on every CPU tier (512-bit, AVX2, reference): every
prefix of a valid encoding, every byte at every position, malformed padding.
The shared test harness (codec_test.go) is reused by the codecs that follow.

go.mod: go 1.26.4, the toolchain this was tested on.

* hex: AVX-512/AVX2 encode and decode, same results as encoding/hex

HexEncode, HexDecode, HexEncodeToString, HexDecodeString, AppendHexEncode,
AppendHexDecode, HexEncodedLen, HexDecodedLen. A 512-bit then a 256-bit kernel
consume whole blocks; encoding/hex finishes the rest and reports every error,
so counts and errors match it exactly. Tested on every CPU tier against
encoding/hex: every byte at every position, all 65536 two-character pairs, many
lengths, and fuzz targets.

* base32: AVX-512 encode and decode, same results as encoding/base32

Base32Encoding with the Std and Hex alphabets and WithPadding, with the method
set of encoding/base32. A 512-bit kernel (plus one partial-vector tail step)
consumes whole 5-byte/8-character groups; encoding/base32 finishes the rest,
including padding, and reports every error. There is no AVX2 tier: AVX2 has no
byte permute and no 16-bit variable shift, so it would need a different
algorithm. Tested on every CPU tier against encoding/base32.

* percent-encoding: AVX-512 VBMI2 QueryEscape/PathEscape/QueryUnescape/PathUnescape

Same results as net/url. A SIMD pass counts what needs escaping, so the input
is returned unchanged, without allocating, when nothing does, and otherwise the
exact result is allocated once. Escape expands 21 bytes per iteration into %
and two hex digits and uses Compress to drop unused lanes; unescape joins the
two hex digits after each % and compresses the digit lanes away. net/url
decodes whatever a kernel leaves, so every error is net/url's own. The byte
class table is derived from net/url at init so the two cannot disagree. Needs
VBMI2; elsewhere net/url's behaviour is reproduced by a scalar loop.

Tested on every CPU tier against net/url: every byte at every position, a %XX at
every offset around the 64-byte block boundary, all 65536 two-character
sequences where a block ends, and fuzz targets.

* README: hex, base32 and percent-encoding, and measured results

Summary table against the fastest other implementation and the standard
library at 64 KB, per-size tables for every codec, where simdenc is not ahead
(short inputs, hex encode vs go-simd), and what each tier needs.