Skip to content
/ svb Public

Faster integer compression in Go with SIMD using StreamVByte codec

License

Notifications You must be signed in to change notification settings

rleiwang/svb

Repository files navigation

Stream VByte in Go with SIMD

This is another pure Go implementation of Stream VByte: Faster Byte-Oriented Integer Compression.

It uses avo by Michael McLoughlin to generate Go assembler code. This Go implementation has referenced https://github.com/lemire/streamvbyte

Random Access

Codec.Get random access implements idea based on Partitioned Elias-Fano Indexes

Speed Test

Uint32Decoder performance benchmark measures the latency to decode 1 million uint32. Codec.Get measures the latency of random access 1000 members among 1 million uint32.

svb/perf ❯❯❯ go test -bench .
Function Cascade Lake Skylake
Uint32Decode32 13562586ns 17394533ns
Uint32Decode128 331368ns 411102ns
Uint32Decode256 327100ns 406230ns
Uint32Decode512 470571ns 569497ns
Codec.Get ns 144818ns

About

Faster integer compression in Go with SIMD using StreamVByte codec

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published