Several package formats we want to open are zstd-compressed and we currently have no reader for it:
.tar.zst directly
.conda (anaconda.org v2 format) is a zip containing pkg-<name>.tar.zst and info-<name>.tar.zst
.deb on current Debian and Ubuntu ships data.tar.zst / control.tar.zst inside the ar container
Go stdlib has no zstd. github.com/klauspost/compress/zstd is pure Go, no cgo, actively maintained, and widely used; we already carry ulikunitz/xz as a compression dep so there is precedent. The alternative is github.com/DataDog/zstd which wraps the C library via cgo and is faster but breaks the pure-Go build.
Once a zstd.Reader is wired into the tar path the same way gzip/bzip2/xz are, .tar.zst is just another suffix in detectFormat, and zstd is another case in archiveFormat for the content-sniff fallback (magic will report zstd once git-pkgs/magic#1 lands). .conda and .deb need their own container handlers on top (zip-of-tars and ar-of-tars respectively) and can follow separately; this issue is just the decompressor.
Follows #21.
Several package formats we want to open are zstd-compressed and we currently have no reader for it:
.tar.zstdirectly.conda(anaconda.org v2 format) is a zip containingpkg-<name>.tar.zstandinfo-<name>.tar.zst.debon current Debian and Ubuntu shipsdata.tar.zst/control.tar.zstinside the ar containerGo stdlib has no zstd.
github.com/klauspost/compress/zstdis pure Go, no cgo, actively maintained, and widely used; we already carryulikunitz/xzas a compression dep so there is precedent. The alternative isgithub.com/DataDog/zstdwhich wraps the C library via cgo and is faster but breaks the pure-Go build.Once a
zstd.Readeris wired into the tar path the same way gzip/bzip2/xz are,.tar.zstis just another suffix indetectFormat, andzstdis another case inarchiveFormatfor the content-sniff fallback (magic will reportzstdonce git-pkgs/magic#1 lands)..condaand.debneed their own container handlers on top (zip-of-tars and ar-of-tars respectively) and can follow separately; this issue is just the decompressor.Follows #21.