Releases: dfa1/zstd-ffm
Release list
v0.14
Added
- docs/how-to.md: "Stream over pooled
DirectByteBuffers
(Netty-style)" recipe, combining the existingByteBuffer-wrapping and
ZstdCompressStreamrecipes into a zero-allocation pattern for pooled-buffer
network stacks. No new API. (#99) - New module
io.github.dfa1.zstd:zstd-rfc9842— RFC 9842 (Compression
Dictionary Transport) support.Rfc9842Frame.wrap/unwrapadd or verify thedczwire format (a
skippable-frame header carrying a SHA-256 dictionary hash around a zstd
frame), so a decoder can self-verify the right dictionary is in hand
before decompressing. No HTTP dependency.
(#91)UseAsDictionaryHeader,AvailableDictionaryHeader, and
DictionaryIdHeaderparse/build the three RFC 9842 HTTP headers as a
framework-agnostic model, backed by a minimal hand-rolled RFC 8941
(Structured Field Values) parser andUse-As-Dictionarypath matching.
Rfc9842Negotiation.from(byte[], String)composes a fetched dictionary
and its header into everything a client needs for later requests.
(#92)- A runnable HTTP/1.1 + HTTP/2 (h2c) demo on embedded Jetty (
rfc9842's
test classpath), and an ArchUnit rule enforcing the module's sans-io
design (nojava.net/java.nio.channels/javax.netoutside the demo).
(#124) - Usage docs: a how-to recipe,
a reference entry,
and the sans-io rationale in docs/explanation.md.
v0.13
- Project renamed
zstd-java→zstd-ffm(GitHub repo, parent POM
artifactId, SonarCloud project key). Published Maven coordinates
(io.github.dfa1.zstd:zstd,:zstd-platform,:bom, native classifiers)
are unaffected — only the internal parent-aggregatorartifactIdchanged.
Fixed
ZstdCompressContext/ZstdDecompressContext.refDictionary(...)held a raw
native pointer to the referencedZstdCompressDictionary/
ZstdDecompressDictionary: closing that dictionary while a context still
referenced it freed the pointer out from under the context, corrupting or
crashing every subsequent compress/decompress call. Both dictionary types
now share ownership with every referencing context via
NativePointerWithRefCount, so the native dictionary is freed only once
every borrower — including the dictionary's own constructor reference —
has let go.refDictionary(...)on an already-closed context, and
loadDictionary(...)/refPrefix(...)superseding a referenced dictionary,
now also drop the reference correctly instead of leaking it.
(#119)
v0.12
This cycle completes a sweep replacing naked primitives at the public API with
validated domain primitives — ZstdByteSize, ZstdCompressionLevel,
ZstdWindowLog, ZstdMagicVariant, ZstdVersion, and ZstdFrameHeader's size
fields. Each validates once, at construction, so a value is proof of its own
validity and illegal states are unrepresentable rather than guarded by scattered
runtime checks (a hostile frame field that cannot be a real size parses to an
empty Optional<ZstdByteSize> instead of throwing). The rationale — types as the
first line of defense — is laid out in
Your compiler is already part of your security team.
Added
ZstdDecompressContext.decompress(byte[])— decompresses a trusted frame using
the decompressed size stored in its header, mirroring the static
Zstd.decompress(byte[]). For untrusted input keep the bounded
decompress(byte[], ZstdByteSize)overload, now documented as the safe entry
point.ZstdWindowLogandZstdMagicVariantvalue types, validated at construction —
the window log against the linked libzstd's accepted range (with0/
ZstdWindowLog.AUTOfor "library chooses"), the magic variant to0..15.ZstdVersionvalue type — a comparable(major, minor, patch)withnumber()
(the packed form) andtoString()(x.y.z), for feature-gating against the
linked libzstd.
Changed
- Breaking: the compression-level bound queries
Zstd.maxCompressionLevel(),
minCompressionLevel(), anddefaultCompressionLevel()are no longer public.
The naked-intqueries now live internally onZstdCompressionLevel; get the
levels as value objects via theZstdCompressionLevel.MAX/FASTEST/DEFAULT
constants (call.value()for the rawintwhen handing one to a native or
third-party API that takesint). - Breaking: renamed
ZstdByteSize.toIntExact()totoArraySize(), which now
throwsZstdException(wasArithmeticException) when the size exceeds the
maximum array length — the narrowing happens at the API boundary, so callers
no longer catch a rawArithmeticException. - Breaking:
ZstdDecompressContext.decompressnow takes aZstdByteSize
bound instead of a nakedint(all three overloads: plain,+ ZstdDictionary,
+ ZstdDecompressDictionary), completing theZstdByteSizemigration. Wrap the
bound innew ZstdByteSize(n). - Breaking: the frame-size probes
ZstdFrame.compressedSize,headerSize,
anddecompressionMarginnow returnZstdByteSizeinstead oflong, and
ZstdDictionary.size()/headerSize()returnZstdByteSizeinstead ofint,
matching the other size accessors — call.value()for the raw number. The
zero-copy segmentcompress/decompressoverloads still return a rawlong
by design (the count feeds straight intoMemorySegment.asSlice). - Breaking:
ZstdCompressContext.windowLogand
ZstdDecompressContext.windowLogMaxnow take aZstdWindowLog, and
ZstdFrame.writeSkippableFrame/ZstdSkippableContent.magicVariant()now use
ZstdMagicVariant. Wrap the rawint(new ZstdWindowLog(n)/
new ZstdMagicVariant(n)), or useZstdWindowLog.AUTO. - Breaking:
Zstd.version()now returns aZstdVersioninstead of a
String, andZstd.versionNumber()is removed. Use
Zstd.version().toString()for thex.y.zstring and
Zstd.version().number()for the packed number. - Breaking:
ZstdFrameHeaderno longer exposes any naked numeric field.
blockSizeMax()andheaderSize()now returnZstdByteSize(werelong/
int; call.value()for the raw number). The rawlong frameContentSize
component is gone —contentSize()is now a realOptional<ZstdByteSize>
record component — andwindowSize()likewise returnsOptional<ZstdByteSize>.
Both areOptionalbecause a hostile header can declare either as an
unrepresentable value at or above2^63, which maps to empty rather than
throwing while the header is parsed.
v0.11
Changed
- Breaking: every public API that took or returned a naked
int/longbyte
size or count now takes/returns aZstdByteSizevalue type, which rejects
negative values at construction (throwingIllegalArgumentException) — once
you hold aZstdByteSizeit is guaranteed valid, no more wondering whether a
rawlongfrom the library still needs checking. Sizes that must fit a
byte[]narrow viaZstdByteSize.toIntExact()(throwing
ArithmeticExceptionaboveInteger.MAX_VALUE), matching the JDK's own
Math.toIntExact; native and streaming totals pass through aslong. A
frame-declared content size that is a zstd sentinel, or otherwise invalid
(including the unsigned range aboveLong.MAX_VALUE, read as a negative
long), fails fast withZstdExceptionvia
ZstdByteSize.fromFrameContentSize(long)rather than reaching the
constructor's generic negative-value guard;ZstdFrameHeader.contentSize()
is nowOptional<ZstdByteSize>(wasOptionalLong) via the sibling
ZstdByteSize.fromFrameHeaderContentSize(long), which treats any negative
reading — not just the sentinel — as absent, since it can only mean an
unrepresentable declared size. Affects
Zstd.decompress(byte[], …),Zstd.compressBound,
Zstd.estimateCompressContextSize/estimateDecompressContextSize/
estimateCompressDictSize/estimateDecompressDictSize,
Zstd.decompressedSize(MemorySegment),
ZstdDictionary.train/trainCover/trainFastCover/finalizeFrom,
ZstdOutputStream.withPledgedSize,ZstdFrame.decompressedSize/
decompressedBound,ZstdFrameHeader.contentSize, andsizeOf()on
ZstdCompressContext/ZstdDecompressContext/ZstdCompressStream/
ZstdDecompressStream/ZstdCompressDictionary/ZstdDecompressDictionary.
Usenew ZstdByteSize(n), orZstdByteSize.ofKiB(n)/ofMiB(n)for a size
expressed in KiB/MiB.
(#96) - Breaking: every public API that took a raw
intcompression level now
takes aZstdCompressionLevelvalue type, which validates the level against
the linked libzstd's accepted range at construction (throwing
IllegalArgumentException) rather than deferring to native clamping/errors.
AffectsZstd.compress(byte[], …),Zstd.estimateCompressContextSize,
Zstd.estimateCompressDictSize,ZstdCompressContext.level,
ZstdCompressStream,ZstdOutputStream,ZstdCompressDictionary,
ZstdDictionary.compressDict/trainCover/trainFastCover/finalizeFrom.
Usenew ZstdCompressionLevel(19)or theDEFAULT/FASTEST/MAXconstants.
TheZstd.min/max/defaultCompressionLevel()bound queries still returnint.
(#93)
v0.10
Added
- Native builds now compile with
ZSTD_MULTITHREAD, making
ZstdCompressParameter.NB_WORKERS(plusJOB_SIZEandOVERLAP_LOG)
functional instead of a silent no-op. Workers engage above zstd's 512 KiB
job-size minimum; multithreaded frames are format-valid but not
byte-identical to single-threaded output. A context that compressed with
workers holds its native worker threads untilclose()—reset()does
not release them, so give such contexts a dedicated owner (never pool
them). See ADR 0015, which
supersedes ADR 0014.
(#80)
v0.9
Added
- Native builds now decode legacy zstd frame formats v0.4-v0.7
(ZSTD_LEGACY_SUPPORT=4, matching zstd-jni's default). v0.1-v0.3 stay
unsupported — they predate zstd's 1.0 stabilization and are essentially
never seen in practice. Verified against a real fixture of five concatenated
legacy frames extracted from zstd's own test suite. (#73)
Changed
linux-x86_64/osx-x86_64/windows-x86_64native builds now include
zstd's hand-written BMI2 Huffman-decode assembly (previously disabled). It
is a no-op on non-x86_64 targets and only activates via zstd's own runtime
CPU detection; benchmarked as throughput-neutral on this project's
synthetic workload but carries no measured downside either.
(#71)aarch64native builds now target an ARMv8-A + CRC baseline
(-mcpu=generic+crc, zig's spelling of-march=armv8-a+crc), instead of
the fully generic baseline. Measured +6.9% compress / +12-14% decompress
throughput on Apple Silicon. (#71)
Security
linux-x86_64/linux-aarch64native builds now link with full RELRO and
immediate binding (-Wl,-z,relro,-z,now), closing off the classic
GOT-overwrite exploit primitive. Verified withllvm-readelf.
(#71)windows-x86_64/windows-aarch64native builds now export only zstd's
public API (ZSTD_*/ZDICT_*) from the DLL, via-DZSTD_DLL_EXPORT=1
(the PE analogue of the-fvisibility=hiddensurface already used on
ELF/Mach-O), instead of dumping every internal symbol
(FSE_*/HUF_*/COVER_*/...) into the export table via
--export-all-symbols. Cut the windows-x86_64 export table from 576 to
185 symbols. (#79)
Fixed
- Building the native library from source on Windows was silently broken:
Maven's exec plugin tried to executebuild-zstd.shdirectly, which only
works via a shebang on macOS/Linux. Windows builds now invoke it through
bashexplicitly. A second latent bug this surfaced — unrecognized/Windows
host OS detection crashed the build script underset -u— is fixed
alongside it. (#75) - Native library compilation now runs every translation unit through a real
parallel work queue (xargs -P) instead of a fixed-size batch-then-wait
loop, and aborts immediately if anyzig ccinvocation fails. Previously a
failed compile under&/waitwas invisible toset -e— it could
silently produce no.oand only surface later as a cryptic link error, or
worse, a link that "succeeded" against a stale.oleft over from a
previous run. (#78)
Investigated and rejected as part of the same effort (see
#70 for full benchmark data):
LTO (real compress regression on x86_64, unsupported on macOS entirely — zig's
Mach-O linker has no LTO support, tracked in
#77) and an x86-64-v3
baseline (mixed result, hurts compress more than it helps decompress). Both
would have traded away this project's existing compress-side edge over
zstd-jni for a smaller decompress-side gain.
v0.8
Added
module-info.java:zstdnow ships as a named JPMS module
(module io.github.dfa1.zstd), exporting the single public API package.
Module-path consumers grant--enable-native-access=io.github.dfa1.zstd
instead ofALL-UNNAMED; classpath consumers are unaffected. See
ADR 0011.
v0.7
Changed
- Breaking: renamed public types to spell out abbreviations, matching the
Zstd<Compress|Decompress><Stream|Parameter>family and zstd's own prose
("compression context", "dictionary"):ZstdCompressCtx→ZstdCompressContext,
ZstdDecompressCtx→ZstdDecompressContext,ZstdCompressDict→
ZstdCompressDictionary,ZstdDecompressDict→ZstdDecompressDictionary.
v0.6
Added
ZstdCompressCtx.refPrefix(MemorySegment)/ZstdDecompressCtx.refPrefix(...)
— reference native content as a single-use prefix (raw-content dictionary) for
the next frame only: the building block for delta compression (compress a new
version against a similar previous one). The prefix is referenced, not copied
or digested, and writes no dictionary ID; the decompressor must reference the
same prefix to decode. BindsZSTD_CCtx_refPrefix/ZSTD_DCtx_refPrefix.
Segment-only by design — heap callers that need a copy should use
loadDictionaryinstead.Zstd.dictId(byte[])/Zstd.dictId(MemorySegment)— read the dictionary id
stamped in raw dictionary bytes without wrapping them in aZstdDictionary.
BindsZSTD_getDictID_fromDict.ZstdDictionaryIdvalue type — arecordwrapping the 32-bit dictionary id
with an unsignedvalue(),isPresent(), and theNONEsentinel for "no id".ZstdFrame.decompressedSize(byte[])/ZstdFrame.decompressedSize(MemorySegment)
— the exact combined decompressed size of all concatenated frames, summed from
each frame header (throws if any frame does not record its size). Complements
decompressedBound(upper bound). BindsZSTD_findDecompressedSize.ZstdFrame.headerSize(byte[])/ZstdFrame.headerSize(MemorySegment)— the size
of a frame's header computed from just its leading bytes (as few as 5), without a
full parse. BindsZSTD_frameHeaderSize.ZstdFrame.decompressionMargin(byte[])/ZstdFrame.decompressionMargin(MemorySegment)
— the extra room needed to decompress a frame in place (output buffer overlaps
the compressed input at its tail), sizeddecompressedSize + margin. Binds
ZSTD_decompressionMargin.ZstdDictionary.compressDict(int)/compressDict()/decompressDict()—
factories for digested dictionaries, e.g.dict.compressDict(19)instead of
new ZstdCompressDict(dict, 19). They signal that the result isAutoCloseable
and are for sharing one digest across contexts viarefDictionary; a single
context should prefer the context-ownedloadDictionary.
Changed
- Every dictionary-id accessor now returns
ZstdDictionaryIdinstead ofint:
ZstdDictionary.id(),ZstdCompressDict.id(),ZstdDecompressDict.id(),
ZstdFrame.dictId(...), andZstdFrameHeader.dictId(). The0sentinel is now
ZstdDictionaryId.NONE, and the id reads as unsigned viavalue(). Zstd.decompress(byte[])now throwsZstdException(instead of letting a raw
ArithmeticExceptionescape) when a frame declares a content size larger than a
Java array can hold. The size comes from the untrusted frame header; use
decompress(byte[], int)to bound output for untrusted input.
v0.5
Added
ZstdCompressCtx.reset(ZstdResetDirective)/ZstdDecompressCtx.reset(...)—
recycle a context's native state between frames without freeing and recreating
it.SESSION_ONLYkeeps the level, parameters, and dictionary;PARAMETERS/
SESSION_AND_PARAMETERSrestore the defaults. BindsZSTD_CCtx_reset/
ZSTD_DCtx_reset.
(3dfd5b8)ZstdCompressCtx.loadDictionary(...)/ZstdDecompressCtx.loadDictionary(...)
(aZstdDictionaryor a nativeMemorySegment) andrefDictionary(...)(a
pre-digestedZstdCompressDict/ZstdDecompressDict, attached by reference,
no copy). A sticky dictionary on the context lets compression combine a
dictionary with the advanced parameters (checksum, window log, long-distance
matching) — impossible through the per-callcompress(src, dict)overloads,
which route the legacy dictionary path. A parameterreset(...)clears it.
BindsZSTD_CCtx_loadDictionary/ZSTD_DCtx_loadDictionary(now on contexts,
not just streams),ZSTD_CCtx_refCDict,ZSTD_DCtx_refDDict.
(3dfd5b8)
Changed
NativeLibrary.classifier()now throws a clearUnsatisfiedLinkErrornaming
the unsupported CPU arch instead of silently mapping it to x86_64 (which
deferred failure to a crypticdlopenerror). Added an explicitamd64
branch so Linux JVMs (which reportos.arch=amd64) still resolve x86_64.
(ea1ac84)
Fixed
- Native JARs are much smaller. The ELF shared library is now stripped at link
time (-s), dropping debug info (libzstd.so4.0M -> ~650K), and the
multi-MB.pdbdebug database and.libimport library that lld emits next
to the Windows.dllare no longer bundled (neither is needed at runtime).
Net: linux-x86_64 native jar 1.2M -> 285K, windows-x86_64 1.2M -> 372K.
(ea1ac84)