Codec-compression: Add Zstd decompressor

[yawkat]

Motivation:

The decompressor API migration tracked in #16743 is being split out from #15667 into smaller, reviewable pieces. This adds the Zstd decompressor slice on top of the core decompressor API that is already present, including the ZstdDecoder window-memory guard added in #16850.

Modification:

Add ZstdDecompressor, backed by zstd-jni ZstdInputStreamNoFinalizer, and add the matching AbstractDecompressorTest coverage using ZstdEncoder.

The decompressor builder exposes maxWindowLog(int) and applies it through ZstdInputStreamNoFinalizer.setLongMax(...) so frames declaring oversized windows are rejected before zstd allocates excessive native memory. takeOutput() also uses the shared compression forward-size as its per-call output chunk size.

Result:

Zstd now has a decompressor implementation for the new API with focused contract coverage and bounded zstd native window memory by default.

Verification:

• ./mvnw -pl codec-compression -am -DskipRemoteStaging=true -Dcheckstyle.skip=false -DskipTests=false -Dtest=ZstdDecompressorTest -Dsurefire.failIfNoSpecifiedTests=false test
• ./mvnw -pl codec-compression -am -DskipRemoteStaging=true -Dcheckstyle.skip=false -DskipTests=false verify

Part of #16743.

[Copilot]

Pull request overview

This PR adds Zstandard (Zstd) support to Netty’s new Decompressor API as part of the decompressor migration tracked in #16743, including contract tests and a guard to cap Zstd window memory usage via setLongMax(...).

Changes:

• Introduces ZstdDecompressor backed by zstd-jni’s ZstdInputStreamNoFinalizer, including a maxWindowLog(int) builder option.
• Adds ZstdDecompressorTest extending the shared AbstractDecompressorTest contract suite and adds focused coverage for rejecting oversized Window_Log.
• Uses the shared CompressionUtil.DEFAULT_MAX_FORWARD_BYTES as the per-call output chunk size in takeOutput().

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File	Description
codec-compression/src/main/java/io/netty/handler/codec/compression/ZstdDecompressor.java	New Zstd implementation of the Decompressor API with configurable window-log cap and streaming output.
codec-compression/src/test/java/io/netty/handler/codec/compression/ZstdDecompressorTest.java	Adds contract coverage plus specific tests for maxWindowLog rejection/acceptance behavior using ZstdEncoder.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[chrisvest]

Had some comments.

[testlens-app]

🚨 TestLens detected 1 failed test 🚨

Here is what you can do:

1. Inspect the test failures carefully.
2. If you are convinced that some of the tests are flaky, you can mute them below.
3. Finally, trigger a rerun by checking the rerun checkbox.

Test Summary

Build PR / macos-aarch64-java11-boringssl build > Netty/Transport/Native/KQueue

Test	Runs	Flakiness
KQueueDatagramUnicastIPv6MappedTest > testSimpleSendWithConnect(TestInfo)	❌	5% 🟠

🏷️ Commit: 1310ac9
▶️ Tests: 33099 executed
⚪️ Checks: 19/19 completed

Test Failures

KQueueDatagramUnicastIPv6MappedTest > testSimpleSendWithConnect(TestInfo) (Netty/Transport/Native/KQueue in Build PR / macos-aarch64-java11-boringssl build)

org.opentest4j.AssertionFailedError
	at io.netty.testsuite.transport.socket.DatagramUnicastTest.testSimpleSendWithConnect0(DatagramUnicastTest.java:414)
	at io.netty.testsuite.transport.socket.DatagramUnicastTest.testSimpleSendWithConnect(DatagramUnicastTest.java:375)
	at io.netty.testsuite.transport.socket.DatagramUnicastTest.testSimpleSendWithConnect(DatagramUnicastTest.java:210)
	at io.netty.testsuite.transport.socket.DatagramUnicastTest$7.run(DatagramUnicastTest.java:204)
	at io.netty.testsuite.transport.socket.DatagramUnicastTest$7.run(DatagramUnicastTest.java:201)
	at io.netty.testsuite.transport.AbstractComboTestsuiteTest.run(AbstractComboTestsuiteTest.java:52)
	at io.netty.testsuite.transport.socket.DatagramUnicastTest.testSimpleSendWithConnect(DatagramUnicastTest.java:201)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1541)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1541)

Muted Tests

Select tests to mute in this pull request:

• KQueueDatagramUnicastIPv6MappedTest > testSimpleSendWithConnect(TestInfo)

Reuse successful test results:

• ♻️ Only rerun the tests that failed or were muted before

Click the checkbox to trigger a rerun:

• Rerun jobs

---

Learn more about TestLens at testlens.app.

[netty-project-bot]

Auto-port PR for 5.0: #17168