Arrow memory allocation fails on Java 25: NettyAllocationManager static initializer throws UnsupportedOperationException

[wangyum]

Problem

When attempting to run Apache Iceberg on Java 25, the Arrow memory subsystem fails to initialize with an ExceptionInInitializerError. The build currently gates on Java 17 or 21 (build.gradle:54), but attempting to support Java 25 surfaces an incompatibility deep in the Arrow/Netty stack.

Environment

• Java version: 25
• Apache Iceberg version: main branch
• Arrow version: 15.0.2
• Netty version: 4.2.12.Final

Stack Trace

java.lang.ExceptionInInitializerError
    at org.apache.arrow.memory.DefaultAllocationManagerFactory.<clinit>(DefaultAllocationManagerFactory.java:26)
    at java.base/java.lang.Class.forName0(Native Method)
    at java.base/java.lang.Class.forName(Class.java:467)
    at java.base/java.lang.Class.forName(Class.java:458)
    at org.apache.arrow.memory.DefaultAllocationManagerOption.getFactory(DefaultAllocationManagerOption.java:108)
    at org.apache.arrow.memory.DefaultAllocationManagerOption.getDefaultAllocationManagerFactory(DefaultAllocationManagerOption.java:98)
    at org.apache.arrow.memory.BaseAllocator$Config.getAllocationManagerFactory(BaseAllocator.java:773)
    at org.apache.arrow.memory.ImmutableConfig.access$801(ImmutableConfig.java:24)
    at org.apache.arrow.memory.ImmutableConfig$InitShim.getAllocationManagerFactory(ImmutableConfig.java:83)
    at org.apache.arrow.memory.ImmutableConfig.<init>(ImmutableConfig.java:47)
    at org.apache.arrow.memory.ImmutableConfig.<init>(ImmutableConfig.java:24)
    at org.apache.arrow.memory.ImmutableConfig$Builder.build(ImmutableConfig.java:485)
    at org.apache.arrow.memory.BaseAllocator.<clinit>(BaseAllocator.java:62)
    at Reproducer.main(Reproducer.java:8)
Caused by: java.lang.UnsupportedOperationException
    at io.netty.buffer.EmptyByteBuf.memoryAddress(EmptyByteBuf.java:961)
    at io.netty.buffer.DuplicatedByteBuf.memoryAddress(DuplicatedByteBuf.java:115)
    at io.netty.buffer.UnsafeDirectLittleEndian.<init>(UnsafeDirectLittleEndian.java:55)
    at io.netty.buffer.UnsafeDirectLittleEndian.<init>(UnsafeDirectLittleEndian.java:40)
    at io.netty.buffer.PooledByteBufAllocatorL.<init>(PooledByteBufAllocatorL.java:50)
    at org.apache.arrow.memory.NettyAllocationManager.<clinit>(NettyAllocationManager.java:51)
    ... 14 more

Root Cause

The failure originates in NettyAllocationManager's static initializer, which instantiates PooledByteBufAllocatorL. That allocator internally constructs a UnsafeDirectLittleEndian wrapper and probes the native memory address of a DuplicatedByteBuf(EmptyByteBuf). On Java 25, the sun.misc.Unsafe direct memory access path used by older Netty is broken for this use case, causing EmptyByteBuf.memoryAddress() to throw UnsupportedOperationException.

This is a layered incompatibility:

1. Netty 4.x (pre-5.0) relies on sun.misc.Unsafe for native direct memory address resolution.
2. Java 21+ increasingly restricts these patterns, and Java 25 (with JEP 471 deprecating Unsafe memory-access methods for removal) breaks them outright.
3. Arrow 15.0.2 depends on this legacy Netty allocator path via arrow-memory-netty.

The iceberg-arrow module pulls in both arrow-memory-netty and arrow-memory-unsafe, both of which use the failing static initializer path through BaseAllocator.

Affected Code

• build.gradle:54 — Java version guard (currently rejects anything other than JDK 17/21 at build time)
• build.gradle:951-958 — iceberg-arrow depends on arrow-memory-netty and arrow-memory-unsafe
• libs.versions.toml:32 — arrow = "15.0.2"
• libs.versions.toml:79 — netty-buffer = "4.2.12.Final"

Possible Solutions

1. Upgrade Arrow — Arrow 16+ or later may include fixes for Java 21/25 compatibility that avoid the broken PooledByteBufAllocatorL initialization. This is the most sustainable path.

2. Use the arrow-memory-unsafe allocator exclusively — Arrow provides a Netty-free allocator path. Setting the system property -Darrow.memory.allocator=unsafe or restructuring dependencies to rely solely on arrow-memory-unsafe (bypassing arrow-memory-netty) may avoid the failing static initializer.

3. Extend the Java version guard — Regardless of the dependency fix chosen, build.gradle:54 needs to be updated to allow Java 25 once the underlying incompatibility is resolved.

References

• JEP 471: Deprecate the Memory-Access Methods in sun.misc.Unsafe for Removal
• Apache Arrow Java compatibility tracking
• Netty issue: PooledByteBufAllocatorL unsafe memory access broken on modern JDKs