CVE‐2026‐59646

Title: DTLS handshake reassembler allocates buffer from unchecked 24-bit length.

Issue affecting: BC before 1.85, BC-LTS before 2.73.12, BC-FJA before bctls-fips 1.0.24, 2.0.24 and 2.1.24.

Fixed versions: BC 1.85, BC-LTS 2.73.12, BC-FJA bctls-fips 1.0.24, 2.0.24 and 2.1.24.

Platform affected: Java 8 and later.

In Bouncy Castle's DTLS stack, DTLSReliableHandshake.processRecord() reads the 24-bit handshake 'length' field from incoming records and constructs a new DTLSReassembler for each in-window message_seq, whose constructor immediately allocates a byte[length] buffer. Unlike the stream-TLS path, no check against peer.getMaxHandshakeMessageSize() is applied. Because up to 16 message_seq values may be buffered per connection, a single ~205-byte epoch-0 record containing 16 fragment headers with length=0xFFFFFF and fragment_length=0 forces ~256MiB of allocation before any authentication. An unauthenticated remote attacker can drive the JVM to OutOfMemoryError with a handful of tiny UDP datagrams.

The fix was introduced in commit 2ea38942c791 and commit 2d98721e71bb.

Credit: Alex Gaynor in collaboration with Claude and Anthropic Research.