-
Notifications
You must be signed in to change notification settings - Fork 605
CVE 2026 63566
Issue affecting: BC C# .NET 2.6.2 and earlier, and the 2.7.0-beta.98 pre-release.
Fixed versions: BC C# .NET 2.7.0
Platform affected: All CLRs.
DTLS sends handshake messages in fragments, and each fragment states the total length of the message it belongs to in a 24-bit field chosen by the sender. Earlier versions allocated the reassembly buffer for a message at that declared length as soon as its first fragment arrived. They did not check it against the maximum handshake message size, a limit the TLS implementation already enforced. A fragment carrying no data was enough to commit almost 16 MB, and up to 16 messages can be pending at a time. A single datagram of a few hundred bytes could therefore make one handshake allocate about 256 MB. This happens in the unencrypted part of the handshake, before the peer has been authenticated.
Applications using DtlsServerProtocol or DtlsClientProtocol are exposed: a DTLS server to any client that can reach it, and a DTLS client to the server it connects to. A server that uses DtlsVerifier creates handshake state only after a client has answered its cookie challenge. That rules out spoofed source addresses, but not an attacker using their own address. By default a DTLS handshake has no overall timeout, so the memory stays allocated for as long as the attacker keeps the handshake open. Repeating this over several handshakes can exhaust process memory and cause an OutOfMemoryException that affects the whole application. TLS (TlsClientProtocol, TlsServerProtocol) is not affected.
BC C# .NET 2.7.0 applies the maximum handshake message size to DTLS as well. A fragment that declares a longer message is discarded before any buffer is allocated. The limit is the value returned by GetMaxHandshakeMessageSize() on the TlsClient or TlsServer, 32768 bytes by default, with a minimum of 1024. An oversized message is ignored rather than answered with an alert, so a handshake that legitimately needs larger messages, for example to carry a long certificate chain, will not complete. Applications in that situation should override GetMaxHandshakeMessageSize() to raise the limit.
Earlier versions have no setting that prevents this, because GetMaxHandshakeMessageSize() is only consulted for TLS, so upgrading is recommended. Where that is not immediately possible, there are three ways to reduce the exposure: set a finite handshake timeout through GetHandshakeTimeoutMillis(), so that the memory held by a stalled handshake is released; use DtlsVerifier on servers; and limit the number of DTLS handshakes in progress at the same time.
The corresponding issue in BC Java was fixed in BC Java 1.85 (CVE-2026-59646).
Fix Commits:
- https://github.com/bcgit/bc-csharp/commit/212d06474166d5e85be72c2dd4898313aa5dd5f3 (adds the limit)
- https://github.com/bcgit/bc-csharp/commit/99afef612458f70f28e7824a09719bfc3a169bda (moves the 1024-byte minimum into a shared helper)
Credit: Discovered by Claude, Anthropic's AI assistant, and triaged by the Anthropic security team in collaboration with Anthropic Research.