TLS with EdDSA private keys not supported with OpenSSL

[SanjayVas]

Despite the fact that OpenSSL supports EdDSA with the ed25519 and ed448 algorithms as of 1.1.1 (and BoringSSL for even longer), Netty fails when using these algorithms.

From the stack trace below, it appears that Netty doesn't try the EdDSA Java private key type for the ECDHE_ECDSA auth method. EdDSA is supported with that auth method in TLS 1.3 and with an extension to TLS 1.2.

Note that the EdDSA key type is only available in Java 15+ or by using a JCE provider such as BouncyCastle.

See the related issue filed for grpc-java: grpc/grpc-java#7774

Expected behavior

Server TLS succeeds when using a certificate with the ED25519 or ED448 signature algorithm (Java private key type EdDSA).

Actual behavior

Server TLS fails with the following stack trace:

io.netty.handler.codec.DecoderException: javax.net.ssl.SSLHandshakeException: Unable to find key material for auth method(s): [ECDHE_ECDSA, ECDHE_ECDSA, ECDHE_RSA, ECDHE_RSA, ECDHE_RSA, ECDHE_RSA, RSA, RSA, RSA]
        at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:478)
        at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
        at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
        at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
        at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166)
        at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:719)
        at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:655)
        at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:581)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
        at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
        at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
        at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: javax.net.ssl.SSLHandshakeException: Unable to find key material for auth method(s): [ECDHE_ECDSA, ECDHE_ECDSA, ECDHE_RSA, ECDHE_RSA, ECDHE_RSA, ECDHE_RSA, RSA, RSA, RSA]
        at io.netty.handler.ssl.OpenSslKeyMaterialManager.setKeyMaterialServerSide(OpenSslKeyMaterialManager.java:90)
        at io.netty.handler.ssl.ReferenceCountedOpenSslServerContext$OpenSslServerCertificateCallback.handle(ReferenceCountedOpenSslServerContext.java:229)
        at io.netty.internal.tcnative.SSL.readFromSSL(Native Method)
        at io.netty.handler.ssl.ReferenceCountedOpenSslEngine.readPlaintextData(ReferenceCountedOpenSslEngine.java:596)
        at io.netty.handler.ssl.ReferenceCountedOpenSslEngine.unwrap(ReferenceCountedOpenSslEngine.java:1203)
        at io.netty.handler.ssl.ReferenceCountedOpenSslEngine.unwrap(ReferenceCountedOpenSslEngine.java:1325)
        at io.netty.handler.ssl.ReferenceCountedOpenSslEngine.unwrap(ReferenceCountedOpenSslEngine.java:1368)
        at io.netty.handler.ssl.SslHandler$SslEngineType$1.unwrap(SslHandler.java:206)
        at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1387)
        at io.netty.handler.ssl.SslHandler.decodeNonJdkCompatible(SslHandler.java:1294)
        at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1331)
        at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:508)
        at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:447)
        ... 17 more

Steps to reproduce

1. Generate a self-signed CA certificate and private key.
2. Generate and sign certificate and private key pairs for the RSA and ED25519 algorithms.
3. Run a Netty server with the RSA certificate and private key.
4. Test the server using the openssl s_client command, specifying the CA certificate.
5. Observe that the TLS handshake succeeds.
6. Run the Netty server with the ED25519 certificate and private key.
7. Test the server using the openssl s_client command, specifying the CA certificate.
8. Observe that the TLS handshake fails and that the server throws the above exception.

Minimal yet complete reproducer code (or URL to code)

https://github.com/SanjayVas/netty-tls-example

Netty version

netty-all:4.1.56.Final (Maven)
netty-tcnative-boringssl-static:2.0.35.Final (Maven)

JVM version (e.g. java -version)

openjdk version "11.0.6" 2020-01-14 LTS
OpenJDK Runtime Environment Zulu11.37+17-CA (build 11.0.6+10-LTS)
OpenJDK 64-Bit Server VM Zulu11.37+17-CA (build 11.0.6+10-LTS, mixed mode)

OS version

x86_64 GNU/Linux (Debian Testing "Bullseye")

[normanmaurer]

Will have a look... thanks!

[normanmaurer]

Also not supported at the moment in Conscrypt: google/conscrypt#829

[normanmaurer]

Related to http://openjdk.java.net/jeps/339

[umapriyadarsi]

It almost exactly a year since this issue was opened, and I too am having this issue.

My environment:
I am using grpc-netty-shaded-1.43.1.jar from grpc/grpc-java
JDK 17
M1 macOS Monterery
BouncyCastle

Is there any update on this? Everything is fine when I am using RSA keys, and also EcDSA keys on P-256,P-384 and P-521 NIST curves. I even tested RSA with both TLS 1.2 and TLS 1.3 protocols, and the EcDSA with 1.3 alone.

But the moment I try it with Ed25519 or Ed448 keys, stuff hits the fan.

@SanjayVas were you able to solve it, or try any workaround?

@normanmaurer any pointers on how to proceed would help. I also check the OpenJDK link you mentioned, and created my key pair as mentioned in that link, but that does not solve the issue either.

[SanjayVas]

@SanjayVas were you able to solve it, or try any workaround?

I was not. My project is currently stuck using ECDSA with a NIST curve.

[helios2k6]

This issue is also blocking us as well. Is there any timeline or thought going into this issue?

[johannww]

This is also affecting my pull request for Hyperledger Fabric here.

The Fabric Java gateway cannot communicate with a peer when the peer uses an ed25519 TLS certificate.

[dparmar]

I am also facing this issue. Is anyone planning to look into it?