SslStream.AuthenticateAsClientAsync behaves wrong with TLS 1.3

[jbe2277]

Description

I need to connect to a server via TcpClient and SslStream which requires TLS >= 1.2. The server can be configured to require a valid client certificate (TLS mutual authentication).

My scenario: Server is configured to require a client certificate but my client calls SslStream.AuthenticateAsClientAsync without any client certificates.

Client environment:

• Windows (>= v10)
• .NET 6

Reproduction Steps

None

Expected behavior

The method AuthenticateAsClientAsync throws an AuthenticationException because the server requires a valid client certificate, but none is provided.

This works as expected on Win10 20H2 with TLS 1.2.

• Note: This Windows version does not support TLS 1.3 and because of that TLS 1.2 is used.

Actual behavior

• Environment: Win11 21H2 with TLS 1.3
• The AuthenticateAsClientAsync method returns successfully and the SslStream.IsAuthenticated property returns true.
  • WRONG BEHAVIOR as the authentication could not be successfully without providing the client certificates.
• However, when the SslStream is used the following exception is thrown:

System.IO.IOException
  HResult=0x80131620
  Message=The decryption operation failed, see inner exception.
  Source=System.Net.Security
  StackTrace:
   at System.Net.Security.SslStream.<ReadAsyncInternal>d__186`1.MoveNext()
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1.ConfiguredTaskAwaiter.GetResult()
   ...
   
Inner Exception 1:
Win32Exception: The message received was unexpected or badly formatted.

Regression?

TLS 1.2: Correct behavior
TLS 1.3: Wrong behavior

Known Workarounds

No response

Configuration

No response

Other information

No response

Tagging subscribers to this area: @dotnet/ncl, @vcsjones
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Description

I need to connect to a server via TcpClient and SslStream which requires TLS >= 1.2. The server can be configured to require a valid client certificate (TLS mutual authentication).

My scenario: Server is configured to require a client certificate but my client calls SslStream.AuthenticateAsClientAsync without any client certificates.

Client environment:

• Windows (>= v10)
• .NET 6

Reproduction Steps

None

Expected behavior

The method AuthenticateAsClientAsync throws an AuthenticationException because the server requires a valid client certificate, but none is provided.

This works as expected on Win10 20H2 with TLS 1.2.

• Note: This Windows version does not support TLS 1.3 and because of that TLS 1.2 is used.

Actual behavior

• Environment: Win11 21H2 with TLS 1.3
• The AuthenticateAsClientAsync method returns successfully and the SslStream.IsAuthenticated property returns true.
  • WRONG BEHAVIOR as the authentication could not be successfully without providing the client certificates.
• However, when the SslStream is used the following exception is thrown:

System.IO.IOException
  HResult=0x80131620
  Message=The decryption operation failed, see inner exception.
  Source=System.Net.Security
  StackTrace:
   at System.Net.Security.SslStream.<ReadAsyncInternal>d__186`1.MoveNext()
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1.ConfiguredTaskAwaiter.GetResult()
   ...
   
Inner Exception 1:
Win32Exception: The message received was unexpected or badly formatted.

Regression?

TLS 1.2: Correct behavior
TLS 1.3: Wrong behavior

Known Workarounds

No response

Configuration

No response

Other information

No response

Author:	jbe2277
Assignees:	-
Labels:	area-System.Net.Security, untriaged
Milestone:	-

[wfurt]

Can you post simple repro @jbe2277? There was Windows bug where it would fail with 0x80090317 in scenario like yours. That should be fixed if you are running with all updates.

SslStream.IsAuthenticated has no relation to client certificate. On Client, it will be true as long as you complete handshake and access server's certificate.

[jbe2277]

1. My Windows 11 machine has all updates installed. The issue still occurs.
2. I tried to create a simple repo:
   • see: https://github.com/jbe2277/TlsClientAuthIssue
   • In my repo I use .NET 6 for the server side too. This behaves quite different to my scenario where the server is implemented with C++, Linux and OpenSSL. I don't have details as the server side is not in my control.
   • with my repo I see other issues which might be related with the issue I have reported originally:

When the client does not provide the client certificate the AuthenticateAsClient still pass successfully. Also the client.Write will not detect an error. Only when using client.Read we get a very generic IOException:

clientTask Exception: System.IO.IOException: The decryption operation failed, see inner exception.
 ---> System.ComponentModel.Win32Exception (0x80090327): An unknown error occurred while processing the certificate.

Use case: I need to get the information on the client side that the TLS mutual authentication failed because of a missing or wrong client certificate. In such case we will show how the user can solve it. But with a generic IOException which behaves very different on how the TLS on the server side is implemented it makes this very difficult.

Expected: I would expect that AuthenticateAsClient throws the AuthenticationException when the TLS mutual authentication fails.

Here is the output of my simple repo. Changing from Tls12 to Tls13 shows the same behavior here:

127.0.0.1:51938
--- Success ---------------------------------------------------------------------------
clientCert: CN=testclienteku.contoso.com
serverCert: CN=testservereku.contoso.com
requireClientCert: True

client.IsAuthenticated: True
server.IsAuthenticated: True
client.IsMutuallyAuthenticated: True
server.IsMutuallyAuthenticated: True
client.SslProtocol: Tls12
client.NegotiatedCipherSuite: TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384

Client: result received from server: HELLO
--- Missing client cert ---------------------------------------------------------------
clientCert:
serverCert: CN=testservereku.contoso.com
requireClientCert: True

serverAuthTask Exception: System.Security.Authentication.AuthenticationException: The remote certificate was rejected by the provided RemoteCertificateValidationCallback.
   at System.Net.Security.SslStream.SendAuthResetSignal(ProtocolToken message, ExceptionDispatchInfo exception)
   at System.Net.Security.SslStream.CompleteHandshake(SslAuthenticationOptions sslAuthenticationOptions)
   at System.Net.Security.SslStream.ForceAuthenticationAsync[TIOAdapter](TIOAdapter adapter, Boolean receiveFirst, Byte[] reAuthenticationData, Boolean isApm)
   at Program.<>c__DisplayClass0_0.<<<Main>$>g__Run|0>d.MoveNext() in C:\Dev\GitHub\TlsClientAuthIssue\TlsClientAuthIssue\MinimalRepro\Program.cs:line 51

client.IsAuthenticated: True
server.IsAuthenticated: False
client.IsMutuallyAuthenticated: False
server.IsMutuallyAuthenticated: False
client.SslProtocol: Tls12
client.NegotiatedCipherSuite: TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384


clientTask Exception: System.IO.IOException: The decryption operation failed, see inner exception.
 ---> System.ComponentModel.Win32Exception (0x80090327): An unknown error occurred while processing the certificate.
   --- End of inner exception stack trace ---
   at System.Net.Security.SslStream.ReadAsyncInternal[TIOAdapter](TIOAdapter adapter, Memory`1 buffer)
   at Program.<>c__DisplayClass0_1.<<<Main>$>g__ClientWriteAndRead|1>d.MoveNext() in C:\Dev\GitHub\TlsClientAuthIssue\TlsClientAuthIssue\MinimalRepro\Program.cs:line 75
--- End of stack trace from previous location ---
   at Program.<>c__DisplayClass0_0.<<<Main>$>g__Run|0>d.MoveNext() in C:\Dev\GitHub\TlsClientAuthIssue\TlsClientAuthIssue\MinimalRepro\Program.cs:line 66

serverTask Exception: System.Security.Authentication.AuthenticationException: The remote certificate was rejected by the provided RemoteCertificateValidationCallback.
   at System.Net.Security.SslStream.<ThrowIfExceptional>g__ThrowExceptional|138_0(ExceptionDispatchInfo e)
   at System.Net.Security.SslStream.ReadAsync(Memory`1 buffer, CancellationToken cancellationToken)
   at Program.<>c__DisplayClass0_1.<<<Main>$>g__ServerReadAndWrite|2>d.MoveNext() in C:\Dev\GitHub\TlsClientAuthIssue\TlsClientAuthIssue\MinimalRepro\Program.cs:line 83
--- End of stack trace from previous location ---
   at Program.<>c__DisplayClass0_0.<<<Main>$>g__Run|0>d.MoveNext() in C:\Dev\GitHub\TlsClientAuthIssue\TlsClientAuthIssue\MinimalRepro\Program.cs:line 68

[wfurt]

I think I understand now more what is happening. This is running TLS 12 on W11:

The peers exchange secrets in packets 62 & 64. That makes the handshake complete & successful. Then the server sends Encrypted Alert as separate message to indicate validation problem but that is already handled by the Read on Client. The error massage is not great but it does come from underlying Windows stack.

It is certainly possible the exchange or timing is different with OpenSSL. I don't think this is generally fixable as it depends on timing and particular stack behavior. I put up some mixup code to SslStream to recognize the alter but that will not work with Tls 1.3 as everything after is visible only as application data (and SslStream does not have access the the session encryption key)

You could possible use IsMutuallyAuthenticated as hint but even that is not quite safe as the client may send certificate server may reject later.

[antonfirsov]

Triage: we should improve the error messages, and see if we can do better.

[jbe2277]

@wfurt Thanks for the explanation.

Unfortunately, using IsMutuallyAuthenticated is not reliable. The following issue is something we run into:

#65563 SslStream.IsMutuallyAuthenticated returns wrong value when cached creds are used

[wfurt]

I know. But as I mentioned I don't see anything we can do to force the exception to AuthenticateAsClientAsync because the alert arrives after the handshake is completed.
My only plan for now is to look for possibility of better error.

[jbe2277]

I understand. A better error would help as well. Thanks.

[rzikm]

Triage: The behavioral difference is due to differences between TLS 1.3 and TLS 1.2, and there is nothing reasonable we can do to unify this behavior across versions.