Mono issue with TLS due to Client Certificates

[OleksandrChizh]

Hi. We have a .NET 4.0 application for sending https(TLS 1.0) requests using HttpWebRequest. The application works correctly with the windows version of Mono but when we send a request on Ubuntu 16.04 using Mono 5.4.1.6 we get an error, see here: https://pastebin.com/hP9dZvuF
Just to test the app we sent a request to https://www.google.com, https://github.com, etc. and we got no errors but when we send a request to required server the issue appears.
We tried to import required certificates manually using certmgr and mozroots but it didn't solve the issue.
We are thinking it's an issue of Mono because we compiled the same code with .NET Core and it works well. Is there any chance to solve this issue?
Thank you!

CQNET: Error getting response stream (ReadDone1): ReceiveFailure - Error
System.Net.WebException: Error getting response stream (ReadDone1): ReceiveFailure ---> System.Security.Authentication.AuthenticationException: A call to SSPI failed, see inner exception. ---> Mono.Btls.MonoBtlsException: Ssl error:100000b6:SSL routines:OPENSSL_internal:NO_RENEGOTIATION
at /build/mono-5.4.1.6/external/boringssl/ssl/s3_pkt.c:446
at Mono.Btls.MonoBtlsContext.Read (System.Byte[] buffer, System.Int32 offset, System.Int32 size, System.Boolean& wantMore) [0x00050] in <50d80b08c1a5449282b22aedf03ce925>:0
at Mono.Net.Security.MobileAuthenticatedStream.ProcessRead (Mono.Net.Security.BufferOffsetSize userBuffer) [0x00011] in <50d80b08c1a5449282b22aedf03ce925>:0
at (wrapper remoting-invoke-with-check) Mono.Net.Security.MobileAuthenticatedStream:ProcessRead (Mono.Net.Security.BufferOffsetSize)
at Mono.Net.Security.AsyncReadRequest.Run (Mono.Net.Security.AsyncOperationStatus status) [0x0000c] in <50d80b08c1a5449282b22aedf03ce925>:0
at Mono.Net.Security.AsyncProtocolRequest+d__24.MoveNext () [0x000ff] in <50d80b08c1a5449282b22aedf03ce925>:0
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () [0x0000c] in <62f5937022004555807e6c57c33f6684>:0
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Threading.Tasks.Task task) [0x0003e] in <62f5937022004555807e6c57c33f6684>:0
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Threading.Tasks.Task task) [0x00028] in <62f5937022004555807e6c57c33f6684>:0
at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd (System.Threading.Tasks.Task task) [0x00008] in <62f5937022004555807e6c57c33f6684>:0
at System.Runtime.CompilerServices.ConfiguredTaskAwaitable+ConfiguredTaskAwaiter.GetResult () [0x00000] in <62f5937022004555807e6c57c33f6684>:0
at Mono.Net.Security.AsyncProtocolRequest+d__23.MoveNext () [0x0008b] in <50d80b08c1a5449282b22aedf03ce925>:0
--- End of inner exception stack trace ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () [0x0000c] in <62f5937022004555807e6c57c33f6684>:0
at Mono.Net.Security.MobileAuthenticatedStream+d__58.MoveNext () [0x001bf] in <50d80b08c1a5449282b22aedf03ce925>:0
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () [0x0000c] in <62f5937022004555807e6c57c33f6684>:0
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Threading.Tasks.Task task) [0x0003e] in <62f5937022004555807e6c57c33f6684>:0
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Threading.Tasks.Task task) [0x00028] in <62f5937022004555807e6c57c33f6684>:0
at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd (System.Threading.Tasks.Task task) [0x00008] in <62f5937022004555807e6c57c33f6684>:0
at System.Runtime.CompilerServices.TaskAwaiter1[TResult].GetResult () [0x00000] in <62f5937022004555807e6c57c33f6684>:0
at System.Threading.Tasks.TaskToApm.End[TResult] (System.IAsyncResult asyncResult) [0x0002e] in <62f5937022004555807e6c57c33f6684>:0
at Mono.Net.Security.MobileAuthenticatedStream.EndRead (System.IAsyncResult asyncResult) [0x00000] in <50d80b08c1a5449282b22aedf03ce925>:0
at System.Net.WebConnection.ReadDone (System.IAsyncResult result) [0x0001b] in <50d80b08c1a5449282b22aedf03ce925>:0
--- End of inner exception stack trace ---
at System.Net.HttpWebRequest.EndGetResponse (System.IAsyncResult asyncResult) [0x00058] in <50d80b08c1a5449282b22aedf03ce925>:0
at System.Net.HttpWebRequest.GetResponse () [0x0000e] in <50d80b08c1a5449282b22aedf03ce925>:0
at k.s (System.String bqi, System.Int32 bqj) [0x00043] in <31d60139ab8f4173b33ee35d33fa2fdb>

[brendanzagaeski]

Cross-referencing note to the Mono team for a couple existing reports about the not-yet implemented TLS renegotiation support for the Mono.Btls and Mono.AppleTls implementations:

• https://bugzilla.xamarin.com/show_bug.cgi?id=53912
• https://bugzilla.xamarin.com/show_bug.cgi?id=43332

[marek-safar]

/cc @baulig

[brendanzagaeski]

Related to the title change for this issue, note that use of client certificates (or more precisely, a server's request for client certificates after the initial TLS negotiation) is not strictly the only reason a TLS renegotiation can happen, but it is the typical reason.

[mysl]

Hi there. We met this problem when upgrading mono and using Btls provider (because our customer requires using TLS1.2). May I know if there is any plan about when the support for client certificate will be implemented? Thanks a lot!

[brendanzagaeski]

My comments on Xamarin Bugzilla Bug 57106 include some discussion about timelines for adding an up-to-date TLS renegotiation feature to Mono. Inclusion into a nightly build of course might happen sooner than the timelines mentioned there for inclusion into a "go-live" release. As also mentioned there, scheduling for work on the feature by the Xamarin Mono team will be discussed among the team during the first quarter of this year.

[mysl]

@brendanzagaeski @baulig Do you guys now have more concrete plan or estimate when this could be done? Thank you so much!

[nealef]

I have been looking at the renegotiation situation and found that if I set the renegotiation mode to freely (in this example) then when a handshake message comes in the btls will process this request and kick off the handshake process where it will generate a hello message. However, this message never gets sent. It gets put into a buffer by MobileAuthenticatedStream:InternalWrite but because we have already been through the finally clause of the ProcessAuthentication task and asyncHandshakeRequest has been cleared, the message never leaves the buffer (the result of asyncRequest = asyncHandshakeRequest ?? asyncWriteRequest in InternalWrite is null). Eventually, the server will timeout and close the connection.

[brendanzagaeski]

Yes, MobileAuthenticatedStream.InternalWrite() is indeed a key location involved in supporting renegotiation. See also Xamarin Bugzilla Bug 57106, Comment 11:

The MobileAuthenticatedStream.InternalWrite() method currently specifically checks that the connection mode is an SSLWrite() or SSLHandshake(), so it discards the message that comes from SSLRead(). After SSLRead() has "sent" (discarded) the "ClientHello" message, it goes back to trying to read the next reply from the server, but of course since the server is actually still waiting to receive the "ClientHello" message, nothing else happens, and the connection eventually times out.

In the end, altering InternalWrite() to handle messages from SSLRead() the same way as it handles messages from SSLWrite() might be a valid option, but it will need careful consideration. The full renegotiation process involves some other steps too, and care will need to be taken with attention to security to ensure that none of the new steps break any assumptions of the existing code.

[nealef]

Just did a quick experiment and changed InternalWrite:

var asyncRequest = asyncHandshakeRequest ?? asyncWriteRequest ?? asyncReadRequest;

The test case I am using then worked (this code works on .NET and using the legacy driver on Mono). It also worked with my openSSL provider that I've been working on. I am wondering what things need to be checked to ensure this is correct and robust?

[brendanzagaeski]

My understanding is that the things that need to be considered are unfortunately fairly subtle, involving how the SslStream read-write process as a whole is currently designed in Mono, and taking into account various scenarios such as network interruptions and stress-testing, things like packet injection and replay attacks, and various possibilities where users might interact directly with the SslStream API rather than going through HTTP requests. (Raw SslStream usage can include additional possibilities that don't occur in HTTP-only traffic. If I recall correctly, I think one of the trickier SslStream possibilities involves having reads and writes in progress at the same time.) As I understand it, it is that review and consideration work that the Mono team will be undertaking (tentatively in the timeframe of the next several months).

Martin might be able to share some technical corrections on that summary, but I think that at least roughly captures the flavor of the work to be done.

[nealef]

I assume their are additional considerations for the mono environment that requires the use of System/Mono.Net.Security such that using the CoreFx would be incomplete or incompatible?

[Florian-Rh]

Is there ANY fix or Workaround for this issue? As far as I can see, there is currently no way to secure a web request with a client certificate in Mono. That's a big Problem, isn't it?

[keke78ui9]

We had same issue when we call our API with client certificates accept. However, we recently upgrade xamarin.form from 2.3.4 to 2.5.0.280555, and change HttpClient implementation to Android, SSL/TLS implementation to Native TLS1.2+, and able to call our back-end API. Hopefully this information could help.

[brendanzagaeski]

As a limited additional piece of information, note that client certificates requested during the initial TLS handshake will work correctly. It is TLS renegotiation in particular that is not yet supported. So adjusting the server-side settings to request client certificates site-wide during the initial TLS handshake, for all traffic on the target TCP port, will allow client certificates to be used successfully. But that might not fit within the design constraints of all users, and it's only possible when the server software allows it.

[Rombersoft]

Hurriedly you closed issue. I have this problem.
Mono is

Mono JIT compiler version 5.12.0.226 (tarball Thu May  3 09:49:46 UTC 2018)
Copyright (C) 2002-2014 Novell, Inc, Xamarin Inc and Contributors. www.mono-project.com
	TLS:           __thread
	SIGSEGV:       altstack
	Notifications: epoll
	Architecture:  amd64
	Disabled:      none
	Misc:          softdebug 
	Interpreter:   yes
	LLVM:          supported, not enabled.
	GC:            sgen (concurrent by default)

Message

Error getting response stream (ReadDoneAsync2): ReceiveFailure

StackTrace

  at System.Net.WebResponseStream+<InitReadAsync>d__61.MoveNext () [0x000f5] in <fc308f916aec4e4283e0c1d4b761760a>:0 
--- End of stack trace from previous location where exception was thrown ---
  at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Threading.Tasks.Task task) [0x0003e] in <71d8ad678db34313b7f718a414dfcb25>:0 
  at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Threading.Tasks.Task task) [0x00028] in <71d8ad678db34313b7f718a414dfcb25>:0 
  at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd (System.Threading.Tasks.Task task) [0x00008] in <71d8ad678db34313b7f718a414dfcb25>:0 
  at System.Runtime.CompilerServices.ConfiguredTaskAwaitable+ConfiguredTaskAwaiter.GetResult () [0x00000] in <71d8ad678db34313b7f718a414dfcb25>:0 
  at System.Net.WebOperation+<Run>d__57.MoveNext () [0x001d9] in <fc308f916aec4e4283e0c1d4b761760a>:0 
--- End of stack trace from previous location where exception was thrown ---
  at System.Net.WebCompletionSource`1+<WaitForCompletion>d__15[T].MoveNext () [0x00094] in <fc308f916aec4e4283e0c1d4b761760a>:0 
--- End of stack trace from previous location where exception was thrown ---
  at System.Net.HttpWebRequest+<RunWithTimeoutWorker>d__244`1[T].MoveNext () [0x000ba] in <fc308f916aec4e4283e0c1d4b761760a>:0 
--- End of stack trace from previous location where exception was thrown ---
  at System.Net.HttpWebRequest.GetResponse () [0x00013] in <fc308f916aec4e4283e0c1d4b761760a>:0 
  at PanelAdministrating.MyControl.SendRequest (PanelAdministrating.VovaRequest request) [0x000c9] in /media/destructor/Documents/Projects/PanelAdministrating/PanelAdministrating/MyControl.cs:162 

Source Code

private string SendRequest(object request)
		{
			try
			{
                Uri url = new Uri(Settings.Instance.Serv);
				var data = Encoding.GetEncoding(1251).GetBytes(JsonConvert.SerializeObject(request));
				var stream = (HttpWebRequest) WebRequest.Create(url);
				stream.UserAgent = "RegulPay/3.0";
				stream.Method = "POST";
				stream.ProtocolVersion = HttpVersion.Version10;
				stream.ContentType = "application/json";
				stream.Timeout = 50000;
				stream.ContentLength = data.Length;
				stream.Headers.Add("Accept-Charset", "windows-1251,utf-8;q=0.7,*;q=0.7");
				stream.KeepAlive = false;
				stream.ContentLength = data.Length;
				var newStream = stream.GetRequestStream();
				newStream.Write(data, 0, data.Length);
                		var response = stream.GetResponse();
				var reader = new StreamReader(response.GetResponseStream(), Encoding.GetEncoding(1251));
				string ab = reader.ReadToEnd();
				switch(response.ContentType)
				{
					case "zip":
					ab = Unzip(ab);
						break;
					case "json":
						break;
				}
                reader.Dispose();
                response.Dispose();
                newStream.Dispose();
				return ab;
			}
			catch (Exception e)
			{
				return  "{\"Result\":-111,\"Message\":\"" + e.ToString() + "\"}";
			}
		}

internet on LAN cable. This error throws not always but often.

[Florian-Rh]

While it might be fixed inside the mono framework, that's not a big help for those who use Xamarin, as the updated mono version has not been included in Xamarin so far. In that case, as far as I know, the only option is a native workaround for each platform. It took me quite some time to figure this out, so maybe I can safe someone some time by sharing this native Android workaround for Xamarin developers who had the same problem:

        public async Task<string> SendAuthenticatedWebRequest(string requestUri, string method, byte[] clientCertificateDump, char[] clientCertificatePassword)
        {
            //Include trusted root certificate (public key)
            var trustStore = KeyStore.GetInstance(KeyStore.DefaultType);
            var trustManagerFactory = TrustManagerFactory.GetInstance("X509");
            Java.Security.Cert.X509Certificate trustCertificate = null;
            using (var rootCertStream = Application.Context.Assets.Open("rootcert.cer"))
            {
                CertificateFactory certificatFactory = CertificateFactory.GetInstance("X.509");
                trustCertificate = (Java.Security.Cert.X509Certificate)certificatFactory.GenerateCertificate(rootCertStream);
                string alias = trustCertificate.SubjectX500Principal.Name;
                trustStore.Load(null);
                trustStore.SetCertificateEntry(alias, trustCertificate);
            }
            trustManagerFactory.Init(trustStore);
            var trustManagers = trustManagerFactory.GetTrustManagers();

            //include client certificate
            MemoryStream ms = new MemoryStream(clientCertificateDump);
            var keystore = KeyStore.GetInstance("PKCS12");
            keystore.Load(ms, clientCertificatePassword);
            KeyManagerFactory kmf = KeyManagerFactory.GetInstance("X509");
            kmf.Init(keystore, clientCertificatePassword);
            var keyManagers = kmf.GetKeyManagers();
        
            //initialize SSL Context
            SSLContext sSLContext = SSLContext.GetInstance("TLS");
            sSLContext.Init(keyManagers, trustManagers, null);

            String result = null;
            HttpsURLConnection urlConnection = null;
        
            try
            {
                //OPEN HTTPS Connection
                URL requestedUrl = new URL(requestUri);
                urlConnection = (HttpsURLConnection)requestedUrl.OpenConnection();
                if (urlConnection is HttpsURLConnection)
                {
                    ((HttpsURLConnection)urlConnection).SSLSocketFactory = sSLContext.SocketFactory;
                }
                urlConnection.RequestMethod = method;

                //SEND your request
                result = await Task.Run(async () =>
                {
                    StreamReader sr = new StreamReader(urlConnection.InputStream);
                    return await sr.ReadToEndAsync();
                });
                var lastContentType = urlConnection.ContentType;
            }
            catch (Exception ex)
            {
                result = ex.ToString();
            }
            finally
            {
                //CLOSE HTTPS Connection
                if (urlConnection != null)
                {
                    urlConnection.Disconnect();
                }
            }
            return result;
        }

[acaliaro]

@keke78ui9 could you share your Android implementation?

[acaliaro]

why this is closed?

[keke78ui9]

@acaliaro I'm using Xamarin.Form and use HttpClient at shared projects. The HttpClient class is very standard code, not many special things there but I could show a simple example as follows.

using (var client = new HttpClient())
{
	HttpResponseMessage apiRes = null;
	switch (httpMethod)
	{
		case HttpMethodType.Post:
			apiRes = await client.PostAsync(requestUri, content);
			break;
		case HttpMethodType.Get:
			apiRes = await client.GetAsync(requestUri);
			break;
	}

As I mentioned before, In my case upgrade to higher version and change Android HttpClient and SSL/TLS setting helped me fixed my issue. I attached the Android Options setting for references.

[craylward]

Pretty sure my issue with Xamarin.Android and Azure is related to this: Azure/azure-iot-sdk-csharp#561