Dart/Pub HTTP Certificate Issues

[JSanford42]

I have done extensive research on this issue prior to this post. I have been scouring open and closed issues on Github and any form of related posts on Stack Overflow. I am using Dart 2.12.4 on Windows 10.

I feel I should set the stage regarding my environment. I have no control over the network configuration. I do not have administrator privileges on my machine. I am unable (and unwilling) to disable the virus scanner. I am unable to update most applications myself. I am behind a proxy. The proxy automatically uses an internal CA and certificate chain if a site's certificates are not already available in the certificate store on the proxy.

I have done extensive testing using the certificate check code from dart-lang/pub#983 posted by DartBot on June 5, 2015. I modified it to comply with null safety but otherwise left it intact.

Here are the tests I have performed. All tests have the HTTP_PROXY variables set. Without them set, I get semaphore timeout errors on all connections. The results of these tests are the same when certificates are provided via the DART_VM_OPTIONS variable.

• I connect to a site where the certificates are known to the proxy and valid. The bad certificate callback is not executed.
• I connect to a site where the certificates are not known to the proxy so the internal certificates are used. The bad certificate callback is executed. The same URL (https://pub.dartlang.org/api/packages/pedantic) loads fine in the browser and has no certificate errors.
• I set the trusted certificates to the internal ones using SecurityContext.setTrustedCertificates. The bad certificate callback is executed.
• I set the certificate chain to the internal one using SecurityContext.useCertificateChain. The bad certificate callback is executed.
• I installed the Google Trust Service root and subordinate certificates into the user certificate store. The bad certificate callback is executed. This shows that the proxy is using the internal certificates if the proxy itself does not have the site certificates.

I have verified that every certificate in the certificate path are installed on my machine and match the ones used by the proxy.

My own code is easy to fix. I set the bad certificate callback to return true and everything moves along just fine. I would prefer not to do that but it is the only option available.

I have a problem with Pub not providing the ability to specify trusted hosts or disable strict SSL checks. It has been made abundantly clear that Dart uses an internal certificate database and that there is no way to append/supplement that. There was an update recently that stated that Dart would use the Windows certificate store but that appears to not work.

[jonasfj]

The proxy automatically uses an internal CA and certificate chain if a site's certificates are not already available in the certificate store on the proxy.

What does this mean?

Does the connection between the client and proxy use TLS or just HTTP?
How is the proxy communicating the certificate chain used by the server to the client?

I'm confused would the proxy behave differently if pub.dartlang.org were using a different certificate that is known to the proxy?

(Maybe I'm not fully updated on how HTTP proxies work).

---

Did this use to work? We've recently seen pub.dartlang.org switch to using a new certificate chain after automatic renewal.

I modified it to comply with null safety but otherwise left it intact.

Consider posting a gist for completeness :)

---

I have a problem with Pub not providing the ability to specify trusted hosts or disable strict SSL checks....There was an update recently that stated that Dart would use the Windows certificate store but that appears to not work.

That sounds like a problem. It's probably reasonable to have a way to use self-signed certificates with Dart SDK.

Just to clarify:

• Dart SDK connect using HTTP_PROXY to sites that have a certificate-chain your proxy has installed, and,
• Dart SDK cannot connect using HTTP_PROXY to sites that have a certificate-chain your proxy doesn't support.

I'm unsure why this is...? is the proxy man-in-the-middle inspecting TLS connections, but unable to do so for certain certificates? without having to use an internal self-signed certificate that you would have to install into the Dart certificate store (or windows certificate store, if Dart SDK is using those).

[JSanford42]

The proxy automatically uses an internal CA and certificate chain if a site's certificates are not already available in the certificate store on the proxy.

What does this mean?

Does the connection between the client and proxy use TLS or just HTTP?
How is the proxy communicating the certificate chain used by the server to the client?

I'm confused would the proxy behave differently if pub.dartlang.org were using a different certificate that is known to the proxy?

(Maybe I'm not fully updated on how HTTP proxies work).
I look at pub.dartlang.org in a browser that is not on my network. I see the GTS and GlobalSign certificate chain. When using a browser on the network, I see the certificate chain for the internal CA. The connection from my machine to the proxy is HTTPS.

This behavior is not for every site. When I view www.microsoft.com both on the network and off the network I see the certificate chain from the host server, not the internal chain.

This points to the proxy using the internal certificates to ensure a valid HTTPS connection on the internal network when it does not have the up to date certificates for the host server.

Did this use to work? We've recently seen pub.dartlang.org switch to using a new certificate chain after automatic renewal.

I modified it to comply with null safety but otherwise left it intact.

Consider posting a gist for completeness :)

It did work in the past. I had to stop my Dart development for a while due to another project. I encountered this when I returned to my code a few months ago.

I wish I could post a gist of my changes. My organization blocks gist for some reason. The only change was to the printCertificate function. The certificate parameter was changed to nullable and a null check if block was added around the method body.

I have a problem with Pub not providing the ability to specify trusted hosts or disable strict SSL checks....There was an update recently that stated that Dart would use the Windows certificate store but that appears to not work.

That sounds like a problem. It's probably reasonable to have a way to use self-signed certificates with Dart SDK.

Just to clarify:

• Dart SDK connect using HTTP_PROXY to sites that have a certificate-chain your proxy has installed, and,
• Dart SDK cannot connect using HTTP_PROXY to sites that have a certificate-chain your proxy doesn't support.

I'm unsure why this is...? is the proxy man-in-the-middle inspecting TLS connections, but unable to do so for certain certificates? without having to use an internal self-signed certificate that you would have to install into the Dart certificate store (or windows certificate store, if Dart SDK is using those).

I am not privy to the details of the network configuration. I can only test various possibilities and draw conclusions from the results. I did test npm and pip since they are not as forgiving as Microsoft package managers like NuGet.

npm works and I see the internal certificates when viewing it in a browser. It does not have a problem with the internal certificates.

pip had a problem with the certificates but it can be configured to set a list of trusted hosts. I set the list to the 3 hosts used to install packages and it worked fine.

The more I think about this issue, the more I think I should post this in the pub repo. It occurred to me that I am not looking for a fix for this in Dart but in pub. I am looking for the ability to configure pub to allow for trusted hosts so I can download and update packages and not worry about how my network may change from day to day.

A fix for this in Dart itself does not make much sense. A developer that writes something using HTTP that wants to allow this functionality can do it in the implementation of HttpClient.badCertificateCallback.

[jonasfj]

This points to the proxy using the internal certificates to ensure a valid HTTPS connection on the internal network when it does not have the up to date certificates for the host server.

So maybe the proxy is allowing traffic for microsoft.com to pass through without looking into the traffic. And a possible solution is to ask your network administrator to pass-through TLS traffic for pub.dartlang.org and storage.googleapis.com.

I hope the proxy doesn't have keys to pretend to be microsoft.com.

A fix for this in Dart itself does not make much sense.

It might make sense to ensure that Dart is correctly using the system certificate stores on windows.

I think it's a lot better to have the Dart SDK using system certificates rather than expanding dart pub to support using custom certificates.