Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Custom SNI (Server Name Indication) support in HttpClient #20876

Closed
alukyan opened this issue Apr 3, 2017 · 9 comments
Closed

Custom SNI (Server Name Indication) support in HttpClient #20876

alukyan opened this issue Apr 3, 2017 · 9 comments
Labels
area-System.Net.Http question Answer questions and provide assistance, not an issue with source code or documentation.
Milestone

Comments

@alukyan
Copy link

alukyan commented Apr 3, 2017

I have a case where I need to establish HTTPS connection to an endpoint that may hosts several virtual HTTPS endpoints which name does not match the endpoint name and I need to use SNI to select a virtual server. https://en.wikipedia.org/wiki/Server_Name_Indication

Experiments with HttpClient show that Client Hello message contains the same name as the endpoint name and I cannot find a way to change it to a different virtual server name. Any suggestion on how to do this?

@davidsh
Copy link
Contributor

davidsh commented Apr 4, 2017

Have you tried to add the "Host" request header manually which might be different from your DNS name you use in your http://servername.com uri?

@alukyan
Copy link
Author

alukyan commented Apr 4, 2017

We tried setting Headers.Host to a different value, but when looking at the protocol exchange in WireShark we still see that Client Hello message contains the same URI that is passed in request.

SNI feature is really useful to talk to HTTPS service behind a proxy to connect to secure service inside private network.
And number of languages support it:

  • Python - Connection.set_tlsext_host_name(name) - Specify the byte string to send as the server name in the client hello message.
  • Golang -
type ClientHelloInfo struct {
        // ServerName indicates the name of the server requested by the client
        // in order to support virtual hosting. ServerName is only set if the
        // client is using SNI (see
        // http://tools.ietf.org/html/rfc4366#section-3.1).
        ServerName string

This TLS extension has been accepted as a standard quite some time ago (more than 10 years already), so I am wondering why .NET still does not support it.
This missing feature makes us terminate HTTPS at the proxy and then re-encrypt the traffic again - basically inserting our own "man-in-the-middle". Not good from security standpoint.

@geoffkizer
Copy link
Contributor

I agree, we should support this.

@davidsh, are you saying that this should work? Is this just a bug?

@stephentoub FYI

@alukyan
Copy link
Author

alukyan commented Apr 4, 2017

To clarify, .NET protocol exchange contains Client Hello message, but it is always set to the same URI as the request and HttpClient does not provide means to change it to custom value.

Current SNI implementation in .NET will work only if an endpoint has multiple public DNS records (classic website example), which is not always possible to do for REST API servers, especially with clusters where microservices are running behind application load balancers. Changing value in the Host header changes just a header, but with HTTPS all traffic, including headers is encrypted so this cannot be used for virtual host selection on a proxy unless HTTPS is terminated there.

Looking at implementation in Golang - they do not link the value set in Host header to the value set in Client Hello message, it can be configured independently

@davidsh
Copy link
Contributor

davidsh commented Apr 4, 2017

There are no HttpClient nor HttpWebRequest APIs to specify an SNI name that is different from the DNS name. This is no support in either .NET Framework nor .NET Core for this in the HTTP stacks. This would API support as well as plumbing down to the transport layers (TLS/SSL) of the stacks in both Windows and *Nix.

And for SslStream class, we do not have SNI support either (although we have an active issue with requests for that feature).

@geoffkizer
Copy link
Contributor

@alukyan, thanks for explaining in more detail.

@geoffkizer
Copy link
Contributor

I do wonder whether we should be checking for a Host header and simply using this in the SNI message. It seems like this would make the common scenario work. I'm trying to imagine a scenario where you want the SNI hostname to be different than the Host header, and I'm not coming up with anything.

@Marcus-L
Copy link

Marcus-L commented Sep 28, 2017

This is needed for verifying domain names using TLS-SNI with Let's Encrypt (a free SSL certificate service). Per the draft (https://tools.ietf.org/html/draft-ietf-acme-acme-01#section-7.3), SNI entries with invalid (but known) hostnames are created on a web server, which are verified by a third party server.

Example, I want to verify that I own domain marcuslum.com, so:

  1. I perform the steps to get a key from Lets Encrypt, then use this key to derive a dns name (i.e. "1234.5678.acme.invalid")
  2. I create a self signed certificate with this domain as subject & subject alternate name
  3. I install this certificate on my webserver and configure SNI so that the verifier can make a request to my server (via the IP address of marcuslum.com), but with SNI set to "1234.5678.acme.invalid"

As far as I can tell the only ways to do make the HTTPS call via HttpClient or HttpWebRequest in .NET are to:

  1. temporarily write to the local hosts file (or to my dns server) to modify lookups for 1234.5678.acme.invalid to resolve to the same IP as marcuslum.com - requires local admin
  2. hook into wsock32.dll's gethostbyname function as described here - not sure if this works, I didn't test it
  3. use ServicePoint.BindIPEndPointDelegate with ServicePointManager as described here - but I think this only works if the SNI host has a valid DNS host entry

Marcus-L referenced this issue in Marcus-L/Certify Sep 28, 2017
Uses hosts file to enable SNI checking in CheckSNI (see https://github.com/dotnet/corefx/issues/17852#issuecomment-332939829)
Add logging to CheckURL, to return debug info to the user in case of failure
webprofusion-chrisc referenced this issue in webprofusion/certify Oct 2, 2017
* Add Challenge Type to UI

* minor fixes and refactoring

* implement tls-sni-01 challenge

* reset ServerCertificateValidationCallback after use

* Fix CheckSNI, Update CheckURL

Uses hosts file to enable SNI checking in CheckSNI (see https://github.com/dotnet/corefx/issues/17852#issuecomment-332939829)
Add logging to CheckURL, to return debug info to the user in case of failure

* Refactor Challenge Response Testing

Pulls out the testing of challenge responses from the real request flow to allow response simulation.

Implements simulated tests for tls-sni-01 and http-01

* Add Simulated Challenge Response Testing to UI

Allows the user to test simulated challenge response validation without having to make a real ACME challenge.

Useful for testing configuration/permissiions without "burning" real request limit quota.

* Change ACMESharp submodule to track tls-sni branch until PR is accepted
@davidsh
Copy link
Contributor

davidsh commented Apr 8, 2018

We tried setting Headers.Host to a different value, but when looking at the protocol exchange in WireShark we still see that Client Hello message contains the same URI that is passed in request.

Now that we use a new HTTP stack (SocketsHttpHandler) as the default for .NET Core 2.1, SNI is now working. By default, the SNI will be the DNS name of the Uri used in the request. If a custom, "Host:" request header is added, then the SNI will use that name in the TLS/SSL handshake.

If there are further questions regarding this, please open up a new issue. Thx.

cc: @karelz

@davidsh davidsh closed this as completed Apr 8, 2018
@msftgits msftgits transferred this issue from dotnet/corefx Jan 31, 2020
@msftgits msftgits added this to the 2.1.0 milestone Jan 31, 2020
@dotnet dotnet locked as resolved and limited conversation to collaborators Dec 24, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
area-System.Net.Http question Answer questions and provide assistance, not an issue with source code or documentation.
Projects
None yet
Development

No branches or pull requests

5 participants