Add support to HttpClient for mutual authentication

[mconnew]

On the desktop, WCF uses mutual authentication when using Kerberos authentication. We specify the servers SPN by adding the relevant SPN to AuthenticationManager.CustomTargetNameDictionary. We need a way to:

1. Enable mutual authentication
2. Override the SPN we expect the server to be using

The second item is needed if mutual auth is enabled as on the service side only a system process is able to use the host/hostname SPN. When the remote service is using HTTP.SYS in a non-system process and not using Kernal mode authentication, a unique SPN must be created and used. If mutual auth is enabled, this would break without being able to specify the SPN for the server on the client side.

[davidsh]

The Windows version of HttpClient relies on native WinHTTP. So, we would need that native component enhanced as well.

For WinHTTP:
https://msdn.microsoft.com/en-us/library/windows/desktop/aa384066(v=vs.85).aspx

We can get the SPN used. But there doesn't seem to be a way to set it to a custom value before the request is processed:

WINHTTP_OPTION_SERVER_SPN_USED
Gets the server Server Principal Name that WinHTTP supplied to SSPI during authentication. This string value can be passed toSspiPromptForCredentials after an authentication failure.

[mconnew]

What about for other platforms? E.g. Unix, UWP?

[zhenlan]

This is blocking WCF Windows authentication SPN scenario (dotnet/wcf#1252) and UPN scenario (dotnet/wcf#10).

[CIPop]

@davidsh, @karelz Now that we do have AuthenticationManager we may want to review the original issue as a bug for netstandard2.0.

[karelz]

The functionality is not supported by the underlying WinHttp stack in Winwdows. Probably not even in Curl. Unless we have full managed networking stack, we won't be able to reasonably implement it across the board.
Closing.

@zhenlan @mconnew you marked this as blocking, can you please check if it is still blocking and if there are any workarounds for you.

[mconnew]

The short answer is this is blocking, there aren't workarounds, and this is doable on all platforms except perhaps UWP.

We are consuming HttpClient and not WinHTTP or libCurl. These are implementation details of HttpClient and are an abstraction for a very good reason. I shouldn't need to worry about how it's implemented. If the choice of implementation isn't able to achieve functional parity with the full framework, then maybe the implementation needs to be modified/replaced.

I also spent about 30 seconds with Bing and found the documentation on how to do this with libCurl. You set the CURLOPT_SERVICE_NAME option to use a non default SPN. So it should be relatively simple to implement this on non-windows platforms. I know it can work on Windows as the full framework can do it. As it's possible to do on Windows and Unix, I see no reason that this should be closed.

This is still blocking. Here are two important scenarios that are blocked by this. This all presumes using Kerberos authentication.

Communicating with a self hosted WCF service
On the full framework, only processes running as SYSTEM or NETWORK SERVICE can use the default SPN. When a WCF service is self hosted, it's not usually run as either of these processes for good reason. In this case, a non-default principal name is needed to be used. WCF defaults to using the UPN of the user that the process is running as, but a new SPN can be created in the domain and the machine/user given permission to use that SPN and then WCF can be explicitly configured to use that SPN. On the client side, WCF would be configured with an explicit UPN or SPN. If the principal name the server is using doesn't match with the principal name that the client uses when requesting an authentication token from the token provider, the authentication fails. The only workaround is to modify the server to run as either SYSTEM or NETWORK SERVICE and update all existing clients to use the default SPN. This will involve an unavoidable outage as you need to coordinate updating all the clients and server together. It would also have security implications as you might now be running the service with a higher privilege than desired, or have permission problems accessing resources such as network shares as you are no longer running with the same user account. Requiring customers to run their service with a system account isn't a workaround, it's a dangerous kludge.

Multiple hosts serving the same service
This can be done multiple ways, one of which is with multiple DNS CNAME records to round robin multiple servers. In this case, each server will have a different hostname so a different default SPN. You would create an SPN specifically for your service and configure the client and all servers to use the same SPN. There isn't a work around that I'm aware of for this scenario.

[karelz]

We are consuming HttpClient and not WinHTTP or libCurl

Understood. Sadly, we are limited by current implementation(s). If we can fix it on Linux/Mac with libcurl, we can do that (thanks for pointer) - @Priya91 can you please check it out?

Long-term we should definitely review our strategy which networking stack to use - OS dependent (today), uniform (e.g. libcurl on all OSs), or our own (managed)? It is something we are actively discussing - incl. what is the mid-term vs. long-term solution. Performance plays a key role here as well.

Scoping the bug as Linux-only to track the libcurl tactical fix.

[mconnew]

I made a suggestion here about adding the ability to specify libcurl options via HttpClientHandler.Properties which are transparently applied to libcurl. While it might not be the right solution there, the concept might be worth considering for this issue. It wouldn't be locking in a new API to solve this issue on Linux which means more time can be put into how a more general solution could be made and I believe would be generally helpful to solve other problems which might come up.

[Priya91]

We don't have a way to specify libcurl options currently, and this would require a larger discussion on how we expose the libcurl options. Since this facility is not available on windows, exposing a new API will be specific to Unix platforms. We are not adding new APIs that are very specific to Unix for 2.0, hence pushing it out to Future.

cc @karelz

[mconnew]

As there isn't a plan to implement this on Linux, I'm changing this bug back to all platforms as this is still an issue on all platforms.

[mconnew]

@davidsh, why do you consider this no longer blocking WCF? Do you have a solution that we can use now?

[karelz]

I think we should close this issue as Won't Fix.
Realistically, the only thing which could help the scenario is to include the feature in ManagedHandler (in that case it is a feature, not bug) or someone taking and maintaining the old .NET Framework Networking stack and use it in WCF (not something we plan to do).

Adding 'blocking-partner' label is not going to help prioritize this issue in any way on our backlog. Filing (a new) feature request on ManagedHandler is a reasonable thing to do.
Happy to chat more offline if it helps. Just trying to be practical here ...

[rmkerr]

Closing as this is tracked by dotnet/corefx#27745.