Fix NTLM authentication from macOS to Windows machines (#54101)

* Fix NTLM authentication from macOS to Windows machines

The GSSAPI implementation on macOS has partially broken NTLM implementation. It only supports NTLMv2 with the message integrity code (MIC) as specified by the MS-NLMP specification. The MIC is calculated using HMAC-MD5 authentication code over the exchanged NTLM messages with a key named ExportedSessionKey. The proper generation of ExportedSessionKey requires the implementation to negotiate correct capabilities, namely NTLMSSP_NEGOTIATE_KEY_EXCH and at least one of NTLMSSP_NEGOTIATE_SIGN or NTLMSSP_NEGOTIATE_SEAL flags. By default the macOS implementation negotiates NTLMSSP_NEGOTIATE_KEY_EXCH and sends MIC but fails to set one of the additional flags that would make the key exchange valid. This results in violation of the following part of the NTLM specification:

"A session key MUST always exist to generate the MIC (section 3.1.5.1.2) in the authenticate message. NTLMSSP_NEGOTIATE_ALWAYS_SIGN MUST be set in the NEGOTIATE_MESSAGE to the server and the CHALLENGE_MESSAGE to the client."

Adding the GSS_C_INTEG_FLAG flag forces macOS to properly negitiate all the necessary flags (NTLMSSP_NEGOTIATE_ALWAYS_SIGN and NTLMSSP_NEGOTIATE_SIGN) to make the MIC exchange valid. This in turn enables the whole NTLM exchange to be recognized as valid by Windows server side.

The gss-ntlmssp package on Linux interprets the GSS_C_INTEG_FLAG flag as additional negotiation of NTLMSSP_NEGOTIATE_SIGN and NTLMSSP_NEGOTIATE_KEY_EXCH. That should not hurt anything and in fact it may improve security depending on specific configuration. The flag was already specified when NTLM was used by System.Net.Mail.SmtpClient.

* Add temporary test to verify NTLM and Negotiate connections from all platforms

Notably this is expected to fail on Android and tvOS where GSSAPI is not available.

The test machine is single purpose virtual machine running in Azure with clean Windows Server 2019 installation and all the patches. It hosts IIS with single static page behind an NT authentication. The test account and the machine itself is disposable and does NOT contain any sensitive data. Nevertheless I don't expect this test to be merged.

* Use full domain name in test credentials

* Revert "Use full domain name in test credentials"

This reverts commit ea4dc61.

* Revert "Add temporary test to verify NTLM and Negotiate connections from all platforms"

This reverts commit abf44ce.

src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/AuthenticationHelper.NtAuth.cs

@@ -119,7 +119,7 @@ private static async Task<HttpResponseMessage> SendWithNtAuthAsync(HttpRequestMe
                         }
 
                         ChannelBinding? channelBinding = connection.TransportContext?.GetChannelBinding(ChannelBindingKind.Endpoint);
-                        NTAuthentication authContext = new NTAuthentication(isServer: false, challenge.SchemeName, challenge.Credential, spn, ContextFlagsPal.Connection, channelBinding);
+                        NTAuthentication authContext = new NTAuthentication(isServer: false, challenge.SchemeName, challenge.Credential, spn, ContextFlagsPal.Connection | ContextFlagsPal.InitIntegrity, channelBinding);
                         string? challengeData = challenge.ChallengeData;
                         try
                         {