HTTP request is unauthorized with client authentication scheme 'Ntlm

[LukeTOBrien]

Describe the bug

I am developing a ASP service that is executing an SSRS report via WCF and I encountered this error when UnitTesting the WCF project in Visual Studio 2022.
These test used to work okay, whenever new versions of System.ServiceModel.XXX Nuget packages are released, I recieve the following error:

The CustomBinding on the ServiceEndpoint with contract 'ReportingService2005Soap' lacks a TransportBindingElement.  Every binding must have at least one binding element that derives from TransportBindingElement

This forces me to update the Nuget packages to the latest and resolves the issue.
However this time I have updated the Nuget packages to verson 8 and I receive a new error when I run the unit tests:

'The HTTP request is unauthorized with client authentication scheme 'Ntlm'. The authentication header received from the server was 'NTLM'.'

My code has not changed, I have only updated the Nuget packages, so I am wondering if there is a change in version 8 that would cause this issue or a new property that needs to be set?

This is my service:

private RSService.ReportingService2005SoapClient _rs = new RSService.ReportingService2005SoapClient(RSService.ReportingService2005SoapClient.EndpointConfiguration.ReportingService2005Soap);

I am setting the binding in my constructor:

            var binding = _rs.Endpoint.Binding as BasicHttpBinding;
            binding.Security.Mode = BasicHttpSecurityMode.TransportCredentialOnly;
            binding.Security.Transport.ClientCredentialType = HttpClientCredentialType.Ntlm;

And the credential just before I call a SOAP method:

rs.ClientCredentials.Windows.ClientCredential = new NetworkCredential("****", "****");

I have done searches and the issue seems similar to #3694 but not quite the same.
My .csproj:

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>net8.0</TargetFramework>
    <ImplicitUsings>enable</ImplicitUsings>
    <Nullable>enable</Nullable>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="System.ServiceModel.Duplex" Version="6.0.0" />
    <PackageReference Include="System.ServiceModel.Federation" Version="8.0.0" />
    <PackageReference Include="System.ServiceModel.Http" Version="8.0.0" />
    <PackageReference Include="System.ServiceModel.NetTcp" Version="8.0.0" />
    <PackageReference Include="System.ServiceModel.Security" Version="6.0.0" />
  </ItemGroup>

  <ItemGroup>
    <ProjectReference Include="..\EnterPro.Gateways\EnterPro.Gateways.vbproj" />
  </ItemGroup>

</Project>

Any help would be greatly appreciated.

[mconnew]

Can you confirm what version of .NET you were using when it worked before? What version of WCF packages were you using then? Have you regenerated the client using a more recent version of WCF Connected Services or dotnet-svcutil?

There's some things which don't make sense. Earlier generated versions of a client would generate a CloseAsync method, but newer versions don't as we added a CloseAsync method to ClientBase<TChannel> which is the base class of the generated client. For your app not to break when updating the versions of WCF packages you are using, you prior version had to be 6.x. We have made only minor changes between 6.2.0 and 8.0.0, and no changes in the Http implementation at all between those versions. No compilation errors suggests you only went from 6.x to 8.x, which wouldn't have seen a behavior change. If you came from 4.x, then there would be a compilation error in the generated client.

The error:

The CustomBinding on the ServiceEndpoint with contract 'ReportingService2005Soap' lacks a TransportBindingElement.  Every binding must have at least one binding element that derives from TransportBindingElement

Happens when you are creating a ChannelFactory and the CustomBinding you've provided doesn't contain a TransportBindingElement. We've never made any changes which would cause that. As the binding comes from the generated client, this could happen if it was using a CustomBinding and you edited the generated code without realizing and accidentally removed a line. But the snippet you provided casts the binding as BasicHttpBinding, which means it isn't using a CustomBinding. This exception is only possible if you created a CustomBinding and had a bug in your code which omitted the HttpTransportBindingElement from the CustomBinding. But then it doesn't make sense to provide that snippet of code.

The .csproj you provided should have failed to compile as there is no 8.0.0 version of System.ServiceModel.Security or System.ServiceModel.Duplex, they've been rolled up into Primitives (which you aren't referencing directly, but really should).

For the authentication, we take your NetworkCredential, put it inside a CredentialCache with a restriction to only be used for the auth type specific (ntlm in your case), and set that on the credentials property of HttpClientHandler.

You've got an interesting set of parameters you are setting on the binding. You've specified BasicHttpSecurityMode.TransportCredentialOnly which says you are going to use HTTP (no SSL/TLS security as not HTTPS), with NTLM credentials. This is not secure at all as NTLM is only safe to use over a secure connection. As the socket itself is authenticated with NTLM and not the single request, you are open to MITM attacks. I'm wondering if HttpClientHandler (which is backed by SocketsHttpHandler) refuses to send NTLM credentials over a non-secure connection. For this to be a potential cause, you would have needed to change the version of .NET you are using. I checked the breaking changes documentation back to .NET 6 and there was nothing mentioned. You also said the only thing you changed was the package versions you are referencing and nothing else.

There's a lot here which doesn't make sense for the only thing you changed was the WCF package versions, and the details you've presented don't make sense to even be able to compile your app. Are you sure nothing else changed?

[LukeTOBrien]

Thanks for your detailed reply.

My mistake, I did not update the Duplex and I updated System.ServiceModel.Security to version 6.
The .NET version I was using before would of been 7.
I should also say that this project is a port from the original project that targets the full .NET Framework 4.6.1.

Unfortunately our test report server is unsecure http, in production our clients use secure https and it did work (at least .NET 7) in production, so perhaps it is refusing to send over unsecure http like you say.

I updated the packages from 4.10.*:

  <ItemGroup>
    <PackageReference Include="System.ServiceModel.Duplex" Version="6.0.0" />
    <PackageReference Include="System.ServiceModel.Federation" Version="4.10.*" />
    <PackageReference Include="System.ServiceModel.Http" Version="4.10.*" />
    <PackageReference Include="System.ServiceModel.NetTcp" Version="4.10.*" />
    <PackageReference Include="System.ServiceModel.Security" Version="4.10.*" />
  </ItemGroup>

There's a lot here which doesn't make sense for the only thing you changed was the WCF package versions, and the details you've presented don't make sense to even be able to compile your app. Are you sure nothing else changed?

We have also change to a new server, but the Framework 4.6 project is still working fine.
I am inclined to believe that it is an issue with unsecure http.

[mconnew]

If it's a limitation of unsecure http, it's not a limitation that WCF is enforcing. The security mode TransportCredentialOnly is there to make you explicitly state that you understand you are using http with no encryption and still want to send credentials in the clear. The intended primary scenario from the server side is using an SSL offloading proxy server/load balancer.

As it's a test server and not production, could you try switching to Digest auth for your test environment? There's a chance it won't work because to support digest auth, the domain controller (or local machine if using a local account) needs to keep an encrypted copy of the raw password for the account. The client used to do that too when you logged in to enable authenticating without providing your password again, but that was disabled by default since Windows 8.1. But as you are using an explicit password, I think it should work as I believe the change was only restricting the client holding on to the password and not the domain controller. Anyway, it might be worth trying for a test environment.

Another option would be to use HTTPS. If you have iis express installed, it creates a development certificate you can use. It's already bound to ports 44300 to 44399.

If that doesn't work, then the next stop would be to open an issue in the dotnet/runtime repo. It would probably be best to test using HttpClient/HttpClientHandler directly to your server using NTLM auth to validate it is a problem with HTTPs. Just host a static file in a folder where the web.config is configured for windows auth and try to get it. They prefer when you report an issue which repro's using HttpClient outside of WCF.