Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Managed implementation of NTLM for Android and tvOS #66879

Merged
merged 18 commits into from Apr 19, 2022

Conversation

filipnavara
Copy link
Member

@filipnavara filipnavara commented Mar 19, 2022

Contributes to #62264

This is based on @wfurt's managed NTLM code and extended to cover couple more scenarios (SPNs, channel bindings, message integrity code). It integrates it into HTTP authentication for SocketsHttpHandler for Android and tvOS platforms. Notably, it does not integrate into SMTP client and NegotiateStream implementations but it would be rather trivial to add after the basic code is merged.

TODO:

  • "If NTLM v2 authentication is used and the CHALLENGE_MESSAGE does not contain both MsvAvNbComputerName and MsvAvNbDomainName AVPairs and either Integrity is TRUE or Confidentiality is TRUE, then return STATUS_LOGON_FAILURE ([MS-ERREF] section 2.3.1)."
  • Remove test connecting to private server and decommission the server
  • Verify thrown exceptions and move any necessary strings into resources

@ghost ghost added the community-contribution Indicates that the PR has been added by a community member label Mar 19, 2022
@dotnet-issue-labeler
Copy link

I couldn't figure out the best area label to add to this PR. If you have write-permissions please help me learn by adding exactly one area label.

@ghost
Copy link

ghost commented Mar 19, 2022

Tagging subscribers to this area: @dotnet/ncl, @vcsjones
See info in area-owners.md if you want to be subscribed.

Issue Details

Contributes to #62264

This is based on @wfurt's managed NTLM code and extended to cover couple more scenarios (SPNs, channel bindings, message integrity code). It integrates it into HTTP authentication for SocketsHttpHandler for Android and tvOS platforms. Notably, it does not integrate into SMTP client and NegotiateStream implementations but it would be rather trivial to add after the basic code is merged.

TODO:

  • "If NTLM v2 authentication is used and the CHALLENGE_MESSAGE does not contain both MsvAvNbComputerName and MsvAvNbDomainName AVPairs and either Integrity is TRUE or Confidentiality is TRUE, then return STATUS_LOGON_FAILURE ([MS-ERREF] section 2.3.1)."
  • Remove tast connecting to private server and decommission the server
  • Verify thrown exceptions and move any necessary strings into resources
Author: filipnavara
Assignees: -
Labels:

area-System.Net.Security, community-contribution

Milestone: -

}
}

public void Transform(ReadOnlySpan<byte> input, Span<byte> output)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks to me like Transform is only ever called once on any given instance. Rather than an instantiable class, this should just be

internal static class RC4
{
    internal static void Transform(ReadOnlySpan<byte> key, ReadOnlySpan<byte> input, Span<byte> output)
    {
        Span<byte> state = stackalloc byte[256];

        ...
    }
}

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's the case for the MIC calculation but it would be reused for Encrypt/Decrypt if we implement it later (for SMTP authentication and NegotiateStream).

// as NTLM has only one challenege message.
if (!NtlmOid.Equals(mech))
{
throw new Win32Exception(NTE_FAIL, $"'{mech}' mechanism is not supported");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exception messages should be in resx (and thus use format placeholders instead of interpolation)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, I have this one in the TODO in the initial comment.

Copy link
Member

@wfurt wfurt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

generally LGTM.
Thanks @filipnavara.
This looks like great start.

@wfurt
Copy link
Member

wfurt commented Mar 25, 2022

enterprise tests failures are tracked by #66970

@filipnavara
Copy link
Member Author

enterprise tests failures are tracked by #66970

Some of the failures are/were from the temporary test I added. I'll remove it on next pass.

@filipnavara filipnavara marked this pull request as ready for review March 26, 2022 09:40
@steveisok
Copy link
Member

/azp run runtime-extra-platforms

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@steveisok
Copy link
Member

steveisok commented Mar 31, 2022

On tvOS devices, it's crashing pretty early. Quite likely not related to this change directly, but we do want to enable System.Net.Security as a result.

Full log

[19:31:28.3617780] 	0x1007ef4ec - /private/var/containers/Bundle/Application/E11371CD-4EC3-43F1-8264-E7216646D880/System.Net.Security.Tests.app/System.Net.Security.Tests : mono_dump_native_crash_info
[19:31:28.3737980] 	0x1007d74c0 - /private/var/containers/Bundle/Application/E11371CD-4EC3-43F1-8264-E7216646D880/System.Net.Security.Tests.app/System.Net.Security.Tests : mono_handle_native_crash
[19:31:28.3810150] 	0x105006de8 - /private/var/containers/Bundle/Application/E11371CD-4EC3-43F1-8264-E7216646D880/System.Net.Security.Tests.app/System.Net.Security.Tests : sigabrt_signal_handler.cold.1
[19:31:28.3863620] 	0x1007eed00 - /private/var/containers/Bundle/Application/E11371CD-4EC3-43F1-8264-E7216646D880/System.Net.Security.Tests.app/System.Net.Security.Tests : mono_runtime_setup_stat_profiler
[19:31:28.3864000] 	0x1972ee050 - /usr/lib/system/libsystem_platform.dylib : <redacted>
[19:31:28.3864090] 	0x1972f2c30 - /usr/lib/system/libsystem_pthread.dylib : pthread_kill
[19:31:28.3864200] 	0x1971da8ac - /usr/lib/system/libsystem_c.dylib : abort
[19:31:28.3928080] 	0x1006b9934 - /private/var/containers/Bundle/Application/E11371CD-4EC3-43F1-8264-E7216646D880/System.Net.Security.Tests.app/System.Net.Security.Tests : ves_icall_System_Environment_get_TickCount
[19:31:28.3978620] 	0x1006bc080 - /private/var/containers/Bundle/Application/E11371CD-4EC3-43F1-8264-E7216646D880/System.Net.Security.Tests.app/System.Net.Security.Tests : ves_icall_System_Environment_GetCommandLineArgs_raw
[19:31:28.4049240] 	0x100f53d98 - /private/var/containers/Bundle/Application/E11371CD-4EC3-43F1-8264-E7216646D880/System.Net.Security.Tests.app/System.Net.Security.Tests : wrapper_managed_to_native_System_Environment_FailFast_string_System_Exception_string
[19:31:28.4126650] 	0x1034a7054 - /private/var/containers/Bundle/Application/E11371CD-4EC3-43F1-8264-E7216646D880/System.Net.Security.Tests.app/System.Net.Security.Tests : corlib_System_Diagnostics_DebugProvider_FailCore_string_string_string_string
[19:31:28.4202150] 	0x1020c6cc0 - /private/var/containers/Bundle/Application/E11371CD-4EC3-43F1-8264-E7216646D880/System.Net.Security.Tests.app/System.Net.Security.Tests : System_Diagnostics_TraceSource_System_Diagnostics_Trace_Fail_string
[19:31:28.4271830] 	0x1011204d8 - /private/var/containers/Bundle/Application/E11371CD-4EC3-43F1-8264-E7216646D880/System.Net.Security.Tests.app/System.Net.Security.Tests : wrapper_runtime_invoke_object_runtime_invoke_dynamic_intptr_intptr_intptr_intptr
[19:31:28.4325310] 	0x1007b7d44 - /private/var/containers/Bundle/Application/E11371CD-4EC3-43F1-8264-E7216646D880/System.Net.Security.Tests.app/System.Net.Security.Tests : mono_jit_runtime_invoke
[19:31:28.4379350] 	0x1006f2548 - /private/var/containers/Bundle/Application/E11371CD-4EC3-43F1-8264-E7216646D880/System.Net.Security.Tests.app/System.Net.Security.Tests : mono_runtime_try_invoke
[19:31:28.4431650] 	0x1006f1068 - /private/var/containers/Bundle/Application/E11371CD-4EC3-43F1-8264-E7216646D880/System.Net.Security.Tests.app/System.Net.Security.Tests : mono_runtime_class_init_full
[19:31:28.4486550] 	0x1007c96e0 - /private/var/containers/Bundle/Application/E11371CD-4EC3-43F1-8264-E7216646D880/System.Net.Security.Tests.app/System.Net.Security.Tests : init_method
[19:31:28.4537370] 	0x1007e94a4 - /private/var/containers/Bundle/Application/E11371CD-4EC3-43F1-8264-E7216646D880/System.Net.Security.Tests.app/System.Net.Security.Tests : mini_llvm_init_method
[19:31:28.4610160] 	0x102d1d6c0 - /private/var/containers/Bundle/Application/E11371CD-4EC3-43F1-8264-E7216646D880/System.Net.Security.Tests.app/System.Net.Security.Tests : mono_aot_System_Net_Security_Tests_init_method
[19:31:28.4661730] 	0x1007b7d44 - /private/var/containers/Bundle/Application/E11371CD-4EC3-43F1-8264-E7216646D880/System.Net.Security.Tests.app/System.Net.Security.Tests : mono_jit_runtime_invoke
[19:31:28.4714710] 	0x1006f2548 - /private/var/containers/Bundle/Application/E11371CD-4EC3-43F1-8264-E7216646D880/System.Net.Security.Tests.app/System.Net.Security.Tests : mono_runtime_try_invoke
[19:31:28.4767890] 	0x1006f1068 - /private/var/containers/Bundle/Application/E11371CD-4EC3-43F1-8264-E7216646D880/System.Net.Security.Tests.app/System.Net.Security.Tests : mono_runtime_class_init_full
[19:31:28.4819310] 	0x1007b366c - /private/var/containers/Bundle/Application/E11371CD-4EC3-43F1-8264-E7216646D880/System.Net.Security.Tests.app/System.Net.Security.Tests : jit_compile_method_with_opt
[19:31:28.4871640] 	0x1007b7a28 - /private/var/containers/Bundle/Application/E11371CD-4EC3-43F1-8264-E7216646D880/System.Net.Security.Tests.app/System.Net.Security.Tests : mono_jit_runtime_invoke
[19:31:28.4923120] 	0x1006f0bac - /private/var/containers/Bundle/Application/E11371CD-4EC3-43F1-8264-E7216646D880/System.Net.Security.Tests.app/System.Net.Security.Tests : mono_runtime_invoke_checked
[19:31:28.4975990] 	0x1006f907c - /private/var/containers/Bundle/Application/E11371CD-4EC3-43F1-8264-E7216646D880/System.Net.Security.Tests.app/System.Net.Security.Tests : mono_runtime_try_invoke_span
[19:31:28.5031350] 	0x1006b3efc - /private/var/containers/Bundle/Application/E11371CD-4EC3-43F1-8264-E7216646D880/System.Net.Security.Tests.app/System.Net.Security.Tests : ves_icall_InternalInvoke
[19:31:28.5080910] 	0x1006be2cc - /private/var/containers/Bundle/Application/E11371CD-4EC3-43F1-8264-E7216646D880/System.Net.Security.Tests.app/System.Net.Security.Tests : ves_icall_InternalInvoke_raw
[19:31:28.5140210] 	0x100f7a1b0 - /private/var/containers/Bundle/Application/E11371CD-4EC3-43F1-8264-E7216646D880/System.Net.Security.Tests.app/System.Net.Security.Tests : wrapper_managed_to_native_System_Reflection_RuntimeMethodInfo_InternalInvoke_System_Reflection_RuntimeMethodInfo_object_System_Span_1_object__System_Exception_
[19:31:28.5194270] 	0x100f7a2c4 - /private/var/containers/Bundle/Application/E11371CD-4EC3-43F1-8264-E7216646D880/System.Net.Security.Tests.app/System.Net.Security.Tests : System_Reflection_RuntimeMethodInfo_InvokeWorker_object_System_Reflection_BindingFlags_System_Span_1_object
[19:31:28.5263610] 	0x103407da8 - /private/var/containers/Bundle/Application/E11371CD-4EC3-43F1-8264-E7216646D880/System.Net.Security.Tests.app/System.Net.Security.Tests : corlib_System_Reflection_RuntimeMethodInfo_Invoke_object_System_Reflection_BindingFlags_System_Reflection_Binder_object___System_Globalization_CultureInfo
[19:31:28.5336300] 	0x104ec3fa0 - /private/var/containers/Bundle/Application/E11371CD-4EC3-43F1-8264-E7216646D880/System.Net.Security.Tests.app/System.Net.Security.Tests : xunit_execution_dotnet_Xunit_Sdk_XunitWorkerThread__c__QueueUserWorkItemb__5_0_object
[19:31:28.5409930] 	0x10334f8a8 - /private/var/containers/Bundle/Application/E11371CD-4EC3-43F1-8264-E7216646D880/System.Net.Security.Tests.app/System.Net.Security.Tests : corlib_System_Threading_Tasks_Task_ExecuteEntryUnsafe_System_Threading_Thread
[19:31:28.5460300] 	0x1007b7d44 - /private/var/containers/Bundle/Application/E11371CD-4EC3-43F1-8264-E7216646D880/System.Net.Security.Tests.app/System.Net.Security.Tests : mono_jit_runtime_invoke
[19:31:28.5509570] 	0x1006f0bac - /private/var/containers/Bundle/Application/E11371CD-4EC3-43F1-8264-E7216646D880/System.Net.Security.Tests.app/System.Net.Security.Tests : mono_runtime_invoke_checked
[19:31:28.5566090] 	0x10070a110 - /private/var/containers/Bundle/Application/E11371CD-4EC3-43F1-8264-E7216646D880/System.Net.Security.Tests.app/System.Net.Security.Tests : start_wrapper_internal
[19:31:28.5618520] 	0x100709ea0 - /private/var/containers/Bundle/Application/E11371CD-4EC3-43F1-8264-E7216646D880/System.Net.Security.Tests.app/System.Net.Security.Tests : start_wrapper

@wfurt
Copy link
Member

wfurt commented Apr 12, 2022

Should we disable tvOS for now and focus on Android? It seems like it is failing as well but from the logs I cannot see what is wrong.

@filipnavara
Copy link
Member Author

filipnavara commented Apr 13, 2022

The System.Net.Security.Tests crashes are completely unrelated to these changes. In fact, the only relevant test in that batch is the MD4 one for the managed implementation. I did NOT enable the implementation of NTAuthentication in System.Net.Security, which is only used for NegotiateStream. This PR really only affects System.Net.Http and so we should enable the relevant tests there.

@wfurt
Copy link
Member

wfurt commented Apr 19, 2022

Tests may be #68206

Copy link
Member

@wfurt wfurt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@wfurt wfurt merged commit a6508a0 into dotnet:main Apr 19, 2022
directhex pushed a commit to directhex/runtime that referenced this pull request Apr 21, 2022
* Move MD4 implementation into Common/src/System/Net/Security

* Add minimal RC4 implementation

* WIP: Integrate managed Ntlm implementation into SocketsHttpHandler

Co-authored-by: Tomas Weinfurt <tweinfurt@yahoo.com>

* WIP: Makeshift tests for NTLM/Negotiate authentication

* Fix compilation, clean up some of the hashing

* Avoid using a temporary buffer

* Add computation of signing keys, sealing keys and mechListMIC

* Various cleanups

* Send SPN in target information

* Add some validation, mark spots with missing validation

* Clean up some of the memory offset manipulation

* Move NTLM version into static variable

* Add support for channel bindings, clean up

* Fix hash calculation in makeNtlm2Hash accidentally broken with last commit.
Read NegTokenResp explicitly.
Add mechListMIC reading and verification.

* Verify last authentication token in HTTP Negotiate authentication

* Address feedback

* Fix tvOS builds by making few methods static

* Enable System.Net.Security tests on Android and iOS

Co-authored-by: Tomas Weinfurt <tweinfurt@yahoo.com>
Co-authored-by: Steve Pfister <steve.pfister@microsoft.com>
@EatonZ
Copy link
Contributor

EatonZ commented May 4, 2022

@filipnavara Thank you for adding a native RC4 implementation to .NET. Would you consider making the class public so it can be used? I have some old code that uses RC4 (not related to NTLM for Android and tvOS) and it would be nice to be able to call into .NET directly.

@filipnavara
Copy link
Member Author

@EatonZ Every new API would have to go through API review process. RC4 is unlikely to pass through that process because the algorithm is insecure and deprecated. Feel free to copy the implementation under the current MIT license though, it is really small.

@EatonZ
Copy link
Contributor

EatonZ commented May 4, 2022

@filipnavara Thanks for the fast response. That is what I was going to do anyway, but thought I'd ask.
Definitely insecure, but it's just how these old files are processed.🙃

@dotnet dotnet locked as resolved and limited conversation to collaborators Jun 3, 2022
@karelz karelz added this to the 7.0.0 milestone Jul 19, 2022
@bartonjs bartonjs added needs-further-triage Issue has been initially triaged, but needs deeper consideration or reconsideration cryptographic-docs-impact labels Aug 26, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
area-System.Net.Security community-contribution Indicates that the PR has been added by a community member needs-further-triage Issue has been initially triaged, but needs deeper consideration or reconsideration
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants