Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Debugging lsass with x64dbg #2411

Closed
dirkjanm opened this issue Jul 30, 2020 · 5 comments
Closed

Debugging lsass with x64dbg #2411

dirkjanm opened this issue Jul 30, 2020 · 5 comments
Labels
question The issue is a question.

Comments

@dirkjanm
Copy link

dirkjanm commented Jul 30, 2020

Hey,

I'm trying to understand some Windows internals and looking at logon packages and stuff, which are part of the lsass process.
I'd like to do this with x64dbg and actually have done so in the past, but I can't get it to work on recent Windows 10 builds. As you may be aware lsass is responsible for a lot of the security things taking place in Windows, so debugging it is quite tricky.
At the moment I attach to the lsass process, x64dbg freezes and stops doing whatever it normally does, at which point a hard reboot is the only option since the whole OS freezes. Likely this is because attaching to a process suspends the threads of this process, but the attach process itself also needs a functioning lsass process for whatever reason, thus causing a deadlock. I've attached x64dbg to x64dbg to figure out where the issue is taking place, and it seems to be the DebugActiveProcess call in kernel32 that hangs the whole thing.

I tried this with various builds of x64dbg ranging from a build from january to the latest snapshot (snapshot_2020-07-29_04-15) and also with various Windows 10 versions (for example 1909) and they all show the same behavior. Interesting enough I did some debugging successfully on lsass with Server 2016 last year (but not sure which build), so it did work at some point.

Is there a way to work around this? I'd love to be able to use x64dbg instead of having to go into kernel debug mode with other tools.

@mrexodia
Copy link
Member

mrexodia commented Jul 31, 2020

You could try something like https://github.com/brock7/xdbg or cheat engine's SEH debugging. I don't really know how to solve a deadlock in the kernel, so if you find anything let me know.

@dirkjanm
Copy link
Author

dirkjanm commented Aug 17, 2020

I couldn't get it working with xdbg, same result. However, I found out the way to do it:

  • Run x64dbg as SYSTEM (using for example psexec)
  • Make sure initial breakpoints are disabled (just to be sure, may work without as well)
  • Make sure debug privilege is not enabled in the settings (having it enabled will make x64dbg crash even when SYSTEM). Not sure if it actively removes the privilege after this setting is disabled, so I restarted x64dbg to be sure.

Apart from that I don't think you can fix it from x64dbg as it seems related to the kernel indeed.

@Mattiwatti
Copy link
Member

Mattiwatti commented Aug 18, 2020

I looked into this because I was highly skeptical of @dirkjanm's claim that disabling the debug privilege option somehow fixed this problem. Why is this? Because a process started as SYSTEM (at least, by psexec) will already have SE_DEBUG_PRIVILEGE enabled by default, so this makes no difference. Furthermore, it simply doesn't make sense that enabling a privilege in your token would cause something as drastic as a system-wide deadlock. However, I tested it and it is actually completely true!

Fortunately it was quite easy to find the cause (highlighted in red - ignore the top part of the stack as my symbols for this version of ntoskrnl seem to be missing):
lolsass
EngineSetDebugPrivilege, the function that enables or disables SE_DEBUG_PRIVILEGE in TitanEngine, is called to enable the privilege. It starts by using LookupPrivilegeValueW to convert the name of the privilege to a LUID. Higher up in the stack you see advapi32.dll making an ALPC RPC call to another process to look up this value. Three guesses which process is responsible for handling this call... ding ding, it's lsass.exe, which now can't respond.

I can't really blame whoever wrote this function, since this is exactly how the MSDN sample code does it, but it's a little-known secret that LookupPrivilegeValueW is a completely pointless function. All SE_XXX_PRIVILEGE values are simple numerical constants that have never been changed. Fixed in this TitanEngine PR: x64dbg/TitanEngine#6.

With the fixed TitanEngine, you can now attach to lsass.exe with the debug privilege option enabled, both as SYSTEM and as a regular administrator. However, be warned that detaching from the process kills it, which will cause Windows to display a 'reboot in 1 minute' message since lsass is a critical process. This only happens when running as administrator, not as SYSTEM. I may look into this some other time as it's really a separate issue.

@dirkjanm
Copy link
Author

dirkjanm commented Aug 18, 2020

Ha nice find! looking forward to be able to use it without jumping through hoops to run it as SYSTEM. Appreciate you looking into it :)

@Mattiwatti
Copy link
Member

Mattiwatti commented Aug 30, 2020

Created another TitanEngine PR with a tentative fix for lsass dying on detach: x64dbg/TitanEngine#7. I say tentative because I can't really explain why this fix works or why there was a difference between admin and SYSTEM users re: this. But I tested detaching multiple times with this patch and lsass no longer seems to die.

Note for @dirkjanm: the undocumented SafeAttach option must be enabled in x64dbg.ini for this change to have an effect. Example INI snippet:

[Engine]
SafeAttach=1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question The issue is a question.
Projects
None yet
Development

No branches or pull requests

3 participants