Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OpenSSH requires be run as service to work - fork of unprivileged child failed #1304

Open
illera88 opened this issue Dec 10, 2018 · 22 comments

Comments

@illera88
Copy link

commented Dec 10, 2018

Please answer the following

"OpenSSH for Windows" version
OpenSSH_for_Windows_7.7p1, LibreSSL 2.6.4

Server OperatingSystem
Windows 10 Enterprise

Client OperatingSystem
Windows 10 Enterprise

What is failing
I cannot get a shell after I successfully log in.

I'm debugging within VS libsshd to understand how it works. I got it listening to a port (not 22 since that requires admin privileges) and the problem is that when I introduce my user and password I automatically get disconnected. In the log I can see:

Server listening on :: port 2233.
Server listening on 0.0.0.0 port 2233.
Could not load host key: __PROGRAMDATA__\\ssh/ssh_host_ecdsa_key
Could not load host key: __PROGRAMDATA__\\ssh/ssh_host_ed25519_key
Failed password for user from ::1 port 3123 ssh2
Accepted password for user from ::1 port 3123 ssh2
CreateProcessAsUserW failed error:1314
fork of unprivileged child failed

I've tried with my built sshd and with the release one and I get the same thing.

Btw: how can I get full debug messages to get print out to stdout?

@illera88 illera88 changed the title Can't fully conn Can't fully connect to sshd- fork of unprivileged child failed Dec 10, 2018
@NoMoreFood

This comment has been minimized.

Copy link

commented Dec 11, 2018

Debugging When Running As A Service

So here's what I do for debugging as a service. Running as a service can result in different behavior compared to running sshd as an interactive user.

  1. Install SSHD using the normal instructions.
  2. Set the following in C:\ProgramData\ssh\sshd_config to enable verbose logging:
    SyslogFacility LOCAL0
    LogLevel DEBUG3
    
  3. Install Microsoft Child Process Debugging Power Tool for Visual Studio. This allows the debugger to pickup any child processes.
  4. Launch Visual Studio as Administrator.
  5. Activate the aforementioned plugin by going to Debug ➡️ Other Debug Targets ➡️ Child Process Debugging Settings ➡️ Check 'Enable child process debugging' ➡️ Click 'Save'.
  6. Replace the executable(s) in 'C:\Program Files\OpenSSH' with your development executables (or make symbolic links for the relevant executables that point to your Debug directory:
    C:\Program Files\OpenSSH>del sshd.exe
    
    C:\Program Files\OpenSSH>MKLINK sshd.exe C:\openssh-portable\bin\x64\Debug\sshd.exe
    symbolic link created for sshd.exe <<===>> C:\openssh-portable\bin\x64\Debug\sshd.exe
    
  7. Start the 'OpenSSH SSH Server' service.
  8. Startup PowerShell as Administrator and tail the sshd log file. This will display messages as they are added to the log file:
    PS C:\> Get-Content -Path C:\ProgramData\ssh\Logs\sshd.log -Wait -Tail 0
    
  9. Attach to the sshd.exe process using Debug ➡️ Attach to Process....
  10. Place breakpoints where desired.
  11. Login using ssh (or another SSH client).
@illera88

This comment has been minimized.

Copy link
Author

commented Dec 11, 2018

Those are great instructions!

Trying right now. I'll come back to the results to see if we can find the root cause

@illera88

This comment has been minimized.

Copy link
Author

commented Dec 11, 2018

I did what you said and it seems that there is no log files created at C:\ProgramData\ssh\Logs\sshd.log any reason why that could be?

thanks

@NoMoreFood

This comment has been minimized.

Copy link

commented Dec 11, 2018

Hmmm... that should be it. Did you start sshd as service? I’ll check my sshd_config against the default when I get back to my computer. I suppose step 2 should technically be after the service starts to get the file in place.

@illera88

This comment has been minimized.

Copy link
Author

commented Dec 11, 2018

No, I did not start it as a service since I want to start it from within the debugger (instead of attaching to it) and I want to create a ssh server that does not require privileges (running on port > 1000)

@illera88

This comment has been minimized.

Copy link
Author

commented Dec 11, 2018

I think I got close from where the bug happens:
It is at spawn_child_internal(char* cmd, char *const argv[], HANDLE in, HANDLE out, HANDLE err, unsigned long flags, HANDLE* as_user, BOOLEAN prepend_module_path) function and the cmd argument instead of being cmd.exe or powershell.exe it is sshd.exe and argv is -z.

Don't understand really why that is happening.

For clarification this happens after successful authentication (that's why I think the process that should be executed there is cmd.exe or powershell.exe .

Ideas?

@NoMoreFood

This comment has been minimized.

Copy link

commented Dec 11, 2018

Ok, my bad. My write up is for debugging as a service. I’ll update them for running as as a regular user in a bit - much of the same should apply. I usually run as a service.

@illera88

This comment has been minimized.

Copy link
Author

commented Dec 11, 2018

I've kept digging and I see that in load_user_profile checks if I'm SYSTEM (have run the process as a service):

int 
load_user_profile(HANDLE user_token, char* user)
{
	wchar_t * user_utf16 = NULL;

	if (!am_system()) {
	    debug("Not running as SYSTEM: skipping loading user profile");
	    return 0;
	}

As I said I don't and I need it to run in user land. Do you think that could be causing the problem?

@illera88

This comment has been minimized.

Copy link
Author

commented Dec 11, 2018

Indeed I tried to run sshd as a service and:

  1. If I use the release binaries provided in this repo it does works perfect.
  2. If I use the debug binaries I did create with VS it does not (I guess this needs further debugging)

In any case it seems pretty obvious that sshd cannot be used if it is not run as a service and it may be related with load_user_profile not passing the if (!am_system()) { condition.

Is the requirement of running sshd as a system a known issue?

Thank you

@NoMoreFood

This comment has been minimized.

Copy link

commented Dec 11, 2018

I know your intent is to not run as a system user, but is it also your intent to run an unprivileged user?

@illera88

This comment has been minimized.

Copy link
Author

commented Dec 11, 2018

@NoMoreFood

This comment has been minimized.

Copy link

commented Dec 11, 2018

Alright, I'm not familiar with this use case and have not tested it myself. I'll give it a test today. I do know that public key authentication (into a different user account) is not going to work because it relies on privileges that a normal user does not have. Password-based authentication can technically work at the OS level assuming Secondary Logon is enabled (it should be by default).

@NoMoreFood

This comment has been minimized.

Copy link

commented Dec 11, 2018

So I did a quick debug from and the problem is indeed to the fact that CreateProcessAsUserW() failed, as indicated by your session output. This is likely for two reasons: 1) The token returned from LogonUserExExW() probably should be a primary token and will never be since LOGON32_LOGON_NETWORK_CLEARTEXT is being used as the logon type, and 2) when using CreateProcessAsUserW(), additional privileges are needed to assign the token to the new process (this is what the 1314 error indicates). In order to run in a truly unprivileged mode, I suspect that CreateProcessWithLogonW() would have to be used instead of the LogonUserExExW()/LogonUserExExW() combo.

I haven't familiarized enough with the functions that call these to confidently infer why it was designed this way; somebody else would have to speak to that.

@illera88

This comment has been minimized.

Copy link
Author

commented Dec 11, 2018

Thank you for your time on debugging this issue. Do you know someone that may know about the internals of the mentioned APIs so we can mention him to take a look at this?

thanks again

@illera88 illera88 changed the title Can't fully connect to sshd- fork of unprivileged child failed OpenSSH requires be run a service to work - fork of unprivileged child failed Dec 11, 2018
@illera88 illera88 changed the title OpenSSH requires be run a service to work - fork of unprivileged child failed OpenSSH requires be run as service to work - fork of unprivileged child failed Dec 11, 2018
@jborean93

This comment has been minimized.

Copy link

commented Dec 12, 2018

IIRC the trouble with CreateProcessWithLogonW() is that it creates a process with an interactive logon type whereas normally an ssh spawned logon would have a network cleartext logon type for password based auth. The other option is CreateProcessWithToken() but that need the SeImpersonatePrivilege which isn't assigned to standard accounts.

I'm curious as to why you want to run this as a non privileged account considering ssh key auth won't ever work.

@illera88

This comment has been minimized.

Copy link
Author

commented Dec 12, 2018

@jborean93

This comment has been minimized.

Copy link

commented Dec 12, 2018

It's not my place to say whether CreateProcessWithLogonW() is an alternative as this isn't my project. Just merely stating the facts as to how moving to that will result in a behaviour change where arguably an interactive logon from a network source isn't really correct. I'm sure there will be other issues that would pop up if this ever changed to support this scenario as logon and process creation is just the tip of the iceberg.

@illera88

This comment has been minimized.

Copy link
Author

commented Dec 12, 2018

@manojampalam

This comment has been minimized.

Copy link
Collaborator

commented Dec 12, 2018

@illera88
sshd needs to run as system to be able to spawn off child process in different security contexts - these include sshd nonprivileged workers and user session proesses. sshd in interactive mode is only supported for debugging purposes. Also make a note of this restriction in https://github.com/PowerShell/Win32-OpenSSH/wiki/Troubleshooting-Steps

sshd.exe -d This will run sshd in interactive mode under currently logged on user (typically as admin). Note that, in this mode, you can only login as the "currently logged on user" and only using "key based auth". To overcome these restrictions, you should instead run psexec -s sshd.exe -d to run sshd as SYSTEM.

Can you please summarize again what your requirement is?

@illera88

This comment has been minimized.

Copy link
Author

commented Dec 12, 2018

@hypersw

This comment has been minimized.

Copy link

commented May 15, 2019

I also need a simple portable SSH server which would serve remote connections under the current user for debug purposes, and preferrably have an option to work without keys because it's a hassle to set up keys every time when dropping the setup to a VM and connecting from a new client.

@n3v3rf411

This comment has been minimized.

Copy link

commented Jun 5, 2019

I am currently using sshd -d wrapped in a batch file that will forever loop the command if it exits.

This is handy replacement over WinRM for Jenkins Windows slaves since some jobs need a UI to execute.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
6 participants
You can’t perform that action at this time.