SSH host key permissions not working

[codearoo]

Please answer the following

"OpenSSH for Windows" version
OpenSSH_7.6p1, LibreSSL 2.5.3
but the GitHub release is openssh 0.0.24.0 which I installed via Chocolatey.

Server OperatingSystem
Windows Server 2008 R2 Datacenter

What is failing
Permissions with host keys.

sshd by default using your install-sshd.ps1 script installs using some "NT SERVICE\SSHD" account which I don't actually see in my systems.. but in older installations it somehow works. However with this version this is not working. It's not able to load the files at all. So I change the permission for 1 of those files to Everyone, and it was able to read it, but then gave the following warning:
2576 15:25:27:432 error: @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
2576 15:25:27:432 error: @ WARNING: UNPROTECTED PRIVATE KEY FILE! @
2576 15:25:27:432 error: @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
2576 15:25:27:432 error: Permissions for 'C:\Program Files\OpenSSH-Win64/./ssh_host_ed25519_key' are too open.
2576 15:25:27:432 error: It is required that your private key files are NOT accessible by others.
2576 15:25:27:432 error: This private key will be ignored.
2576 15:25:27:432 error: key_load_private: bad permissions

OK fine.. so I removed the Everyone permission.
Then I thought to change the user of the SSHD service to be the Local System account.
Well, that allowed it to load the keys, but all of them gave the above warning. Result from that I think is that when trying to connect to the SSH server it immediately rejects.
So... I took one of those keys and removed all permissions of the Administrators and left only SYSTEM to have rights. Same warning... it is not explain WHAT is the bad permission or what they should be.
But I also don't understand how this "NT SERVICE\SSHD" user is used to run the service while not being able to find this account on the computer at all.

Can you guys clean this up? Not sure why for the past several months there isn't a simple install script that just works. It's all this copy and pasting, flipping back and forth between CMD and Powershell.
I've been using this OpenSSH on my Win 2008r2 servers for about a couple of years, with various versions.. and with some minor tweaking it would always work. But this time I'm stumped.

Forgot to show the error coming also from the ssh-add command:
PS C:\Program Files\OpenSSH-Win64> ssh-add ssh_host_dsa_key @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ @ WARNING: UNPROTECTED PRIVATE KEY FILE! @ @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ Permissions for 'ssh_host_dsa_key' are too open. It is required that your private key files are NOT accessible by others. This private key will be ignored. PS C:\Program Files\OpenSSH-Win64> ssh-add ssh_host_rsa_key @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ @ WARNING: UNPROTECTED PRIVATE KEY FILE! @ @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ Permissions for 'ssh_host_rsa_key' are too open. It is required that your private key files are NOT accessible by others. This private key will be ignored. PS C:\Program Files\OpenSSH-Win64> ssh-add ssh_host_ecdsa_key @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ @ WARNING: UNPROTECTED PRIVATE KEY FILE! @ @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ Permissions for 'ssh_host_ecdsa_key' are too open. It is required that your private key files are NOT accessible by others. This private key will be ignored. PS C:\Program Files\OpenSSH-Win64> ssh-add ssh_host_ed25519_key @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ @ WARNING: UNPROTECTED PRIVATE KEY FILE! @ @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ Permissions for 'ssh_host_ed25519_key' are too open. It is required that your private key files are NOT accessible by others. This private key will be ignored.

[codearoo]

Somehow I once managed to change the permissions and get something working with an older version, but then I re-tried with this latest version and I'm really confused. Look at this output below... how is it possible this private key file is accessible by others???

`C:\Program Files\OpenSSH-Win64>ssh-add ssh_host_dsa_key
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@ WARNING: UNPROTECTED PRIVATE KEY FILE! @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
Permissions for 'ssh_host_dsa_key' are too open.
It is required that your private key files are NOT accessible by others.
This private key will be ignored.

C:\Program Files\OpenSSH-Win64>icacls ssh_host_dsa_key
ssh_host_dsa_key NT AUTHORITY\SYSTEM:(R)

Successfully processed 1 files; Failed processing 0 files
`

[codearoo]

Oh!! According to your instructions, this step does not work:
powershell -ExecutionPolicy Bypass '.\FixHostFilePermissions.ps1 -Confirm:$false'

Here is the output:
C:\Program Files\OpenSSH-Win64>powershell -ExecutionPolicy Bypass '.\FixHostFile Permissions.ps1 -Confirm:$false' .\FixHostFilePermissions.ps1 -Confirm:$false

But if you first open up 'powershell' and then run it, THIS is the output, indicating it's actually doing something:
`C:\Program Files\OpenSSH-Win64>powershell
Windows PowerShell
Copyright (C) 2016 Microsoft Corporation. All rights reserved.

PS C:\Program Files\OpenSSH-Win64> .\FixHostFilePermissions.ps1 -Confirm:$false
[*] C:\Program Files\OpenSSH-Win64\sshd_config
looks good

[*] C:\Program Files\OpenSSH-Win64\ssh_host_dsa_key
'NT SERVICE\sshd' now has Read access to 'C:\Program Files\OpenSSH-Win64\ssh_hos
t_dsa_key'.
Repaired permissions

[*] C:\Program Files\OpenSSH-Win64\ssh_host_dsa_key.pub
looks good

[*] C:\Program Files\OpenSSH-Win64\ssh_host_ecdsa_key
looks good

[*] C:\Program Files\OpenSSH-Win64\ssh_host_ecdsa_key.pub
looks good

[*] C:\Program Files\OpenSSH-Win64\ssh_host_ed25519_key
looks good

[*] C:\Program Files\OpenSSH-Win64\ssh_host_ed25519_key.pub
looks good

[*] C:\Program Files\OpenSSH-Win64\ssh_host_rsa_key
looks good

[*] C:\Program Files\OpenSSH-Win64\ssh_host_rsa_key.pub
looks good

Done.`

UGH!! again, if we just had a working script or even correct help, it would save time.

So that was it. This script is not running based on how you indicate to run it in your wiki help.

[codearoo]

So here is a script I wrote named 'setup-full.ps1' which seems to run everything properly. Maybe you'll include something like this in your project to help users.

$ErrorActionPreference = "Stop"

$scriptpath = $MyInvocation.MyCommand.Path
$scriptdir = Split-Path $scriptpath

$sshdpath = Join-Path $scriptdir "sshd.exe"

.\install-sshd.ps1

.\ssh-keygen.exe -A

.\FixHostFilePermissions.ps1 -Confirm:$false

Start-Service ssh-agent
psexec -s "$scriptdir\configure-ssh-keys.bat"

try
{
    New-NetFirewallRule -Program "$sshdpath" -Direction Inbound -Action Allow -DisplayName SSH
}
catch
{
    $ErrorMessage = $_.Exception.Message
    $FailedItem = $_.Exception.ItemName

    Write-Output "Caught error adding FW rule: $($ErrorMessage)"
    Write-Output "Going to attempt using 'netsh' command."

    netsh advfirewall firewall add rule name=SSHD dir=in action=allow program="$sshdpath"
}

Set-Service sshd -StartupType Automatic
Set-Service ssh-agent -StartupType Automatic

Start-Service sshd

[codearoo]

oh ya.. and it requires this other script named 'configure-ssh-keys.bat' as well as a copy of 'psexec'

pushd %~dp0
ssh-add ssh_host_dsa_key
ssh-add ssh_host_rsa_key
ssh-add ssh_host_ecdsa_key
ssh-add ssh_host_ed25519_key
popd

[codearoo]

Another error in your instructions is that you indicate to set up FW rule based on the port. But if you change the port in the sshd_config file, then you'd also have to change the port in the FW. If instead you set up the FW rule for the SSHD.EXE it will use whatever port it's configured for automatically.

[bingbing8]

@codearoo, the issue you saw when executing the script is a powershell issue which has fixed in pscore on git, but not fixed in windows yet. please refer to issue #844. if you need to workaround it, Powershell.exe -ExecutionPolicy Bypass -Command ". .\FixHostFilePermissions.ps1 -Confirm:$false"
Note the first "." is a call operator.

Thanks for your suggestion about the FW. The installation instruction has been updated for above workaround and setup FW rule for sshd.exe instead of by port.

psexec does not work on some platforms so we can't included as general installation steps. If you are interested, please see an example at here about use task scheduler to register host keys under system account. the script is at here

[codearoo]

Awesome.. thanks.
The Wiki has a few more updates maybe?
"Open the firewall on TCP port 22 to allow inbound SSH connections"
and:
"Note: New-NetFirewallRule is for servers only. If you're on a client desktop machine (like Windows 10) try:"
Is not entirely correct because Windows 2008 R2 Server also does not seem to have New-NetFireWallRule.

I'm not sure how that powershell issue is resolved. I upgraded to latest version of Powershell and still had that problem. So .. how is it resolved? And if this other syntax you mentioned works, shouldn't that be put in the wiki doc?

[codearoo]

And by the way, the link for "Alternative installation using the universal installer" does not work; it just goes back to the same page.

[bingbing8]

@codearoo, Alternative installation using the universal installer works for me. Updated the installation instruction based on your findings.
Regarding the powershell bug, did you install latest build from here? Please refer to the PR to find out how is it fixed. Please create new issue if you still see it repros.

[DarwinJS]

@codearoo - here is the target of that link: https://github.com/DarwinJS/ChocoPackages/blob/master/openssh/readme.md

[afcady]

I have got a fix for this that I have made into a script, but before I even upload it here is the most salient part:

@icacls ssh_host_*_key /grant "NT Service\sshd":(R)        >NUL
@icacls ssh_host_*_key /remove "%COMPUTERNAME%\%USERNAME%" >NUL
@icacls ssh_host_*_key /inheritance:r                      >NUL
@icacls ssh_host_*_key /grant "BUILTIN\Administrators:(F)" >NUL
@icacls ssh_host_*_key /grant "NT AUTHORITY\SYSTEM:(F)"    >NUL

Of course this should be patched into ssh-keygen.exe -- that is what is leaving the wrong permissions.

I have a .bat wrapper called ssh_keygen_works.bat that just runs ssh-keygen -A and then fixes the permissions like above. I will put it into the tree here and make a pull request.

By the way the permissions requirements are available by reading the code around here:

Win32-OpenSSH/contrib/win32/win32compat/w32-sshfileperm.c

Lines 94 to 99 in c4d0727

	if (!IsWellKnownSid(owner_sid, WinBuiltinAdministratorsSid) &&
	!IsWellKnownSid(owner_sid, WinLocalSystemSid) &&
	!EqualSid(owner_sid, user_sid)) {
	debug3("Bad owner on %s", path);
	ret = -1;
	goto cleanup;

[afcady]

Here's a direct link to ssh_keygen_works.bat:

https://raw.githubusercontent.com/afcady/Win32-OpenSSH/61fc955c1d1c1d51b917f5d8e83f6d9acf800d95/contrib/win32/openssh/ssh_keygen_works.bat

[manojampalam]

@afcady starting from V1.0.0.0, starting sshd will automatically generate host keys with the correct permissions. There is no need to explicitly run ssh-keygen -A.

[afcady]

@manojampalam if that's true, fixing the underlying issue in ssh-keygen.exe should be easy: just use the code that is used by sshd to set the permissions.

Do you know where that code is?

[afcady]

Found this commit (in a different repo!)

PowerShell/openssh-portable@32ff0c2

However, it appears that it only calls ssh-keygen.exe -A, and does not appear to fix any permissions explicitly.

[manojampalam]

You don't need to. ssh-keygen.exe -A is called in the context of SYSTEM (sshd runs as SYSTEM), the generated keys will automatically have the right permissions when called in SYSTEM context.

[rkeithhill]

But what about the case where a user installs/configures only the openssh client? They need to use ssh-keygen.exe -A to gen a key for connections to other machines (notably Git servers) and that key will likely get passed to ssh-add if it has a passphrase. That is where we are seeing this warning - ssh-add $home\.ssh\id_rsa.

[bagajjal]

I think you got confused.

ssh-keygen -A -> generates the host keys.
ssh-keygen -t -> generates the user keys, used for key-based authentication.

For the scenario mentioned above you don't need ssh-keygen -A

[rkeithhill]

Indeed. Yes, this would be for user keys so ssh-keygen -t. But wouldn't the same problem exist with ssh-keygen creating those keys with potentially wrong (ie default) permissions?

[bagajjal]

ssh-keygen -t - generates the keys with right permissions.