SSH prompts not showing up on Windows

[the-real-nps]

Describe the bug

I'm using the latest version 2.9.3 and #12756's functionality isn't there: when I try to clone using SSH URL and I don't have an appropriate entry in the known_hosts file it ends with "Authentication failed" error. If I have the appropriate entry in known_hosts then cloning works fine. But I have to add the entry by other means (e.g. "ssh URL" in the command line), GitHub Desktop doesn't ask if I want to add it as in the video.

Version & OS

Version: 2.9.3 (x64)
Windows 10

Steps to reproduce the behavior

1. Make sure the known_hosts file doesn't contain an entry for the URL you want to clone from.
2. Open File/Clone repository...
3. Choose tab URL
4. Paste SSH URL
5. Click Clone

Expected behavior

GitHub Desktop shows a prompt asking whether the user wants to continue with given URL. If the user accepts, GitHub Desktop adds an appropriate entry in the known_hosts file and proceeds to clone the repo successfully.

Actual behavior

GitHub Desktop shows "Authentication failed" error and aborts cloning.

[sergiou87]

Hey @the-real-nps! 👋

Sorry to hear it doesn't work for you 😞  Could you share the app logs so I can take a look and see what's wrong? This has worked in my multiple tests on Windows and macOS, so I wonder what could be different here.

You can get the logs from Help -> Show logs in Explorer

[the-real-nps]

Unfortunately, I can't. It's an enterprise github and our company's policy prevents me from sharing stuff like that. But I can show you the error log:

error: [ui] `git -c credential.helper= -c init.defaultBranch=master clone --recursive --progress -- <repo url> <local dir>` exited with an unexpected code: 128.
stderr:
Cloning into '<local dir>'...
Host key verification failed.
fatal: Could not read from remote repository.

Nothing else in the logs was relevant anwyay. Hope that helps. If I can help in any other way please let me know.

[sergiou87]

No worries, thanks for that! Maybe the problem is you're not using an RSA key, that's a bug that was fixed in #12894 but it's not released yet.

Could you try and install the latest beta and see if it works there? 🙏

https://github.com/desktop/desktop#beta-channel

[the-real-nps]

I've just tried the beta, unfortunately, same result, same error in the logs.

[sergiou87]

Ouch! Could you try enabling this setting to see if it helps?

[the-real-nps]

I had OpenSSH enabled from the beginning.

[sergiou87]

Cool! 👍  Then I'd say try two things:

1. Keep the known_hosts without the fingerprint from your enterprise server, and from a console run ssh git@<your server>. For example, if your repo was hosted on GitHub, it'd be ssh git@github.com
   That should show a prompt (in the command line, I'd hope) asking you to confirm if that host should be added to known_hosts. It'd be great if you could share that prompt (removing the fingerprint, hostname and IP address). The app expects it to match to something like this:

The authenticity of host 'github.com (123.123.123)' can't be established.
RSA key fingerprint is SHA256:nThbg6kXUpJWGl7E1IGOCspRomTxdCARLviKw6E5SY8.
Are you sure you want to continue connecting (yes/no/[fingerprint])?

More specifically, this is the regex used to parse the prompt:

desktop/app/src/lib/ssh/ssh.ts

Line 76 in acbaeb0

const promptRegex = /^The authenticity of host '([^ ]+) \(([^\)]+)\)' can't be established.\n([^ ]+) key fingerprint is ([^.]+).\n(?:.*\n)*Are you sure you want to continue connecting \(yes\/no\/\[fingerprint\]\)\? $/

2. Try disabling that checkbox to see if it works like that. I'd expect it not to work either, but it's worth checking 🤞

[the-real-nps]

It does work with OpenSSH disabled. But I assume we want it to work with OpenSSH too?

[sergiou87]

Yay! Yeah, it should work with OpenSSH enabled too 😕 And in that case (1) from my previous message should help clarify if it failed to parse SSH's prompt 🤞

[the-real-nps]

Yeah, I'm going to check that but tomorrow.

[Skwara]

@sergiou87 I think the regex you provided is not matching our (I work with @the-real-nps) prompt.
In our case it is:

The authenticity of host '<host>' (<ip>)' can't be established.
ECDSA key fingerprint is <fingerprint>.
Are you sure you want to continue connecting (yes/no)?

The \/\[fingerprint\] part at the end of your regex is not matched.

[sergiou87]

Hah! Interesting, thank you so much for the info! I'll make that part optional to fix this ASAP ❤️

[Skwara]

I wonder why you decided to make the regex so specific? Are there any drawbacks for just making it ^The authenticity of host .* or something in between?
Also, if you want to be specific, then you should escape the line ending dots for line 1 and 2.

[sergiou87]

We're parsing the host, IP, the kind of key (RSA, ECDSA…) and the fingerprint itself, but it's true we could skip the third line. I'll see if we can make this a bit more resilient, thank you! ❤️

[sergiou87]

@Skwara @the-real-nps a new beta went out yesterday that should fix this issue 😄
Please give it a try and let us know! Thank you for your patience and help 🙇‍♂️

[the-real-nps]

Unfortunately, it doesn't work (Version 2.9.4-beta4 (x64)). I've tested the output (prompt) from ssh <URL> against the new regex: ^The authenticity of host '([^ ]+) \(([^\)]+)\)' can't be established[^.]*\.\n([^ ]+) key fingerprint is ([^.]+)\. Note that I wasn't able to do it directly: ssh <URL> | grep ... doesn't redirect the prompt message to grep. So I manually copied the entire prompt from my Git Bash terminal to Notepad++, then tried the regex. It didn't match but only because the copied prompt had \r\n instead of \n that the regex expects. But it's possible that Windows added the \r while copying so I don't know if the original ssh prompt message contains \r or not. If it's not that then I don't know why it failed.

[sergiou87]

😐
What I don't understand is why your ssh is printing that string 🤔  AFAIK it's using OpenSSH, and it always uses \n and shows [fingerprint] as an option when the prompt starts with The authenticity of host (see https://github.com/openssh/openssh-portable/blob/53237ac789183946dac6dcb8838bc3b6b9b43be1/sshconnect.c#L1169).

Could you open a command line and run this?
c:\Windows\System32\OpenSSH\ssh.exe -V

This is the output I'm getting on Windows 11:

>c:\Windows\System32\OpenSSH\ssh.exe -V
OpenSSH_for_Windows_8.1p1, LibreSSL 3.0.2

Also, which exact version of Windows 10 are you using?

[Skwara]

It also didn't help on my side on 2.9.4-beta4
OpenSSH version: OpenSSH_for_Windows_7.7p1, LibreSSL 2.6.5
Windows version: Windows 10 Enterprise, version 1809, OS build 17763.2183

[the-real-nps]

I have the same setup.

why your ssh is printing that string

Like I said, I don't know if my ssh prints \n or \r\n as Windows might be adding \r during copying.

[sergiou87]

Cool, thanks for that info! I see that version of OpenSSH for Windows indeed has a (yes/no) string instead: https://github.com/PowerShell/openssh-portable/blob/9369d870ade51a44f4c2b7b7f1202fe1fdf7e7bb/sshconnect.c#L1080-L1085

I don't see \r\n being used in that prompt, though. Let's see if I can get that version of OpenSSH for Windows and reproduce this myself…

[the-real-nps]

My exact ssh prompt message is:

The authenticity of host '<URL> (<IP address>)' can't be established.
ECDSA key fingerprint is SHA256:<SHA hash>.
This key is not known by any other names
Are you sure you want to continue connecting (yes/no/[fingerprint])?

[sergiou87]

And it's parsed correctly according to this… https://regex101.com/r/IFOJoV/1

🤯

[the-real-nps]

Well, for me it all matched except for the possible \r.

[sergiou87]

Could you try to do this with that beta4?

Screen.Recording.2021-10-05.at.12.17.37.mov

Basically, in the developer tools, go to ssh.ts and put a breakpoint in the if that checks the results of matching the regex. Then, attempt to fetch/clone the ssh-based repository to make the debugger pause at the breakpoint. In that context, e is the prompt received from SSH (you can inspect it to see what it looks like and whether or not it has \r in it), and t is the object with the regex matches.

[the-real-nps]

I did that and it didn't even pause at the breakpoint. In the debugger I got 2 errors:

Executing fetch: git [...] fetch [...] (took 1.067s)

git [...] fetch [...] exited with an unexpected code: 128.
stderr:
Host key verification failed.
fatal: Could not read from remote repository.

[...]

[AppStore.getAccountForRemoteURL] account found for remote: git@<URL>/<repo name>.git - (has token)

git [...] clone [...] exited with an unexpected code: 128.
stderr:
Cloning into: '<dir>'...
Host key verification failed.
fatal: Could not read from remote repository.

[...]

[Skwara]

What I did:

1. I set the breakpoint on ssh.ts:79.
2. I removed known_hosts file.
3. I tried to clone the repo via ssh - the app didn't stop on the breakpoint and I got the usual error.

Because of the above, I did the following:

1. I brought back known_hosts file.
2. I cloned the repo successfully.
3. I removed known_hosts file.
4. I pressed "Fetch" - the app didn't stop on the breakpoint and I got the usual error.

So just to check, I did the following:

1. I brought back known_hosts file.
2. I pressed "Fetch" - the app didn't stop on the breakpoint and the fetch was successful.

So it doesn't stop at the breakpoint for me. Am I doing something wrong?

[sergiou87]

Hah! That's interesting. Maybe there is another prompt we aren't expecting or you're having a different error, so it's never getting to that SSH prompt 😐

Try setting a breakpoint at line 94 of trampoline-askpass-handler.ts and inspect the contents of e (which in that context corresponds to command):

PS: Thanks so much for helping me debugging this!!!

[Skwara]

I can see that upon fetching, the command.parameters array is empty (but the breakpoint on line 94 is reached).
I've set breakpoints at lines 90, 91 and 94. It reaches them all one by one even though we have a return there. What is going on?!

[sergiou87]

Hmm AFAIK it's not entirely weird to see how it "skips" the return since what you are seeing there is the original source code, but not what is actually running (hence why I tell you to inspect e, or t or things like that), which is the TypeScript code transpiled to JS and then minified/optimized.

Did you check command.parameters or e.parameters? In a production/beta build command.parameters shouldn't exist. e.parameters is what you should inspect in that build:

[Skwara]

I inspected e and I see parameters: Array(0) length: 0.

[sergiou87]

😥  Then I don't understand anything 😐

We can also try to run whatever GitHub Desktop is doing. If you have something like Git Bash, you can try this:

GIT_SSH_COMMAND="C:/Windows/System32/OpenSSH/ssh.exe" git [...] clone [...]

Where those [...] are what you see in the logs. The GIT_SSH_COMMAND env variable will make git use Windows' OpenSSH, which is the one failing. With that, you should be able to see the same behavior, or that's what I'd expect 🤞

[Skwara]

git -c credential.helper= -c init-defaultBranch=master clone --recursive --progress -- <url>
While running the above (in Git Bash), I get a popup error window with the following:
Title: Error!
Message: Usage: git askpass <prompt>
Buttons: OK

After clicking on OK, I get the usual:
Host key verification failed.

[the-real-nps]

I get a different result (also Git Bash):

GIT_SSH_COMMAND="C:/Windows/System32/OpenSSH/ssh.exe" git [...] clone [...]
Cloning into '<dir>'...
CreateProcessW failed error:193
ssh_askpass: posix_spawn: Unknown error
Host key verification failed.
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

[Skwara]

I also tried running it in windows CMD. It asked me about adding to known_hosts and cloned without problems.

[sergiou87]

Could you try removing the -c credential.helper= part? I'm surprised it asks for askpass with just those parameters… O_o

[the-real-nps]

Same result for me without -c credential.helper=.

[sergiou87]

I am completely clueless here… I don't understand why it fails the host key verification without prompting first. Do you have anything in your ~/.ssh/config maybe? I wish I could reproduce that behavior here 😕

[the-real-nps]

Don't even have ~/.ssh/config.

[Skwara]

@sergiou87 After further investigation we've established that the problem was not GitHub Desktop's fault. Upon running the same command that GitHub Desktop is running we received errors stated above (different for each of us). Because of that I think these errors are the root cause, not GitHub Desktop.
The upside is that at least we found another known_hosts prompt variant so when I'll fix the root cause, GitHub Desktop should work out of the box.
Thank you for your help.

[sergiou87]

Ohh! Thanks for reporting back! ❤️