Git hangs during SSH session to Windows host

[brunovieira97]

I have a Windows desktop that acts as a server for development. On it, latest Git for Windows version with bundled GCM Core.

I'm running VS Code Remote SSH, with its backend on said Windows desktop, and whenever I run a Git command like pull, it hangs indefinitely. Accessing the host through RDP shows me a huge number of GCM Core instances.

My Code workspace has 15 repositories, but there are 63 instances (!) of the Credential Manager running. Probably VS Code's automatic fetch has something to do with the instance count, but anyway, I think the Credential Manager should work during SSH sessions instead of hanging.

VSCode's Git UI (has been like this for an hour) (client):

VSCode's Git output (client):

> git config --get commit.template
> git config --local branch.homolog.github-pr-owner-number
> git log --oneline --cherry homolog...homolog@{upstream} --
> git pull --tags origin homolog
> git log --oneline --cherry homolog...homolog@{upstream} --
> git pull --tags origin homolog
> git log --oneline --cherry homolog...homolog@{upstream} --
> git pull --tags origin homolog
> git log --oneline --cherry homolog...homolog@{upstream} --
> git pull --tags origin homolog
> git log --oneline --cherry homolog...homolog@{upstream} --
> git pull --tags origin homolog
> git log --oneline --cherry homolog...homolog@{upstream} --
> git pull --tags origin homolog
> git log --oneline --cherry homolog...homolog@{upstream} --
> git pull --tags origin homolog
> git log --oneline --cherry homolog...homolog@{upstream} --
> git pull --tags origin homolog
> git log --oneline --cherry homolog...homolog@{upstream} --
> git pull --tags origin homolog
> git log --oneline --cherry homolog...homolog@{upstream} --
> git pull --tags origin homolog
> git log --oneline --cherry master...master@{upstream} --
> git pull --tags origin master
> git log --oneline --cherry master...master@{upstream} --
> git pull --tags origin master
> git log --oneline --cherry master...master@{upstream} --
> git pull --tags origin master
> git log --oneline --cherry master...master@{upstream} --
> git pull --tags origin master
> git log --oneline --cherry master...master@{upstream} --
> git pull --tags origin master

GCM instances on the SSH server:

PS> ps | grep "git-credential-manager"
     24    34,36      46,49      35,92    3140   0 git-credential-manager
     24    34,50      46,62      34,64    3368   0 git-credential-manager
     24    34,32      44,79      30,77    4092   0 git-credential-manager
     24    34,48      46,66      32,52    4908   0 git-credential-manager
     24    34,54      46,70      30,66    5660   0 git-credential-manager
     24    34,41      45,12      36,62    5676   0 git-credential-manager
     24    34,66      44,77      34,20    7788   0 git-credential-manager
     24    34,55      46,69      30,97   10040   0 git-credential-manager
     24    34,55      46,69      29,89   10628   0 git-credential-manager
     24    34,34      46,48      27,58   10652   0 git-credential-manager
     24    34,40      46,56      27,34   10788   0 git-credential-manager
     24    34,43      46,57      29,56   12364   0 git-credential-manager
     24    34,50      46,62      28,00   13124   0 git-credential-manager
     24    34,44      46,62      33,92   13440   0 git-credential-manager
     24    34,28      46,49      24,03   13940   0 git-credential-manager
     24    34,48      44,88      32,70   14052   0 git-credential-manager
     24    34,41      44,87      31,92   14088   0 git-credential-manager
     24    34,36      46,54      24,86   14512   0 git-credential-manager
     24    34,38      45,09      34,23   14548   0 git-credential-manager
     24    34,52      45,11      33,69   14596   0 git-credential-manager
     24    34,34      46,55      35,91   14688   0 git-credential-manager
     24    34,47      45,05      37,95   14772   0 git-credential-manager
     24    34,46      46,59      30,88   14840   0 git-credential-manager
     24    34,41      44,93      33,80   14852   0 git-credential-manager
     24    34,34      46,54      34,62   14876   0 git-credential-manager
     24    34,34      44,88      32,22   15048   0 git-credential-manager
     24    34,38      46,58      30,03   15116   0 git-credential-manager
     24    34,51      46,64      31,88   15124   0 git-credential-manager
     24    34,43      45,02      31,36   15300   0 git-credential-manager
     24    34,52      46,65      36,50   15660   0 git-credential-manager
     24    34,38      44,93      32,64   15792   0 git-credential-manager
     24    34,23      44,55      35,11   15932   0 git-credential-manager
     24    34,44      44,70      34,89   15968   0 git-credential-manager
     24    34,44      44,69      35,52   16028   0 git-credential-manager
     24    34,36      44,73      32,94   16172   0 git-credential-manager
     24    34,33      46,49      33,39   16292   0 git-credential-manager
     24    34,43      46,57      34,44   16576   0 git-credential-manager
     24    34,43      46,59      29,73   16592   0 git-credential-manager
     24    34,45      46,62      31,94   17180   0 git-credential-manager
     24    34,44      46,59      32,30   17248   0 git-credential-manager
     24    34,48      46,61      31,47   17428   0 git-credential-manager
     24    34,51      46,64      30,94   17608   0 git-credential-manager
     24    34,40      46,57      32,20   17756   0 git-credential-manager
     24    34,31      46,48      31,53   17932   0 git-credential-manager
     24    34,45      46,57      35,67   18660   0 git-credential-manager
     24    34,46      46,57      32,95   18936   0 git-credential-manager
     24    34,36      46,55      31,91   18944   0 git-credential-manager
     24    34,41      46,57      28,33   19580   0 git-credential-manager
     24    34,39      46,53      24,59   19660   0 git-credential-manager
     24    34,47      46,61      32,92   19844   0 git-credential-manager
     24    34,39      46,51      32,80   19932   0 git-credential-manager
     24    34,47      46,60      32,72   20308   0 git-credential-manager
     24    34,50      46,66      34,03   20316   0 git-credential-manager
     24    34,39      46,52      31,70   20808   0 git-credential-manager
     24    34,32      46,51      29,61   21128   0 git-credential-manager
     24    34,47      46,62      25,45   21224   0 git-credential-manager
     24    34,49      46,59      28,50   21700   0 git-credential-manager
     24    34,50      46,71      26,53   21872   0 git-credential-manager
     24    34,39      46,51      25,12   22204   0 git-credential-manager
     24    34,34      46,54      26,66   22384   0 git-credential-manager
     24    34,39      46,60      23,31   22580   0 git-credential-manager
     24    34,40      46,53      23,80   22872   0 git-credential-manager
     24    34,37      46,53      23,88   23400   0 git-credential-manager

[brunovieira97]

Had to force kill the GCM instances on Task Manager, and VSCode reported:

Missing or invalid credentials.
Skip silent fetch commands
Missing or invalid credentials.
Skip silent fetch commands
fatal: Failed to enumerate credentials. [0x520]

[brunovieira97]

Anyone?

[mjl5007]

I'm running a similar setup, and having a similar issue.

In my case, the Windows 10 host is also running WSL2 with Ubuntu 20 LTS, and I have configured Windows' OpenSSH server to use bash.exe as the default shell, which means when connecting to this host via SSH I'm actually dropped into the WSL/Ubuntu environment. I'm then connecting from a macOS machine to the WSL environment via VSCode Remote-SSH (see here).

On the Windows host, I have a recent version of Git for Windows installed, and Git on Windows is configured to use the manager-core helper -- everything works fine using Git locally on the Windows host. Git within Ubuntu/WSL2 was initially configured to use the old GCM (since the MS docs here haven't been updated since GCM Windows was deprecated -- MicrosoftDocs/WSL#995). Eventually I realized my error and reconfigured Git within Ubuntu/WSL2 to use GCM Core.

Using VSCode Remote-WSL on the Windows host itself, I'm able to connect to repos within the Ubuntu/WSL2 environment and fetch/pull/push to HTTPS remotes on Azure DevOps just fine. The same operations also work if I log into Ubuntu/WSL2 via the command-line (Ubuntu app shortcut, wsl.exe, etc.) and run e.g. git pull manually. This all worked when Git within Ubuntu/WSL2 was configured to use GCM Windows, as well as after I changed it to use GCM Core.

However, when I connect to the WSL2 environment by SSH to the Windows host from my macOS machine -- either directly from a terminal (e.g. ssh user@windows-host) or with VSCode Remote-SSH, fetch/pull/push operations do not work. When Git within WSL2 was using GCM Windows, Git fetch/pull/push operations just hung, as the OP described. After reconfiguring Git within WSL2 to use GCM Core, those operations now fail within seconds with the following error message:

fatal: Failed to enumerate credentials. [0x520]
fatal: A specified logon session does not exist. It may already have been terminated
And I'm then prompted for a password for the Git remote (either on the command line if I'm logged in on a terminal, or via the VSCode password popup prompt if I'm using VSCode Remote-SSH).

Since everything works when logged into WSL2 on a terminal on the Windows host, but not when logged in on a terminal via SSH from a remote host, I suspect A) it has nothing to do with VSCode, and B) it has something to do with the login session being from a remote machine rather than from the local Windows host. The second line of the error message -- "specified logon session does not exist" -- makes me think that the remote method doesn't have a locally-attached logon session in which GCM Core can do what it needs to do, so GCM Core just fails and Git thus reverts to the standard password prompt.

[mjl5007]

Should've looked before posting; guessing #325 is the same underlying issue.

[mjcheetham]

Hi @brunovieira97,

It looks like you're seeing lots of the old GCM for Windows instances (process: git-credential-manager), not GCM Core (process: git-credential-manager-core):

24 34,47 46,61 32,92 19844 0 git-credential-manager
24 34,39 46,51 32,80 19932 0 git-credential-manager
24 34,47 46,60 32,72 20308 0 git-credential-manager
24 34,50 46,66 34,03 20316 0 git-credential-manager

@mjl5007,

Since everything works when logged into WSL2 on a terminal on the Windows host, but not when logged in on a terminal via SSH from a remote host, I suspect A) it has nothing to do with VSCode, and B) it has something to do with the login session being from a remote machine rather than from the local Windows host. The second line of the error message -- "specified logon session does not exist" -- makes me think that the remote method doesn't have a locally-attached logon session in which GCM Core can do what it needs to do, so GCM Core just fails and Git thus reverts to the standard password prompt.

Should've looked before posting; guessing #325 is the same underlying issue.

I suspect this is indeed the same issue as in #325. When connecting over SSH, the OpenSSH server does not create a logon session that is sufficient for the WinCred APIs to operate (where we store credentials with the OS).

[mjcheetham]

@mjl5007 I believe your SSH/remote-session issue may be solved by a new feature to allow a different credential store on Windows. Please see my comment on another issue.