[git-clone-ssh recipe] Failed to retrieve list of SSH authentication methods: Failed getting response

[graphicore]

Originally I'm trying to do a git fetch, but the git-clone-ssh recipe has the same issue. I'm using a modified version of the recipe to make it a real world example:

• using a path to real SSH keys on my system and assert to proof they exist
• using the "git@github.com:libgit2/pygit2.git" url that GitHub suggests for ssh cloning

#! /usr/bin/env python3
import pygit2
import os

class MyRemoteCallbacks(pygit2.RemoteCallbacks):
    def credentials(self, url, username_from_url, allowed_types):
        if allowed_types & pygit2.credentials.GIT_CREDENTIAL_USERNAME:
            return pygit2.Username("git")
        elif allowed_types & pygit2.credentials.GIT_CREDENTIAL_SSH_KEY:
            sshkeys = os.path.join(os.getenv("HOME"), '.ssh')
            pubkey = os.path.join(sshkeys, 'id_rsa.pub')
            privkey = os.path.join(sshkeys, 'id_rsa')
            assert os.path.isfile(pubkey), f'isfile({pubkey})'
            assert os.path.isfile(privkey), f'isfile({pubkey})'
            return pygit2.Keypair("git", pubkey, privkey, "")
        else:
            return None

print("Cloning pygit2 over ssh")
pygit2.clone_repository("git@github.com:libgit2/pygit2.git", "pygit2.git",
                        callbacks=MyRemoteCallbacks())

After a while (and multiple calls to the callback method) the script fails:

(venv) $ ./gitclone.py 
Cloning pygit2 over ssh
Traceback (most recent call last):
  File "./gitclone.py", line 21, in <module>
    pygit2.clone_repository("git@github.com:libgit2/pygit2.git", "pygit2.git",
  File "/home/username/path/to/example/gftools/venv/lib64/python3.8/site-packages/pygit2/__init__.py", line 214, in clone_repository
    payload.check_error(err)
  File "/home/username/path/to/example/gftools/venv/lib64/python3.8/site-packages/pygit2/callbacks.py", line 93, in check_error
    check_error(error_code)
  File "/home/username/path/to/example/gftools/venv/lib64/python3.8/site-packages/pygit2/errors.py", line 65, in check_error
    raise GitError(message)
_pygit2.GitError: Failed to retrieve list of SSH authentication methods: Failed getting response

If i use the underdocumented pygit2.KeypairFromAgent Keypair Constructor the script works without problems:

#! /usr/bin/env python3
import pygit2
import os

class MyRemoteCallbacks(pygit2.RemoteCallbacks):

    def credentials(self, url, username_from_url, allowed_types):
        if allowed_types & pygit2.credentials.GIT_CREDENTIAL_USERNAME:
            return pygit2.Username("git")
        elif allowed_types & pygit2.credentials.GIT_CREDENTIAL_SSH_KEY:
            return pygit2.KeypairFromAgent("git")
        else:
            return None

print("Cloning pygit2 over ssh")
pygit2.clone_repository("git@github.com:libgit2/pygit2.git", "pygit2.git",
                        callbacks=MyRemoteCallbacks())

Here's the result:

(venv) $ ./gitclone.py 
Cloning pygit2 over ssh
(venv) $ ls pygit2.git/
appveyor.yml  AUTHORS.rst  CHANGELOG.rst  COPYING  docs  Makefile  misc  pygit2  pyproject.toml  pytest.ini  README.rst  setup.cfg  setup.py  SPONSORS.rst  src  test  travis

expected:

• The return pygit2.Keypair("git", pubkey, privkey, "") version should be fixed or the documentation should show how to use it correctly in real live.
• pygit2.KeypairFromAgent("git") should also be an example in the git-clone-recipe. It is actually really what I'm going to use and what is the best fit for my needs, but because of the issue and the example in the git-clone-recipe I was side tracked a considerable amount of time.

Here's a related issue: saltstack/salt#57121
(I'm also interested in leaving a trace of my findings in the web, so that others can solve their issues faster, hence this is very verbose.)

[martinellison]

Does anyone have any ideas about this? I have the same error (but using Rust).

[cprogrammer]

Same issue here. This is what I have found.

The source ssh.c

	/* either error, or the remote accepts NONE auth, which is bizarre, let's punt */
	if (list == NULL && !libssh2_userauth_authenticated(session)) {
		ssh_error(session, "Failed to retrieve list of SSH authentication methods");
		return -1;
	} 

tell us that "the remote accepts NONE auth, which is bizarre"

In my case this is not bizarre and is related to this issue : ssh-keygen-does-not-create-rsa-private-key

The key is generated by ssh-keygen on Debian Buster, that has
https://packages.debian.org/buster/libssh2-1 1.8.0-2.1

the code is compiled on Debian Stretch, that has
https://packages.debian.org/stretch/libssh2-1 1.7.0-1+deb9u1

The change between these two that might impact is
https://libssh2.org/changes.html#1.8.0
*openssl: add OpenSSL 1.1.0 compatibility

This seems to be related to a diff in key format that can be easily checked by comparing the private key:
libssh2 1.8.0-2.1

-----BEGIN OPENSSH PRIVATE KEY-----
b3BlbnNzaC1rZXktdjEAAAAABG5vbmUAAAAEbm9uZQAAAAAAAAABAAACFwAAAAdzc2gtcn
.....

libssh2 1.7.0-1 : old style PEM key

-----BEGIN RSA PRIVATE KEY-----
MIIJKAIBAAKCAgEAu8YSVyYYtbvZQ5g4ntpHTstVbhXRYijx45/QlFBmHdzdFE1s

A solution is to generate the key in the old, recognizable format by using -m PEM
ssh-keygen -m PEM -t rsa -b 2048 -f jabba -C 'ronnie-jabba'

Didn't found a solution that avoids generating the key ....

Hope this is useful.

[ysalmon]

I have to report that the problem arises with one of my old-style PEM private keys.
It does not, however, if I decrypt it into a (temporary) file and use that.

[serg-vinnie]

I had same error when I accidentally swapped private and public keys

[viq]

@cprogrammer you can convert existing keys between formats, e.g. ssh-keygen -e -f ~/.ssh/id_rsa -m PEM > ~/.ssh/id_rsa.pem

Though doing that didn't solve the issue for me, yet.

[DaAwesomeP]

This bug seems to make it impossible to use non-RSA keys like ed25519?

[gvecchicert]

please see saltstack/salt#57121 for my workaround

[jdavid]

Try with the latest release v1.11.1 which includes openssl 1.1