Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Paramiko regards valid RSA private keys as invalid #340

Closed
vitalyisaev2 opened this issue May 29, 2014 · 17 comments

Comments

@vitalyisaev2
Copy link

commented May 29, 2014

OS: RHEL 6.4, product version: python-paramiko-1.7.5-2.1.el6.noarch

I did not manage to pass to paramiko RSA private key issued by oVirt manager CA (also known as Red Hat Enterprise Virtualization Manager).

[root@vitaly registration]# ssh rhevaio
root@rhevaio's password: 
Last login: Thu May 29 15:03:33 2014 from 10.0.55.84
[root@rhevaio ~]# scp /etc/pki/ovirt-engine/keys/engine_id_rsa root@vitaly.dev.ru:/tmp/ololo/engine_id_rsa_RHEVAIO
root@vitaly.dev.ru's password: 
engine_id_rsa                                                                                                                                                                100% 1708     1.7KB/s   00:00    
In [1]: import paramiko

In [2]: pk = paramiko.RSAKey.from_private_key_file("/tmp/ololo/engine_id_rsa_RHEVAIO")
---------------------------------------------------------------------------
SSHException                              Traceback (most recent call last)

/home/vitaly/progs/registration/<ipython console> in <module>()

/usr/lib/python2.6/site-packages/paramiko/pkey.pyc in from_private_key_file(cls, filename, password)
    195         @raise SSHException: if the key file is invalid
    196         """
--> 197         key = cls(filename=filename, password=password)
    198         return key
    199     from_private_key_file = classmethod(from_private_key_file)

/usr/lib/python2.6/site-packages/paramiko/rsakey.pyc in __init__(self, msg, data, filename, password, vals, file_obj)
     49             return
     50         if filename is not None:
---> 51             self._from_private_key_file(filename, password)
     52             return
     53         if (msg is None) and (data is not None):

/usr/lib/python2.6/site-packages/paramiko/rsakey.pyc in _from_private_key_file(self, filename, password)
    162 
    163     def _from_private_key_file(self, filename, password):
--> 164         data = self._read_private_key_file('RSA', filename, password)
    165         self._decode_key(data)
    166 

/usr/lib/python2.6/site-packages/paramiko/pkey.pyc in _read_private_key_file(self, tag, filename, password)
    277         """
    278         f = open(filename, 'r')
--> 279         data = self._read_private_key(tag, f, password)
    280         f.close()
    281         return data

/usr/lib/python2.6/site-packages/paramiko/pkey.pyc in _read_private_key(self, tag, f, password)
    287             start += 1
    288         if start >= len(lines):
--> 289             raise SSHException('not a valid ' + tag + ' private key file')
    290         # parse any headers first

    291         headers = {}

SSHException: not a valid RSA private key file

However, I can use this key to start trivial ssh session with oVirt server. It's absolutely OK:

ssh -i /tmp/ololo/engine_id_rsa_RHEVAIO root@<host>

See below the body of the key (I've changed some random bytes in signature due to security reasons). Have no idea why doesn't it match Paramiko's standards :

-----BEGIN PRIVATE KEY-----
MIIEvwIBADANBgkqhkiG9w0BAQEFAASCBKkwggSlAgEAAoIBAQCUGqhnDEccgzzl
7ZG3yOg6ohSm+Eks6YbZGYLZ79xjbvOff9OHCklMPrYWYlCxscaBj3OT2mG2pY2v
ibkBiy3Y5N97LyyhBgk1vVFBww8Bg/kcnwaJ/VmzU6ZqG4b1f+iTo1ZadxYGLML5
IzPQwrj2LHaWi5LfKnRIKy8Vw2GevfNDB3w5jkKxJ/pq+cocUpOIfdbKCfVaE90b
1PMN9JOtKgqomu1eWpn9dbsKQNDogKtLTPTg0e/i7ytRQErP8krO8FV/LIAp/Jmo
hEsDPNuduofTl5vbHf3TgRqAq60fVNzq82XhQkYWnEu5n1sIwaxll4/k8rcr9BPa
eC87SyQZAgMBAAECggEAVSrLQo7t6biScYpfLHetC+Key7Glqt0TvY6ND7R+YPOx
xzvjsRk6M00wusiIrNaVV3Xdud1wKW8kfhhNulSF5d8foJ/tIx/sdenLa2y3GWEC
BPJx+TImvhfRb90qi8ylOxlHlDg9AwoEiO40Ys1HPdEt1L6olAq+2a1zoyk63uRu
SjIGYOgW4UpOLw8h04PZKTzpQsl0rYcFeVhbiU3uNvbmywxMxQOUQeADD7lY4K3v
boNFFDsxEaAIo5vVkYSSuS383XhJhiHRe6RfaKwyR4Wwn+KtFa03gT12RWBhxWtz
B9+xYTKxc1w1qHBo3Ki8gLbQZQDxOd9chjUpyDX1AQKBgQDOLBnez+1TOWyFmbKh
jdfngGfgENzoMWsIz0SrXoiKp7e5YuhP+vRRNUvbM0fZzSfhxg4LmH05RpvpyCSH
0tAgsHDae5LTGbfGExPzrnYUe7x5Jololavsa4Lp8IgUWjUpyzWE4fBhm9un9mh4
v6LgMj80dnI/LJJ3P8uAdNuJeQKBgQC35eASrVRJeDoT2syfvsxs603irE45aD34
y1Tz25CxQlRepAyJ/iMTFqw/twsqdc0Jze/9EsZSHM2shkXLVbJqJlGFKorXMkct
7nrQthX+6Qq2GMa8tJ5ejkAK2ACbJ9eM5hReq+rgGQifcBxj1loceYht0+i+TCZC
qetw/7pnoQKBgQC4RHZCzDfi4Dn8FmivN6U2RW+RtcCazkBJuWY4e98xfcEZMitM
AJGIW0gW+/hF07THd0pqVrnlDRSgl77+ng5iiQ/0VenePugfYkG807q5h/XlmUqg
FiI4xcZV7CLTfeRyHCsTPxSnSLPDpNmuwiiClKYk75AgometqRjrEIZ+kQKBgQCs
m/3oSfzVQR3ySOrL/kDO80YVvvlldOzGO8QvaCW127ZpZr3GQF4Xqq5e16aJYizk
SQI7noW7yQFcMGQOaQ+gBvt5bOcI+k5JVJC+mkqnC4JcTYEs2rPTuN1v0DMxbwIp
ZEQ2YFODqLupO4fKRgYquABQZhkNlVhMNkWhuMb0QQKBgQCpVfbzX2VxkBIarkFD
m10iKstxnczWfob2rP4Kmq+vOW9Q9awZBy2n4HkVrAKAPJssLuTbZOerSDfc0/PP
NzkL5G8PKkpZuH/d9YzEksNRht8HGwIHfqM/CjZSpYttOTYGIEIur6t/V8Xn43Ru
1Si5Uh2U6IcsU0DBgU54EfhXjw==
-----END PRIVATE KEY-----
@rfjakob

This comment has been minimized.

Copy link

commented Jun 6, 2014

Normally, keys start with this line, and this is what paramiko is checking for:

-----BEGIN RSA PRIVATE KEY-----

But as SSH does not seem to care if "RSA " is there or not, paramiko shouldn't, either, I guess.

@bitprophet

This comment has been minimized.

Copy link
Member

commented Aug 8, 2014

If you're truly on Paramiko 1.7.5, that version is quite ancient at this point and wholly unsupported :( sorry! If you can recreate this on an edition newer than 1.8 or so, please feel free to reopen. thanks!

@bitprophet bitprophet closed this Aug 8, 2014
@ghost

This comment has been minimized.

Copy link

commented Nov 8, 2014

I am experiencing this problem with paramiko in Debian Jessie (python-paramiko 1.15.1-1) with a key generated with: ssh-keygen -b4096 and adding a passphrase. The problem only occurs when applying a passphrase to the key.

I am using paramiko through duplicity and get the following error message
BackendException: ssh connection to test@remote failed: not a valid EC private key file
I then found out through the fine manual that duplicity won't allow me to use passphrased SSH key files and still had this thread open, so I figured I'll report my findings here.

I have not read enough about paramiko to know whether you intend to support passphrased SSH key files or not, but at least this post may offer an explanation to the original poster as well as to any future web searches that experience the "not a valid RSA private key file" or whatever format was expected.

@Ghost0817

This comment has been minimized.

Copy link

commented Apr 13, 2015

How about SSH2 PUBLIC KEY connection?

@edouard-lopez

This comment has been minimized.

Copy link

commented Sep 29, 2016

Still happen with 2.0.2.

Env

version

paramiko==2.0.2

OS

Docker
Linux Mint 18

Key generation

ssh-keygen -q \
            -t rsa \
            -b 4096 \
            -f "${SSH_KEYFILE}" \
            -N "${EMPTY}" \
            -O permit-port-forwarding \
            -C "Automatically generated by MAST script"

Connection code

import os
import paramiko

def open_ssh_connection(hostname, port=22, ssh_key=None):
    paramiko.util.log_to_file('/tmp/ssh.log')  # sets up logging

    try:
        client = paramiko.SSHClient()
        if ssh_key:
            print('key: ' + ssh_key )
            client.load_system_host_keys(ssh_key)
            client.load_host_keys(ssh_key)
        client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
        client.connect(hostname, port=port, timeout=0.3, username='toto')

    except paramiko.BadHostKeyException as e:
        raise Exception('BadHostKeyException on ' + hostname)
    except paramiko.AuthenticationException as e:
        raise Exception('AuthenticationException on ' + hostname)
    except paramiko.SSHException as e:
        raise Exception('SSHException on ' + hostname)

    return client

my_key = os.path.expanduser('/home/mast/.ssh/id_rsa.toto')
hostname = '192.168.2.231'
connection = open_ssh_connection(hostname)
stdin, stdout, stderr = connection.exec_command('ls -l')

for line in stdout:
    print(line)

Error

key: /home/mast/.ssh/id_rsa.toto
Traceback (most recent call last):
File "/tmp/ssh.py", line 15, in open_ssh_connection
client.connect(hostname, port=port, timeout=0.3, username='toto')
File "/usr/local/lib/python3.5/site-packages/paramiko/client.py", line 380, in connect
look_for_keys, gss_auth, gss_kex, gss_deleg_creds, gss_host)
File "/usr/local/lib/python3.5/site-packages/paramiko/client.py", line 603, in _auth
raise saved_exception
File "/usr/local/lib/python3.5/site-packages/paramiko/client.py", line 577, in _auth
key = pkey_class.from_private_key_file(filename, password)
File "/usr/local/lib/python3.5/site-packages/paramiko/pkey.py", line 196, in from_private_key_file
key = cls(filename=filename, password=password)
File "/usr/local/lib/python3.5/site-packages/paramiko/rsakey.py", line 45, in __init__
self._from_private_key_file(filename, password)
File "/usr/local/lib/python3.5/site-packages/paramiko/rsakey.py", line 163, in _from_private_key_file
data = self._read_private_key_file('RSA', filename, password)
File "/usr/local/lib/python3.5/site-packages/paramiko/pkey.py", line 268, in _read_private_key_file
data = self._read_private_key(tag, f, password)
File "/usr/local/lib/python3.5/site-packages/paramiko/pkey.py", line 277, in _read_private_key
raise SSHException('not a valid ' + tag + ' private key file')
paramiko.ssh_exception.SSHException: not a valid RSA private key file

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/tmp/ssh.py", line 30, in <module>
connection = open_ssh_connection(hostname, port)
File "/tmp/ssh.py", line 22, in open_ssh_connection
raise Exception('SSHException on ' + hostname)
Exception: SSHException on 192.168.2.231
@radssh

This comment has been minimized.

Copy link
Contributor

commented Sep 29, 2016

@edouard-lopez - Seems to be some confusion between server identification keys (host keys) and user identification keys in the code you posted. Host keys are typically stored in ~/.ssh/known_hosts file and have a list of public keys that have been accepted during the first ever connection to the host by the client (presumably after verifying the fingerprint by some other means). There is also a system-wide file, /etc/ssh/ssh_known_hosts that can provide a list of trusted host keys available for all users. These are the two files that would typically be passed to client.load_host_keys() and client.load_system_host_keys(), not a private key file.

When you connect to a server via SSH, the client uses the remote server's host key to verify that the server identity matches the host that was trusted from an earlier connection with the matching (and accepted) public key on file in known_hosts. If there is no matching host entry in the known_hosts file, the default behavior is to not trust the unknown host, and drop the connection. The AutoAddPolicy is used to change that default behavior to that the key will be accepted when there is no matching entry for the current host.

For user authentication, the client can use a private key instead of a password to login to the remote host, as long as the ssh server allows it, and the public key is in the ~/.ssh/authorized_keys file on the remote host. This is the stage where you would use the id_rsa.toto file as the key_filename parameter to the client.connect() call.

I'd also recommend that you test the key based authentication with ssh -i ~/.ssh/id_rsa.toto toto@host to confirm that the public key is in the right place and the file/directory permissions are correct. If the configuration setup won't work for the normal ssh client, it's not going to work under paramiko

@edouard-lopez

This comment has been minimized.

Copy link

commented Sep 29, 2016

Get it working with:

def fetch_netmask(hostname, port=22):
    private_key = os.path.expanduser('~/.ssh/id_rsa.custom')
    connection = open_ssh_connection('coaxis', hostname, port=port, key=private_key)

    get_netmask = ("ip -oneline -family inet address show | grep {}").format(hostname)
    stdin, stdout, stderr = connection.exec_command(get_netmask)
    address = parse_address(hostname, stdout)
    connection.close()

    return address


def open_ssh_connection(username, hostname, port=22, key=None):
    client = paramiko.SSHClient()
    client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
    client.connect(hostname, port=port, timeout=0.3, username=username, key_filename=key)

    return client
@WesleyBlancoYuan

This comment has been minimized.

Copy link

commented Mar 12, 2018

I am facing the same problem with paramiko 2.4.0. I am with django + apache server + CentOS 7.

I think it is something with the format, not of permission, not for key itself, because with the same key, I can:

  • connect to the server to execute command with ssh in bash. (ssh -i /path/to/private-key host)
  • connect to the server to execute command with python manage.py runserver 0.0.0.0:8080, without changing the code

But when I switch to httpd, the error occurs. If, for anything, it is a permission problem, like it was before I modified the permission and owner of the file and all its parent dir, it was another exception Permission denied.. Now it is not the case.

The exception I got was:

SSHException('not a valid OPENSSH private key file',)

The private key is like:

-----BEGIN RSA PRIVATE KEY-----
MIIEowIBAAKCAQEA0SLRpBwbygz/t7qA1HuDKVJnsf2ZhQP8p1dRF/cCYFOnOYYC
...
...
-----END RSA PRIVATE KEY-----

EDIT:

At last I solved this by

  • delete the public key in the server ending with same combination of user@host. I think paramiko sees some confusion there.
  • specify a user that exists on the remote machine in my connect(user=xxx). It is not documented and implicit; I am John in my machine, so the key generated has name of John@myhost in the key, but when I login into the remote Linux machine without a user named John, I have to connect(user=root/amy/any valid user on remote) to login in... I thought I don't have to specify a username in remote when I login with keypairs.
@WesleyBlancoYuan

This comment has been minimized.

Copy link

commented Mar 27, 2018

It happens again when I try to login to another machine, which hasn't my key pair. This is on purpose to see how paramiko works.

I think the logic of judging if a key is valid is too simply that the exception I get does not indicates the real problem. Please fix.

SSHException: not a valid OPENSSH private key file
@lionfish0

This comment has been minimized.

Copy link

commented Apr 20, 2018

Same problem as described above.
My key starts with:
-----BEGIN RSA PRIVATE KEY-----
Using 2.4.0.

Update:

OK - In my case, I found it isn't failing to identify the key - although it looks like that in the error message. It turns out it is silently trying using RSA. However, when it fails it tries a bunch of other stuff, including sshing with an id_dsa file floating around, which has an RSA key inside. So if you get this message it's probably because it's failed and is trying other things. In my case there were two mistakes: 1. I had the wrong username (passing the wrong thing to --ssh-username)
2. I was using the wrong servers! To generate the dask_servers.txt file I now run:
aws ec2 --region us-east-2 describe-instances --query "Reservations[*].Instances[*].PublicIpAddress" --output=text > dask_servers.txt
once I'd fixed those it sshed successfully (but other errors happened...).

dask-ssh --hostfile dask_servers.txt --ssh-private-key ~/.ssh/research_ohio.pem --ssh-username ubuntu

Notes:
specifically in transport.py and auth_handler.py where this 'issue' lies. Specifically, auth_handler.py maybe should be allowed to raise an exception which isn't caught? So we can see that it's tried and failed authenticating?

@sprive

This comment has been minimized.

Copy link

commented Jan 17, 2019

I solved my issue, which I will share:

changed my code from:
k = paramiko.RSAKey.from_private_key_file(keyfile)

to:
k = paramiko.DSSKey.from_private_key_file(keyfile)

I ran into this 3 years ago, once, and since forgot all about it.. and I had to revisit this code today.
FORTUNATELY, back then, I had the foresight to enter this life-saving comment:

    # Paramiko uses different classes for different key-types.
    # Hint: TESTENV uses RSAKey and TESTENV2 uses DSSKey. Paramiko won't autodetect.

("won't autodetect" is not at all a judgement. Paramiko's one of my favorite Python libraries. Thanks Jeff (and all)!).

@nickconIO

This comment has been minimized.

Copy link

commented Mar 20, 2019

Hi folks, I know this is now closed but just in case anyone else comes here searching for an answer that might not be one of the above, had a similar error message as well.

How was the key generated? @Ghost0817 used ssh-keygen command in his instance. If done without specifying the output as a PEM file, it turns out that Paramiko can raise issues. More info here:

https://github.com/paramiko/paramiko/issues/1313

Hope this helps!

@eepstein

This comment has been minimized.

Copy link

commented Apr 19, 2019

I'm running into this issue with Paramiko 2.4.2.
Using the Paramiko SSHClient class, which actually does try to auto-detect the key type, the key handling is failing on a valid key (works fine with command line ssh -i ...).

Relevant lines:

        if not two_factor:
            for key_filename in key_filenames:
                for pkey_class in (RSAKey, DSSKey, ECDSAKey, Ed25519Key): 
                    try:
                        key = self._key_from_filepath(
                            key_filename, pkey_class, passphrase
                        )
@hyperknot

This comment has been minimized.

Copy link

commented May 14, 2019

This is a problem with recent macOS, whose OpenSSL ssh-keygen has changed the default format. Now a generated SSH key starts with:

-----BEGIN OPENSSH PRIVATE KEY-----

instead of the supported

-----BEGIN RSA PRIVATE KEY-----

To generate a supported key, use the following command:

ssh-keygen -t rsa -b 4096 -C "email@email.com" -m PEM
@bitprophet

This comment has been minimized.

Copy link
Member

commented May 31, 2019

FWIW a lot of the recent comments in here should be addressed whenever I find new-feature-work time for #387, or may be handled in other feature tickets like #602.

@tchayvaz

This comment has been minimized.

Copy link

commented Aug 1, 2019

This is a problem with recent macOS, whose OpenSSL ssh-keygen has changed the default format. Now a generated SSH key starts with:

-----BEGIN OPENSSH PRIVATE KEY-----

instead of the supported

-----BEGIN RSA PRIVATE KEY-----

To generate a supported key, use the following command:

ssh-keygen -t rsa -b 4096 -C "email@email.com" -m PEM

Not just a MacOS issue: Happened to me, using Git Bash in Windows.
Just specifying output format as PEM (-m PEM) solved the problem!
Thanks!

@andreif

This comment has been minimized.

Copy link

commented Aug 12, 2019

If you can, you can convert existing private keys from OpenSSH format to PEM via https://gist.github.com/mingfang/4aba327add0807fa5e7f#gistcomment-2996291

ltalirz added a commit to ltalirz/aiida_core that referenced this issue Aug 28, 2019
AiiDA relies on paramiko for SSH connections, which - so far - only
works with SSH keys in the older "PEM" format.

Recently, the type of key generated on MacOS with
```
ssh-keygen -t rsa
```
changed from PEM to openssh, and the
[corresponding keys no longer work with paramiko](paramiko/paramiko#340 (comment))
(what is doubly confusing is that key will work just fine if you use
`ssh` directly).

Changing the line in the AiiDA documentation from
```
ssh-keygen -t rsa
```

to
```
ssh-keygen -t rsa -b 4096 -m PEM
```
solves the issue.
sphuber added a commit to aiidateam/aiida-core that referenced this issue Aug 28, 2019
AiiDA relies on paramiko for SSH connections, which - so far - only
works with SSH keys in the older "PEM" format. Recently, the type of key
generated on MacOS with:

    ssh-keygen -t rsa

changed from PEM to openssh, and the corresponding keys no longer work
with `paramiko`. See the issue for more details:

   paramiko/paramiko#340

what is doubly confusing is that key will work just fine if you use
`ssh` directly. Generating the key with the `-m` flag:

    ssh-keygen -t rsa -b 4096 -m PEM

restores paramiko compatibility and solves the issue.
d-tomerini pushed a commit to d-tomerini/aiida_core that referenced this issue Sep 30, 2019
AiiDA relies on paramiko for SSH connections, which - so far - only
works with SSH keys in the older "PEM" format. Recently, the type of key
generated on MacOS with:

    ssh-keygen -t rsa

changed from PEM to openssh, and the corresponding keys no longer work
with `paramiko`. See the issue for more details:

   paramiko/paramiko#340

what is doubly confusing is that key will work just fine if you use
`ssh` directly. Generating the key with the `-m` flag:

    ssh-keygen -t rsa -b 4096 -m PEM

restores paramiko compatibility and solves the issue.
d-tomerini pushed a commit to d-tomerini/aiida_core that referenced this issue Oct 16, 2019
AiiDA relies on paramiko for SSH connections, which - so far - only
works with SSH keys in the older "PEM" format. Recently, the type of key
generated on MacOS with:

    ssh-keygen -t rsa

changed from PEM to openssh, and the corresponding keys no longer work
with `paramiko`. See the issue for more details:

   paramiko/paramiko#340

what is doubly confusing is that key will work just fine if you use
`ssh` directly. Generating the key with the `-m` flag:

    ssh-keygen -t rsa -b 4096 -m PEM

restores paramiko compatibility and solves the issue.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
You can’t perform that action at this time.