kexinit - use same kex algorithms as in security_options

[manfred-kaiser]

Send modified kex algorithm list to client:

I had problems with some ssh implementations like MobaXterm (ssh-mitm/ssh-mitm#95) wich is based on SecureBlackbox.

When testing MobaXterm with paramiko versions < 2.9 it works without any problems. Due to the changes introduced in paramiko 2.10 which adds compatibility for OpenSSH 8.9 and the deprecated ssh-rsa algorithms, connections from MobaXterm wont work anymore.

After some tests with the kex algorithms I found out, that the algorithm list is modified and set for the security options. The unmodified list is sent to the client.

The list sent to the client includes "diffie-hellman-group-exchange-sha1" and "diffie-hellman-group-exchange-sha256" but not the list which is stored in the securiry options. (see https://datatracker.ietf.org/doc/html/rfc4419)

If MobaXterm wants to connect, the server raises an error because of an unsupported algorithm.

I have tested the changes with against the OpenSSH 8.9 client. Using this client the "diffie-hellman-group-exchange-sha*" algorithms are used.

Using the modified list with MobaXterm does not include the "diffie-hellman-group-exchange-sha*" algorithms and the client can connect without any problems.

Only add "ext-info-c" if not present

The second change checks if "ext-info-c" is already in the kex_algos list and only adds it, if it is not present: https://datatracker.ietf.org/doc/html/draft-ssh-ext-info-04#section-2.1

The indicator name is added without quotes, and MAY be added at any position in the name-list,...

[bskinn]

Thanks for the PR! Really appreciate the work on this.

After some tests with the kex algorithms I found out, that the algorithm list is modified and set for the security options. The unmodified list is sent to the client.

For my reference, is part of your argument here that the sending of the unmodified algo list to the client is a bug in all cases?

[bskinn]

Also, are there any reasonably simple tests that could be added around this? Definitely recognize that mocking/mimicking specific external hardware/software is not trivial.

[manfred-kaiser]

Sorry for the late response 😞

For my reference, is part of your argument here that the sending of the unmodified algo list to the client is a bug in all cases?

I have not found any information, if the modified or unmodified list should be sent to the clients.

During my tests I had no problems with the modified list.

I have tested following clients:

• OpenSSH

• PuTTY (clients like WinSCP and other which are based on PuTTY are also compatible)

• Filezilla

• MobaXterm's secureblackbox ssh implementation (this is the file transfer, the terminal is based on putty)

• Dropbear (partial - I have some other patches for Dropbear, because Dropbear is not compatible with Paramiko)

[manfred-kaiser]

Also, are there any reasonably simple tests that could be added around this? Definitely recognize that mocking/mimicking specific external hardware/software is not trivial.

Implementing tests might be not possible, because there are a lot of ssh implementations and most of them work different. The only way to test this is by using the client.

With OpenSSH, Dropbear, PuTTY this might be possible, because they are open source and you can reproduce the key exchange part.

The source of MobyXterms secureblackbox ssh implementation is not available and it's not documented why this implementation only works with the modified list.

If you have any suggestions how to implement such tests, I will try to write some.

[bskinn]

If you have any suggestions how to implement such tests, I will try to write some.

No, I had figured that substantial automated tests would be quite hard to write for this. I just wanted to raise the question to make sure it'd been considered. Thanks!

[manfred-kaiser]

I have tested 15 ssh clients (11 different ssh implementations) with the modified algorithm list and they worked without problems. 😄

You can find a list of tested clients in my project: https://github.com/ssh-mitm/ssh-mitm/blob/develop/sshmitm/data/client_info.yml

Note: Some clients are based on another client.

The list is not complete and I have to test more clients, but I have the problem that some windows clients does not run with Wine (Linux) or are commercial 😞

[manfred-kaiser]

I have updated the list of working clients. SSH-MITM uses this list for client fingerprinting and vulnerability detection, but this is also a good compatibility test for paramiko 😉

Now I have tested 17 different ssh implementations There are more than 20 tested clients, but a lot of them are based on PuTTY.
Some other clients like JuiceSSH for Android are also based on already tested implementations. Not all clients where added to the list because they used the same name (SSH banner) as the ssh implementation. In such cases only the ssh implementation is added to the list. (I will create a list of clients and the used ssh implementation)

Tested SSH implementations:

• AbsoluteTelnet: http://www.celestialsoftware.net/
• AsyncSSH: https://asyncssh.readthedocs.io/e
• TrileadSSH2: https://github.com/connectbot/sshlib
• Dropbear: https://matt.ucc.asn.au/dropbear/dropbear.html
• JSch - Java Secure Channel: https://github.com/mwiede/jsch
• libssh: https://www.libssh.org/
• libssh2: https://www.libssh2.org/
• OpenSSH: https://www.openssh.com/
• Paramiko: https://www.paramiko.org/
• RubyNetSsh: http://net-ssh.github.io/net-ssh/
• SecureBlackbox (/n software, Inc.): https://www.nsoftware.com/ipworks/ssh/
• ssh2-streams: https://github.com/mscdex/ssh2-streams'
• TeraTerm: http://ttssh2.osdn.jp/
• TotalCommander SFTP Plugin (Android): https://www.ghisler.com/androidplugins/googleplay/
• TTYEmulator: https://www.ttyemulator.com/
• WolfSSH: https://www.wolfssl.com/products/wolfssh/

PuTTY based:

• PuTTY: https://www.chiark.greenend.org.uk/~sgtatham/putty/
• FileZilla: https://filezilla-project.org/
• WinSCP: https://winscp.net
• KiTTY: http://www.9bis.net/kitty
• MobaXterm (SSH client): https://mobaxterm.mobatek.net/

Note:
I have a patch, which is necessary for Dropbear but this patch is not related to the kex algorithms!
This patch was enabled during all my tests. Perhaps some ssh implementations only work when this patch is enabled.
You can find the patch for dropbear in my repo: https://github.com/ssh-mitm/ssh-mitm/blob/d3346d86ae67887ed6cc676cb7577fab42837e35/sshmitm/workarounds/transport.py#L176-L180

[bskinn]

Thanks for all that work, and for the detailed report, @manfred-kaiser!!

[manfred-kaiser]

Will you merge this pull request or should I change some things?

[bskinn]

I don't have write access to the repo, only triage permissions, so I can't merge.

bitprophet is in the middle of a sustained modernization campaign on all of his OSS projects, focused on dropping Python 2 and old 3.x, and pretty much all of them are on a feature/bugfix freeze until that's done.

AFAIK, a rework of the key/auth internals (with #387 as kind of the focal point) is the next big thing for paramiko after that modernization, and I would expect this PR to get picked up as part of that.

Sorry for the delay, we really do appreciate all the work you've put into this!

[manfred-kaiser]

This pull request fixes some issues with Dropbear SSH and some proprietary ssh libraries like "Secure Blackbox" (a ssh component for Delphi).

The changes in this pull request are well tested against the listed ssh clients (see post above)

[manfred-kaiser]

This pull request has some code changes in the same place as the fix for terrapin-attack and might be incompatible with the latest paramiko version.

I need to update my tool SSH-MITM to the latest paramiko version (ssh-mitm/ssh-mitm#163) and add some workarounds for the mentioned clients in a previous post: #2083 (comment)

@bskinn @bitprophet Are you interested in a patch, which makes paramiko compatible with those clients and servers?

[manfred-kaiser]

I've rebased this PR on the current main branch.

The previously mentioned incompatibility with the Terrapin fix is not an issue — the kex-strict-* indicator is appended after our changes, so there's no conflict.

Summary of the two fixes:

1. When the server filters out diffie-hellman-group-exchange-* algorithms due to missing moduli, the filtered list was stored in a local variable pkex but kex_algos (the list actually sent to the client) was never updated. This caused paramiko to advertise algorithms it can't use, breaking compatibility with Dropbear and MobaXterm's SecureBlackbox implementation.

2. ext-info-c was appended unconditionally, potentially adding it twice if it was already present in preferred_kex.

Both fixes have been running as a monkey-patch workaround in SSH-MITM for several years and were tested against 17+ SSH implementations (see the client list in my previous comment above).

@bitprophet @bskinn Would you be able to take another look at this?

Note: The CI failure is pre-existing on paramiko/paramiko main and unrelated to the changes in this PR.