ssh2: fix positive mpint value packing in kex

[enool]

Packed positive mpint values must be preceded with zero byte when MSB of
the value is set as per rfc4251. Taking modulo 8 on the number of bytes
in the bignum value can not determine value of MSB. Fix the MSB check to
use modulo 8 on the number of significant bits instead.

On the server side, OpenSSH was unable to unpack these mpint values used
in key exchange as they ended up negative.

sshd: ssh_dispatch_run_fatal: Connection from x.x.x.x port yyyyy: bignum is negative [preauth]

Callers of fetch_host_key() are not getting back any errors from these
failures. Consequently, host key scanning would intermittently return
only partial results.

[enool]

Hello, it has been a while since this was created. I understand that this issue may not be trivial to understand so I've created a hacky test script to explain it. I'm hoping this will help you verify the fix.

It uses docker to setup sshd in a container and takes nmap from PATH. Current nmap HEAD takes roughly few minutes to fail.

#!/bin/sh

set -euxv -o pipefail

port=2222
regex='ECDSA.*ED25519'
contimage='alpine:3.15.4'
contname='sshd-test'
contcmd="apk add openssh && ssh-keygen -A && /usr/sbin/sshd -4 -D -e"

docker run --name "$contname" -p "127.0.0.1:${port}:22" -d "$contimage" sh -c "$contcmd"
trap "docker rm -f '$contname'" EXIT ERR

# Wait for sshd to start
while ! nmap -p "$port" --script ssh-hostkey localhost | grep -z "$regex" ; do : ; done

# Run until failure - ore or more expected keys are missing
while nmap -p "$port" --script ssh-hostkey localhost | grep -z "$regex" ; do : ; done

# Print out sshd messages - bignum is negative
docker logs "$contname"

[fyodor]

Thanks for the contribution, @enool . I'm sorry for the delay, but I added this to our roadmap. We have been very busy with Npcap (hoping to do a big release on that this week) and then hopefully we can spend more time on important Nmap odds and ends like this.

[enool]

Thanks for the contribution, @enool . I'm sorry for the delay, but I added this to our roadmap. We have been very busy with Npcap (hoping to do a big release on that this week) and then hopefully we can spend more time on important Nmap odds and ends like this.

Great! Appreciate your reply.

Good luck with the release :)