Outdated SSH client in app image Linux package

[reini-1]

Vagrant version

Vagrant 2.2.9, installed from https://releases.hashicorp.com/vagrant/2.2.9/vagrant_2.2.9_linux_amd64.zip

Host operating system

Distributor ID: Ubuntu
Description: Ubuntu 20.04 LTS
Release: 20.04
Codename: focal

Guest operating system

Distributor ID: Debian
Description: Debian GNU/Linux 10 (buster)
Release: 10
Codename: buster

Vagrantfile

Vagrant.configure("2") do |config|
  config.vm.box = "generic/debian10"
  config.vm.network "forwarded_port", guest: 80, host: 8080
  config.vm.synced_folder "~", "/home/me", mount_options: ["ro"]
  config.vm.synced_folder "~/config", "/config"
  config.vm.synced_folder "~/scratch", "/scratch"
  config.vm.synced_folder "~/common", "/common"
  config.vm.provision "shell", inline: <<-SHELL
    echo "Europe/Vienna" > /etc/timezone
    rm /etc/localtime
    dpkg-reconfigure -f noninteractive tzdata
    dpkg-reconfigure --frontend=noninteractive locales
    update-locale
    apt-get update
    DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" dist-upgrade
    DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" purge nano
    DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" autoremove
    DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install --reinstall vim
    DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install curl jq unzip
    cat /home/me/.ssh/id_ed25519_*.pub >> .ssh/authorized_keys
    cp -va /config/home/{.bashrc*,.vim*,.rg*,.editorconfig,.profile,.tmux*,.ssh,.lnav} .
    chown -vR vagrant:vagrant .
  SHELL
end

Debug output

$ vagrant ssh
/home/xxx/.ssh/config line 10: Bad SSH2 KexAlgorithms '+diffie-hellman-group1-sha1'.

# → removed this lines from my .ssh/config
# Host xxxx
#   KexAlgorithms +diffie-hellman-group1-sha1

$ vagrant ssh
/home/xxx/.ssh/config line 15: Bad protocol 2 host key algorithms '+ssh-dss'.

# → removed this lines from my .ssh/config
# Host yyyy
#   HostkeyAlgorithms +ssh-dss

$ vagrant ssh
/home/xxx/.ssh/config: terminating, 2 bad configuration options

# → removed this lines from my .ssh/config
# Host xxxx
#   PubkeyAcceptedKeyTypes +ssh-dss

$ vagrant ssh
/home/xxx/.ssh/config: terminating, 1 bad configuration options

# → removed this lines from my .ssh/config
# Host *
#   AddKeysToAgent yes

$ vagrant ssh
Last login: Sat Jun 20 15:50:17 2020 from 10.0.2.2
vagrant@debian10:~$ logout
Connection to localhost closed.

# → now it works

When connecting with native ssh from Ubuntu, there is no problem:

$ ssh vagrant@127.0.0.1 -p 2222 -i /home/reini/Source/users/rfischer/scratch/vagrant/consul-testenv/.vagrant/machines/default/virtualbox/private_key -oUserKnownHostsFile=/dev/null -oStrictHostKeyChecking=no -oPasswordAuthentication=no -oIdentitiesOnly=yes
Warning: Permanently added '[127.0.0.1]:2222' (ECDSA) to the list of known hosts.
Last login: Sat Jun 20 16:03:00 2020 from 10.0.2.2
vagrant@debian10:~$ logout
Connection to 127.0.0.1 closed.

I noticed that vagrant has a very old buildin ssh client from 2014 that seems to have problems with these "new" options:

# vagrant build in ssh version:
$ vagrant ssh -- -V
OpenSSH_6.6.1p1 Ubuntu-2ubuntu2.13, OpenSSL 1.0.1f 6 Jan 2014

# native host os ssh version:
$ ssh -V
OpenSSH_8.2p1 Ubuntu-4, OpenSSL 1.1.1f  31 Mar 2020

Specifying the -c or --command did also not work:

$ vagrant ssh --command=/usr/bin/ssh
/home/reini/.ssh/config line 10: Bad SSH2 KexAlgorithms '+diffie-hellman-group1-sha1'.

$ vagrant ssh -c /usr/bin/ssh
/home/reini/.ssh/config line 10: Bad SSH2 KexAlgorithms '+diffie-hellman-group1-sha1'.

$ vagrant ssh --command=/usr/bin/ssh -- -V
OpenSSH_6.6.1p1 Ubuntu-2ubuntu2.13, OpenSSL 1.0.1f 6 Jan 2014

$ vagrant ssh -c /usr/bin/ssh -- -V
OpenSSH_6.6.1p1 Ubuntu-2ubuntu2.13, OpenSSL 1.0.1f 6 Jan 2014

Expected behavior

vagrant ssh works with ssh configuration options that current ssh versions support.

Actual behavior

vagrant ssh did not work.

Steps to reproduce

see debug output

Workaround

As workaround I added the following settings to my .ssh/config, added my own ssh.pub keys to the guest vagrant .ssh/authorized_keys file and use native host ssh directly:

Host vagrant* localhost 127.0.0.1
  User vagrant
  HostName localhost
  Port 2222
  UserKnownHostsFile /dev/null
  StrictHostKeyChecking no
  PasswordAuthentication no
  IdentitiesOnly yes

Now I can login like this:

$ ssh vagrant
Warning: Permanently added '[localhost]:2222' (ECDSA) to the list of known hosts.
Last login: Sat Jun 20 16:04:25 2020 from 10.0.2.2
vagrant@debian10:~$ logout
Connection to localhost closed.

Alternatively you can tell ssh not to use a config file:

$ vagrant ssh -- -F /dev/null
Last login: Sat Jun 20 16:11:41 2020 from 10.0.2.2
vagrant@debian10:~$ logout
Connection to 127.0.0.1 closed.

Final questions

Why uses vagrant such an old built in ssh client and not a current version?
Why is -c or --command ignored or do I something wrong here?
I also noticed that calling vagrant ssh or vagrant in general is very slow.

Thanks in advance.

[briancain]

Hey there @reini-1 - Vagrant doesn't ship an ssh client. It uses the ssh client that's available on your path.

Vagrant will attempt to use the local SSH client installed on the host machine. On POSIX machines, an SSH client must be installed and available on the PATH.

https://www.vagrantup.com/docs/cli/ssh.html#ssh-client-usage

brian@localghost:vagrant-sandbox % be vagrant ssh hashicorp -- -V                                                                                                                                ±[master]
OpenSSH_7.6p1 Ubuntu-4ubuntu0.3, OpenSSL 1.0.2n  7 Dec 2017
brian@localghost:vagrant-sandbox % ssh -V                                                                                                                                                                                           ±[master]
OpenSSH_7.6p1 Ubuntu-4ubuntu0.3, OpenSSL 1.0.2n  7 Dec 2017
brian@localghost:vagrant-sandbox % which ssh                                                                                                                                                                                        ±[master]
/usr/bin/ssh

It seems like you must have multiple ssh clients installed and available on your path if you are seeing two different results. I recommend making sure that the ssh binary that you prefer show up first on the path so that Vagrant uses that instead. Thanks!

[briancain]

It looks like this is apparently not true with the all in one package for linux. I recommend installing the debian package instead. This should get you the desired behavior @reini-1. I'll reopen this and apply the right label for that package.

[briancain]

And it looks like there is already an issue covering this! I'd look here for future updates: #10601

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.