refactor: use StrictHostKeyChecking=accept-new

[jonaseberle]

The Issue

SSH power users and security folks expect ssh to check host keys.

How This PR Solves The Issue

The option StrictHostKeyChecking=accept-new has been introduced in OpenSSH 7.6. It provides the protection against IP-spoofing without the previously necessary user interaction (in case of =yes/true) to accept unknown keys on first connect.

In case a host key has actually changed, OpenSSH gives a helpful warning including the exact command that is necessary to remove the old host key from known_hosts.

Manual Testing Instructions

Using ssh from within containers works as before. If a host key actually changed, the connection is rejected and OpenSSH shows an error message.

Automated Testing Overview

Tests have been adapted to use the web container's default SSH config.

Related Issue Link(s)

(Discussion topic) https://github.com/orgs/ddev/discussions/5029

Release/Deployment Notes

Would be great to see this for 1.22.

[github-actions]

Download the artifacts for this pull request:

• all-ddev-executables.zip
• ddev-linux-amd64.zip
• ddev-linux-arm64.zip
• ddev-macos-amd64.zip
• ddev-macos-arm64.zip
• ddev-windows-amd64-installer.zip

See Testing a PR

[rfay]

I thought this would require a change to image; the only image it affects is the ddev-webserver. I guess I should push a new version for that though.

[rfay]

Pushed new web image and updated versionconstants.go

[rfay]

TestSSHAuth failed with spoof complaint in https://buildkite.com/ddev/wsl2-docker-inside/builds/1871#018932b4-a699-4849-bc1d-b75ef6959db0

[rfay]

Rebased with new pushed/updated image.

[rfay]

This does seem to consistently fail the wsl2-docker-inside tests, I haven't studied it carefully enough to understand why.

[rfay]

I manually ran the TestSSHAuth on wsl2-docker-inside and it worked fine.

[rfay]

Hoping you're able to take a look @jonaseberle - happy to help with recreation or environments. I'll close this about 15 Aug 2023 if it's still stale, but it can also be reopened after that.

[rfay]

I'm going to close this now, but happy to reopen it and help you chase it if it helps. Thanks for the initiative to work on this.

[jonaseberle]

I'm so sorry, I haven't seen the comments earlier.

The host key of test-ssh-server seems to be changing between tests but /home/.ssh-agent/known_hosts is persisted.

I am rebasing this and will prepend all ssh ... calls with a ssh-keygen -f /home/.ssh-agent/known_hosts -R test-ssh-server to remove the offending key.

[jonaseberle]

@rfay could we reopen?
It seems the commits here are not automatically being updated with new ones on the branch:
master...jonaseberle:ddev:task/20230707_jonaseberle_use-ssh-StrictHostKeyChecking=accept-new

[rfay]

Shoot, it seems that I can't reopen because you pushed the new commits, https://gist.github.com/robertpainsi/2c42c15f1ce6dab03a0675348edd4e2c?permalink_comment_id=4323332

Per that gist maybe you can force-push it back to the commit it was on previously, 22fa611

Then I'll reopen, then you can add your commits back? Or if that doesn't work, just reopen a new PR, not too hard. Thanks for working on this!

[rfay]

Oh, now they'll let me!

[rfay]

No, I guess they won't. The button was there for a minute.

[jonaseberle]

Thanks for your patience :)
It looks like that restored the previous history. Is it possible now?

If not I'll just open a new one ;)

[rfay]

Yay, reopened!

[rfay]

Also, if you want to replace or update that test image, I'm all for it. It hasn't been touched for some years.

[rfay]

I guess https://hub.docker.com/r/ddev/test-ssh-server was pushed during the move to ddev org, so it's not that old. However, I just pushed https://hub.docker.com/layers/ddev/test-ssh-server/v1.22.2/images/sha256-4b3aced3bd6e657bf85722e3a1c9a524683d3b078995e8115c4d7893b97ff0de?context=explore if you care to use it, shouldn't be any difference.

Thanks for working on this!

[jonaseberle]

I've now rebased to current master.
It uses now test-ssh-server:1.22.2.

I solved this single conflicts in favor of the latter:

<<<<<<< HEAD
var WebTag = "20230720_ted933_doc_installation_ubuntu_apt_vulnerability" // Note that this can be overridden by make
=======
var WebTag = "20230707_jonaseberle_use-ssh-StrictHostKeyChecking" // Note that this can be overridden by make
>>>>>>> 22fa61186 (Bump to new webimage version)

[jonaseberle]

The failing Linux test ran fine for me 3x locally.

export DDEV_TEST_USE_MUTAGEN= \
  DDEV_TEST_USE_NGINX_PROXY_ROUTER= \
  DDEV_TEST_WEBSERVER_TYPE=apache-fpm \
  DDEV_TEST_NO_BIND_MOUNTS= \
  MAKE_TARGET=test \
  TESTARGS="-failfast -run '(TestDdevFullSite.*|TestDdevImportFiles|TestDdevAllDatabases|Test.*(Push|Pull))'" \
  GOTEST_SHORT='8' 

make TESTARGS="${TESTARGS}" test

@rfay do you have an idea?

[rfay]

An important bug in the ddev-webserver was fixed since yours was pushed. And a test was added for it.

I'll push an updated ddev-webserver that has this fix, and push a dummy commit to run the tests again.

Locally, you'll want to make sure you docker pull ddev/ddev-webserver:20230707_jonaseberle_use-ssh-StrictHostKeyChecking

[rfay]

Now it's back to TestAuthSSH and TestCmdAuthSSH so that at least makes sense.

[jonaseberle]

I have retriggered the failed buildkite runs and opened a topic on Discord regarding the failing MacOS Colima install.

[rfay]

Looks like you solved the test problem!

[rfay]

It seems like you've got this now, and it should work out just fine.

Let me know when you're ready for me to pull it and I'll pull and update the new image(s)

[jonaseberle]

Yes, the failed tests were from the wrong strategy in cleaning up an old host key in known_hosts.

From my side this is ready.

[rfay]

Pushed fresh image and rebased