SSH gateway support

[coderanger]

Creating this for tracking, not actually ready for merge yet.

[coderanger]

So my use case for this is a bit esoteric. I want to use kitchen-docker to get super-fast instance launches for testing, but my laptop is wimpy. So I ran docker on EC2 and accessed the API via an SSH tunnel. This works fine for the docker commands themselves, but when kitchen tries to SSH to the container to do its thing, it sees "localhost:" as the thing to connect to. So if I set the docker server itself as the SSH gateway, everything works.

[ghost]

👍 I'm in a similar situation right now and could really use this as well.

[ChrisLundquist]

I think @fnichol has been playing with a similar workflow and might have some insight.

[fnichol]

Ha, I'm getting to be in a similar situation and think this could benefit a few drivers. Will try to give this a much closer look and play tomorrow. Awesome idea!

[fnichol]

Now that concurrency support (not really parallelism) is in master, my attention goes here next!

[coderanger]

@fnichol So that code totally works, but you can see some comments for bits that need improvement. Mostly the hard part will be making wait_for_sshd deal with calling gateway_session.open and trying to connect through that.

[fnichol]

@coderanger In kitchen-docker, are you using the socket option to communicate with docker? I'm thinking it could be useful to if the docker commands could be issued over SSH as well but that would require work in each driver to support it. The kitchen-vagrant driver also comes to mind that could benefit. Just trying to get my head around it 😄

[fnichol]

@coderanger Actually, if you had a sample .kitchen.yml setup I'd love to see it!

[coderanger]

This is what I was using. I manually opened the SSH tunnel for the actual docker stuff. In a perfect world, that would be integrated into the driver and would use a native Ruby API:

---
driver_plugin: docker
driver_config:
  socket: tcp://localhost:4243
  provision_command: curl -L https://www.opscode.com/chef/install.sh | bash
  require_chef_omnibus: false
  ssh_gateway: ec2-54-211-97-92.compute-1.amazonaws.com
  username: coderanger

[adnichols]

@coderanger you mean like this? test-kitchen/kitchen-docker#27

This would be lovely combined w/ having all the API traffic passed over an ssh connection.

[coderanger]

@adnichols 🌟 🌟 🌟 🌟

[sethvargo]

@coderanger you seemed to indicated this is a 🚧. Is that still the case? This will no longer merge cleanly.

[coderanger]

Hmm, trying to remember what the issues were. I think it might have been that the wait_for_server bits didn't grok the tunnel and tried to check directly since they use raw sockets and not net::ssh.

[jtimberman]

@coderanger The merge conflict is in the gemspec. The pull request branch has safe_yaml v0.9, but HEAD has v1.0.

diff --cc test-kitchen.gemspec
index 4ea45ea,22dff77..0000000
--- a/test-kitchen.gemspec
+++ b/test-kitchen.gemspec
@@@ -23,7 -23,8 +23,12 @@@ Gem::Specification.new do |gem
    gem.add_dependency 'mixlib-shellout', '~> 1.2'
    gem.add_dependency 'net-scp',         '~> 1.1'
    gem.add_dependency 'net-ssh',         '~> 2.7'
++<<<<<<< HEAD
 +  gem.add_dependency 'safe_yaml',       '~> 1.0'
++=======
+   gem.add_dependency 'net-ssh-gateway', '~> 1.2.0'
+   gem.add_dependency 'safe_yaml',       '~> 0.9'
++>>>>>>> First pass on SSH gateway support.
    gem.add_dependency 'thor',            '~> 0.18'

    gem.add_development_dependency 'bundler',   '~> 1.3'

vs.

diff --git a/test-kitchen.gemspec b/test-kitchen.gemspec
index 4ea45ea..b132965 100644
--- a/test-kitchen.gemspec
+++ b/test-kitchen.gemspec
@@ -24,6 +24,7 @@ Gem::Specification.new do |gem|
   gem.add_dependency 'net-scp',         '~> 1.1'
   gem.add_dependency 'net-ssh',         '~> 2.7'
   gem.add_dependency 'safe_yaml',       '~> 1.0'
+  gem.add_dependency 'net-ssh-gateway', '~> 1.2.0'
   gem.add_dependency 'thor',            '~> 0.18'

   gem.add_development_dependency 'bundler',   '~> 1.3'

[steve-jansen]

+1

FYI my use case is a TDD Chef workflow using Jenkins, test-kitchen, and dynamically provisioned OpenStack nodes for kitchen using the kitchen-openstack driver. An SSH gateway avoids the need to assign an OpenStack floating IP to the dynamic OpenStack nodes created by kitchen test. Floating IPs are unfortunately a scarce resource in my shop due to a convoluted network topology.

[jpompe]

+1

[andrewjamesbrown]

@coderanger @fnichol
Is there anything outstanding on this PR that I could help with? I'm keen to have this support in place.

Thanks!

[coderanger]

Closing this because #1091 was merged. Whatever issues I was seeing must have gone away elsewhere (maybe in the SSH core rewrite).