Add softlayer driver

[cpuguy83]

1 issue with this: Is just currently telling softlayer to setup a Ubuntu 14.04 image and run the get.docker.com script.
Both the image and the install script are customizable at the command line... but right now the default setup will not let you actually connect to the docker daemon without going and customizing the docker daemon settings (enable tcp).

Should also enable some tls stuff (or libtrust key auth stuff might be less painful).
Might be cool to get some of the libtrust key auth stuff merged into this branch so we, more easily, setup secure connections to the docker daemons created by hosts.

[nathanleclaire]

Nice!

1 issue with this: Is just currently telling softlayer to setup a Ubuntu 14.04 image and run the get.docker.com script.

This is how I've implemented the AWS driver to work out of the box as well, and I also added a --no-install option to skip that step (since it tacks a good 3 minutes or more onto provisioning). I don't know how Softlayer works, but on AWS it's possible that user can specify a custom AMI with docker pre-installed. Perhaps --no-install (and/or --no-provision to skip all provisioning) should be common flags available to all drivers?

but right now the default setup will not let you actually connect to the docker daemon without going and customizing the docker daemon settings (enable tcp).

Why not pipe an echo of the DOCKER_OPTS line to the configuration file and then restart the service? You have the GetSSHCommand available to do so.

Should also enable some tls stuff (or libtrust key auth stuff might be less painful).
Might be cool to get some of the libtrust key auth stuff merged into this branch so we, more easily, setup secure connections to the docker daemons created by hosts.

+2 Exposed raw TCP ports on the public internet == 😦 The sooner hosts gets implemented with this in mind, the better.

[cpuguy83]

I was having some issues getting SSH to work too quickly after provisioning, even though I have a wait until it's running. Will have to figure something out.

[nathanleclaire]

I was having some issues getting SSH to work too quickly after provisioning, even though I have a wait until it's running. Will have to figure something out.

I see. I was having similar issues with the AWS driver where the instance would report as "Running" before SSH was up. Therefore I poll port 22 before provisioning instead.

Take a look at this: https://github.com/nathanleclaire/docker/blob/aws-driver/hosts/drivers/aws/ec2/ec2.go#L249

Essentially what I'm doing is looping until a connection is made on port 22 using net.DialTimeout, with a timeout of 1 second and a wait between attempts of 1 second. This provides a reasonably robust way of determining whether SSH is available on the host, and it might be a nice function to have available to all drivers at some point.

[cpuguy83]

@nathanleclaire Have you had a look at the AWS/EC2 stuff done for libswarm?

[bfirsh]

How we install Docker on hosts is generally an unsolved problem. Likewise, operating systems. Installing Docker on an Ubuntu installation and then hoping we can upgrade it at some point is going to break.

I prefer the boot2docker/CoreOS model of having a read-only root FS with Docker installed on it, then swapping out the root FS when we want to upgrade.

To upgrade Docker in EC2's case, this would mean creating a new instance with a new AMI, then attaching the same EBS volume to keep persistent data. I'm not sure how this would work on Softlayer's case.

[cpuguy83]

+1 read-only rootfs.
So then what we really need to do is have a spec of what the driver's should be creating.
It may not be 100% possible in every scenario, but that's also something that we can work on with the various providers.

[bfirsh]

Agree that it's not 100% possible in each scenario, so each provider still needs its own exceptions.

EC2 should work perfectly if Docker manages its own set of Docker AMIs.

[bfirsh]

BTW – still waiting for a sales representative at Salesforce to approve my account. ;)

I trust that this works, though...

[cpuguy83]

Yeah, waiting on them to approve an account credit for all the instances I created and destroyed in testing...

[bfirsh]

I've also rebased on Docker 1.3 so this needs rebasing against my branch. (Sorry.) If you don't have the time, I'll get it rebased and merged manually.

[cpuguy83]

Rebased

[cpuguy83]

Ok, so I updated this with a nice little util for creating tls certs. Azure driver should use this isntead of shelling out to openssl (on the client side).
certs are placed in storePath/certs

Fixed issues with setting up the tcp and tls stuff on the host.

Also need to look how docker hosts can instruct the client to use the certs for the given host. Currently have to specify --tlsverify and all the cert paths (or DOCKER_CERT_PATH)

[cpuguy83]

Oh and one more comment.. Still not quite 100%. I am getting a handshake error from the daemon, which I'm not sure where/why that's happening yet.

[cpuguy83]

Ok, all problems are resolved.
Successfully sets up TLS for encryption and auth.

[bfirsh]

We're going to be using this for TLS: moby#8265

[cpuguy83]

Yay and huzzah! Will be much simpler (than having to deal with --tlsverify, certpaths, etc)

[bfirsh]

I have now rebased the host management branch on top of moby#8265 and squashed it:

https://github.com/bfirsh/docker/compare/host-management

This pull request should now be based on top of that. The driver interface hasn't changed, so it shouldn't be too hard to rebase. The main thing which has changed is that drivers are expected to set up identity auth for communication with the host. See this commit for an example of how to do so.

Full update and preview builds coming soon.

[bfirsh]

Aaand... the thing I couldn't reveal before DockerCon: Host management is now Docker Machine!

https://github.com/docker/machine

It is (almost) the same code that is in my host management branch, and the driver interface is identical. Sorry for throwing you in the wrong direction here. Would you mind copying over the code to a fork of Machine and opening a pull request there?

Thanks!

[bfirsh]

Replaced by docker/machine#51