New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

More informations about how to use Generic driver #1357

Closed
Oliboy50 opened this Issue Jun 13, 2015 · 29 comments

Comments

Projects
None yet
@Oliboy50

Oliboy50 commented Jun 13, 2015

I'm trying to use Docker Machine 0.3.0rc2 to create a generic remote machine on a Virtual Private Server.

Here is the OS running on this VPS:
Ubuntu 15.04 (GNU/Linux 2.6.32-042stab106.4 x86_64)

There, I configured the whole thing to be able to connect through SSH only with a user called oliboy50.

Here is what I added to my visudo file:
oliboy50 ALL=(ALL)NOPASSWD: ALL

Now, back on my host machine, here is what I get when I try to create the docker-machine:

docker-machine create \
    -d generic \ 
    --generic-ip-address 37.XXX.XX.XX \
    --generic-ssh-user oliboy50 \
    vps
Importing SSH key...
Enter passphrase for key 'C:\Users\Oliver\.docker\machine\machines\vps\id_rsa':

Then I enter my passphrase, but it keeps asking for my passphrase, each time I enter it (BTW I'm sure the passphrase is the good one and docker-machine understand it because it takes more time to give me an output back when my passphrase is correct):

Enter passphrase for key 'C:\Users\Oliver\.docker\machine\machines\vps\id_rsa':
Enter passphrase for key 'C:\Users\Oliver\.docker\machine\machines\vps\id_rsa':
Enter passphrase for key 'C:\Users\Oliver\.docker\machine\machines\vps\id_rsa':
Enter passphrase for key 'C:\Users\Oliver\.docker\machine\machines\vps\id_rsa':

Then if I exit the process, here is what I get when I list existing machines:

docker-machine ls
NAME   ACTIVE   DRIVER       STATE     URL                       SWARM
dev             virtualbox   Stopped
vps             generic      Running   tcp://37.XXX.XX.XX:2376

Then if I try to set my environment variables:

docker-machine env --shell cmd vps
open C:\Users\Oliver\.docker\machine\machines\vps\ca.pem: The file was not found.

Can someone explain me what I'm doing wrong, or what should I try to make it work?
Thanks

@Oliboy50

This comment has been minimized.

Show comment
Hide comment
@Oliboy50

Oliboy50 Jun 16, 2015

@nathanleclaire I read your whole article about 'generic' driver and swarm, but it didn't help because it doesn't really explain the requirements before how to use it...

Oliboy50 commented Jun 16, 2015

@nathanleclaire I read your whole article about 'generic' driver and swarm, but it didn't help because it doesn't really explain the requirements before how to use it...

@ilkka

This comment has been minimized.

Show comment
Hide comment
@ilkka

ilkka Jun 16, 2015

This is exactly my experience as well, just tried it by running the clients on OS X and pointing them at an Ubuntu 14.04 box.

ilkka commented Jun 16, 2015

This is exactly my experience as well, just tried it by running the clients on OS X and pointing them at an Ubuntu 14.04 box.

@ehazlett

This comment has been minimized.

Show comment
Hide comment
@ehazlett

ehazlett Jun 16, 2015

Member

Currently if you want to use password protected SSH keys you will have to enter the password each time (perhaps we should look into agent support). This is why you keep getting prompted (we have several SSH calls to provision). Actually, now that I think about it, adding that key to your ssh agent might work. I'm not sure if the -i overrides the agent or not.

Member

ehazlett commented Jun 16, 2015

Currently if you want to use password protected SSH keys you will have to enter the password each time (perhaps we should look into agent support). This is why you keep getting prompted (we have several SSH calls to provision). Actually, now that I think about it, adding that key to your ssh agent might work. I'm not sure if the -i overrides the agent or not.

@Oliboy50

This comment has been minimized.

Show comment
Hide comment
@Oliboy50

Oliboy50 Jun 16, 2015

@ehazlett Thanks.

So I followed the github stuff to add my passphrase to my ssh-agent.

And now, docker-machine create ... gives me this:

docker-machine create -d generic --generic-ip-address 37.XXX.XX.XX --generic-ssh-user oliboy50 vps
Importing SSH key...
Error creating machine: Maximum number of retries (60) exceeded
You will want to check the provider to make sure the machine and associated resources were properly removed.

Thanks for your time! (I think this may also help ppl in the same case than mine)

Oliboy50 commented Jun 16, 2015

@ehazlett Thanks.

So I followed the github stuff to add my passphrase to my ssh-agent.

And now, docker-machine create ... gives me this:

docker-machine create -d generic --generic-ip-address 37.XXX.XX.XX --generic-ssh-user oliboy50 vps
Importing SSH key...
Error creating machine: Maximum number of retries (60) exceeded
You will want to check the provider to make sure the machine and associated resources were properly removed.

Thanks for your time! (I think this may also help ppl in the same case than mine)

@clnperez

This comment has been minimized.

Show comment
Hide comment
@clnperez

clnperez Jun 16, 2015

Contributor

Hey @Oliboy50, try running the create with a -D to see what's failing (docker-machine -D create -d...).

Contributor

clnperez commented Jun 16, 2015

Hey @Oliboy50, try running the create with a -D to see what's failing (docker-machine -D create -d...).

@ehazlett

This comment has been minimized.

Show comment
Hide comment
@ehazlett

ehazlett Jun 16, 2015

Member

@clnperez +1 -- I will do some local testing. I think it would be good to support the SSH agent in any case.

Member

ehazlett commented Jun 16, 2015

@clnperez +1 -- I will do some local testing. I think it would be good to support the SSH agent in any case.

@Oliboy50

This comment has been minimized.

Show comment
Hide comment
@Oliboy50

Oliboy50 Jun 16, 2015

@clnperez Thanks for the tip ;)

For some reasons now, I can't use the ssh-agent correctly again (or docker-machine just don't want to talk with it anymore =S)...

Here is my output with -D options anyway:

docker-machine -D create -d generic --generic-ip-address 37.XXX.XX.XX --generic-ssh-user oliboy50 vps
Importing SSH key...
IP: 37.XXX.XX.XX
Getting to WaitForSSH function...
Testing TCP connection to: 37.XXX.XX.XX:22
Using SSH client type: external
About to run SSH command:
exit 0
&{C:\msysgit\bin\ssh.exe [C:\msysgit\bin\ssh.exe -o PasswordAuthentication=no -o IdentitiesOnly=yes -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=quiet -o ConnectionAttempts=3 -o ConnectTimeout=10 -i C:\Users\Oliver\.docker\machine\machines\vps\id_rsa -p 22 oliboy50@37.XXX.XX.XX exit 0] []  <nil> <nil> <nil> [] <nil> <nil> <nil> ?reflect.Value? false [] [] [] [] <nil>}
Enter passphrase for key 'C:\Users\Oliver\.docker\machine\machines\vps\id_rsa': [I typed my passphrase here]
SSH cmd err, output: exit status 255:
Error getting ssh command 'exit 0' : exit status 255
Getting to WaitForSSH function...
... etc...

BTW, it gives me the same output if I type my correct passphrase or if I don't type any passphrase...

Thanks

Oliboy50 commented Jun 16, 2015

@clnperez Thanks for the tip ;)

For some reasons now, I can't use the ssh-agent correctly again (or docker-machine just don't want to talk with it anymore =S)...

Here is my output with -D options anyway:

docker-machine -D create -d generic --generic-ip-address 37.XXX.XX.XX --generic-ssh-user oliboy50 vps
Importing SSH key...
IP: 37.XXX.XX.XX
Getting to WaitForSSH function...
Testing TCP connection to: 37.XXX.XX.XX:22
Using SSH client type: external
About to run SSH command:
exit 0
&{C:\msysgit\bin\ssh.exe [C:\msysgit\bin\ssh.exe -o PasswordAuthentication=no -o IdentitiesOnly=yes -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=quiet -o ConnectionAttempts=3 -o ConnectTimeout=10 -i C:\Users\Oliver\.docker\machine\machines\vps\id_rsa -p 22 oliboy50@37.XXX.XX.XX exit 0] []  <nil> <nil> <nil> [] <nil> <nil> <nil> ?reflect.Value? false [] [] [] [] <nil>}
Enter passphrase for key 'C:\Users\Oliver\.docker\machine\machines\vps\id_rsa': [I typed my passphrase here]
SSH cmd err, output: exit status 255:
Error getting ssh command 'exit 0' : exit status 255
Getting to WaitForSSH function...
... etc...

BTW, it gives me the same output if I type my correct passphrase or if I don't type any passphrase...

Thanks

@Oliboy50

This comment has been minimized.

Show comment
Hide comment
@Oliboy50

Oliboy50 Jun 16, 2015

After some tests, here is what work for me to connect to my remote machine:

C:\msysgit\bin\ssh.exe -o PasswordAuthentication=yes -o IdentitiesOnly=yes -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=quiet -o ConnectionAttempts=3 -o ConnectTimeout=10 -i C:\Users\Oliver\.docker\machine\machines\vps\id_rsa -p 22 oliboy50@37.XXX.XX.XX
Enter passphrase for key 'C:\Users\Oliver\.docker\machine\machines\vps\id_rsa': [I typed my passphrase here]
oliboy50@37.XXX.XX.XX's password: [I typed my user password here]

(note the -o PasswordAuthentication=yes)

So I'm pretty sure that the problem comes from the SSH program needs my remote user password to establish a connection...

So, what kind of settings should I do, on my remote machine, to prevent SSH for asking my user password? As I said in my first comment, I already added this to my visudo stuff: oliboy50 ALL=(ALL)NOPASSWD: ALL. But maybe I'm doing it bad =/

Does anyone know the proper way to do that?

P.S: It seems that I have to add C:\Users\Oliver\.docker\machine\machines\vps\id_rsa to my ssh-agent instead of C:\Users\Oliver\.ssh\id_rsa if I want to persist my passphrase. So it first needs to dupplicate the SSH key at the docker machines location before we can add it to the SSH agent now =/

Oliboy50 commented Jun 16, 2015

After some tests, here is what work for me to connect to my remote machine:

C:\msysgit\bin\ssh.exe -o PasswordAuthentication=yes -o IdentitiesOnly=yes -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=quiet -o ConnectionAttempts=3 -o ConnectTimeout=10 -i C:\Users\Oliver\.docker\machine\machines\vps\id_rsa -p 22 oliboy50@37.XXX.XX.XX
Enter passphrase for key 'C:\Users\Oliver\.docker\machine\machines\vps\id_rsa': [I typed my passphrase here]
oliboy50@37.XXX.XX.XX's password: [I typed my user password here]

(note the -o PasswordAuthentication=yes)

So I'm pretty sure that the problem comes from the SSH program needs my remote user password to establish a connection...

So, what kind of settings should I do, on my remote machine, to prevent SSH for asking my user password? As I said in my first comment, I already added this to my visudo stuff: oliboy50 ALL=(ALL)NOPASSWD: ALL. But maybe I'm doing it bad =/

Does anyone know the proper way to do that?

P.S: It seems that I have to add C:\Users\Oliver\.docker\machine\machines\vps\id_rsa to my ssh-agent instead of C:\Users\Oliver\.ssh\id_rsa if I want to persist my passphrase. So it first needs to dupplicate the SSH key at the docker machines location before we can add it to the SSH agent now =/

@nathanleclaire

This comment has been minimized.

Show comment
Hide comment
@nathanleclaire

nathanleclaire Jun 16, 2015

Contributor

Ah, yes, we turned the PasswordAuthentication=no option on to fix some issues with CentOS provisioning, but perhaps we need to revisit that kludgey fix and see if we can come up with something more elegant, as it clearly has side effects like mentioned here.

Contributor

nathanleclaire commented Jun 16, 2015

Ah, yes, we turned the PasswordAuthentication=no option on to fix some issues with CentOS provisioning, but perhaps we need to revisit that kludgey fix and see if we can come up with something more elegant, as it clearly has side effects like mentioned here.

@marcellodesales

This comment has been minimized.

Show comment
Hide comment
@marcellodesales

marcellodesales Jun 25, 2015

I have Docker 1.6.2 running in a RHEL 7 host and I'm trying to connect to it using docker-machine and it fails with the regular setup...

~/dev/github/intuit/servicesplatform-node/isp on ⭠ master! ⌚ 1:24:41
$ sudo  docker-machine -D create -d generic --generic-ip-address 10.132.53.245 --generic-ssh-user mdesales builder-1
Creating CA: /home/mdesales/.docker/machine/certs/ca.pem
Creating client certificate: /home/mdesales/.docker/machine/certs/cert.pem
Importing SSH key...
IP: 10.132.53.245
Getting to WaitForSSH function...
Testing TCP connection to: 10.132.53.245:22
Using SSH client type: external
About to run SSH command:
exit 0
&{/usr/bin/ssh [/usr/bin/ssh -o PasswordAuthentication=no -o IdentitiesOnly=yes -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=quiet -o ConnectionAttempts=3 -o ConnectTimeout=10 -i /home/mdesales/.docker/machine/machines/builder-1/id_rsa -p 22 mdesales@10.132.53.245 exit 0] []  <nil> <nil> <nil> [] <nil> <nil> <nil> ?reflect.Value? false [] [] [] [] <nil>}
SSH cmd err, output: exit status 255: 
Error getting ssh command 'exit 0' : exit status 255
^C%  

Lots of attempts are made and nothing happens... What else should I do to get it to work?

[mdesales@pppdc9prd8wu ~]$ docker --version
Docker version 1.6.2, build 7c8fca2

[mdesales@pppdc9prd8wu ~]$ sudo docker info
Containers: 1
Images: 10
Storage Driver: devicemapper
 Pool Name: docker-253:0-4374531-pool
 Pool Blocksize: 65.54 kB
 Backing Filesystem: extfs
 Data file: /dev/loop0
 Metadata file: /dev/loop1
 Data Space Used: 407.4 MB
 Data Space Total: 107.4 GB
 Data Space Available: 105.1 GB
 Metadata Space Used: 1.04 MB
 Metadata Space Total: 2.147 GB
 Metadata Space Available: 2.146 GB
 Udev Sync Supported: false
 Data loop file: /app_local/var-lib-docker/devicemapper/devicemapper/data
 Metadata loop file: /app_local/var-lib-docker/devicemapper/devicemapper/metadata
 Library Version: 1.02.82-git (2013-10-04)
Execution Driver: native-0.2
Kernel Version: 3.10.0-229.4.2.el7.x86_64
Operating System: Red Hat Enterprise Linux
CPUs: 2
Total Memory: 7.641 GiB
Name: pppdc9prd8wu
ID: 3M2F:QYY7:Z5DI:YTVI:RAV4:SHPM:C3RC:CWIY:FHFA:ZYAS:SNHG:CMTY

thanks!

marcellodesales commented Jun 25, 2015

I have Docker 1.6.2 running in a RHEL 7 host and I'm trying to connect to it using docker-machine and it fails with the regular setup...

~/dev/github/intuit/servicesplatform-node/isp on ⭠ master! ⌚ 1:24:41
$ sudo  docker-machine -D create -d generic --generic-ip-address 10.132.53.245 --generic-ssh-user mdesales builder-1
Creating CA: /home/mdesales/.docker/machine/certs/ca.pem
Creating client certificate: /home/mdesales/.docker/machine/certs/cert.pem
Importing SSH key...
IP: 10.132.53.245
Getting to WaitForSSH function...
Testing TCP connection to: 10.132.53.245:22
Using SSH client type: external
About to run SSH command:
exit 0
&{/usr/bin/ssh [/usr/bin/ssh -o PasswordAuthentication=no -o IdentitiesOnly=yes -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=quiet -o ConnectionAttempts=3 -o ConnectTimeout=10 -i /home/mdesales/.docker/machine/machines/builder-1/id_rsa -p 22 mdesales@10.132.53.245 exit 0] []  <nil> <nil> <nil> [] <nil> <nil> <nil> ?reflect.Value? false [] [] [] [] <nil>}
SSH cmd err, output: exit status 255: 
Error getting ssh command 'exit 0' : exit status 255
^C%  

Lots of attempts are made and nothing happens... What else should I do to get it to work?

[mdesales@pppdc9prd8wu ~]$ docker --version
Docker version 1.6.2, build 7c8fca2

[mdesales@pppdc9prd8wu ~]$ sudo docker info
Containers: 1
Images: 10
Storage Driver: devicemapper
 Pool Name: docker-253:0-4374531-pool
 Pool Blocksize: 65.54 kB
 Backing Filesystem: extfs
 Data file: /dev/loop0
 Metadata file: /dev/loop1
 Data Space Used: 407.4 MB
 Data Space Total: 107.4 GB
 Data Space Available: 105.1 GB
 Metadata Space Used: 1.04 MB
 Metadata Space Total: 2.147 GB
 Metadata Space Available: 2.146 GB
 Udev Sync Supported: false
 Data loop file: /app_local/var-lib-docker/devicemapper/devicemapper/data
 Metadata loop file: /app_local/var-lib-docker/devicemapper/devicemapper/metadata
 Library Version: 1.02.82-git (2013-10-04)
Execution Driver: native-0.2
Kernel Version: 3.10.0-229.4.2.el7.x86_64
Operating System: Red Hat Enterprise Linux
CPUs: 2
Total Memory: 7.641 GiB
Name: pppdc9prd8wu
ID: 3M2F:QYY7:Z5DI:YTVI:RAV4:SHPM:C3RC:CWIY:FHFA:ZYAS:SNHG:CMTY

thanks!

@arthuroliveira

This comment has been minimized.

Show comment
Hide comment
@arthuroliveira

arthuroliveira Jun 29, 2015

no able to add an existing Host to docker-machine

$ docker-machine create -d generic --generic-ip-address SERVER_IP --generic-ssh-user root --generic-ssh-key /.ssh/id_rsa --generic-ssh-port 22 SERVER_NAME
Importing SSH key... (it keeps asking: Enter your password for the SSH key “id_rsa”)
^C

$ docker-machine env SERVER_NAME
open /Users/USER/.docker/machine/machines/SERVER_NAME/ca.pem: no such file or directory

arthuroliveira commented Jun 29, 2015

no able to add an existing Host to docker-machine

$ docker-machine create -d generic --generic-ip-address SERVER_IP --generic-ssh-user root --generic-ssh-key /.ssh/id_rsa --generic-ssh-port 22 SERVER_NAME
Importing SSH key... (it keeps asking: Enter your password for the SSH key “id_rsa”)
^C

$ docker-machine env SERVER_NAME
open /Users/USER/.docker/machine/machines/SERVER_NAME/ca.pem: no such file or directory

@matglas

This comment has been minimized.

Show comment
Hide comment
@matglas

matglas Jul 16, 2015

Same problem here. Using version docker-machine version 0.3.0 (0a251fe). Running docker-machine create --driver generic --generic-ip-address 192.168.1.117 docker-host-1 with a key without passphrase.

Getting:

Importing SSH key...
Error creating machine: Too many retries.  Last error: Maximum number of retries (60) exceeded
You will want to check the provider to make sure the machine and associated resources were properly removed.

matglas commented Jul 16, 2015

Same problem here. Using version docker-machine version 0.3.0 (0a251fe). Running docker-machine create --driver generic --generic-ip-address 192.168.1.117 docker-host-1 with a key without passphrase.

Getting:

Importing SSH key...
Error creating machine: Too many retries.  Last error: Maximum number of retries (60) exceeded
You will want to check the provider to make sure the machine and associated resources were properly removed.
@jottr

This comment has been minimized.

Show comment
Hide comment
@jottr

jottr Aug 3, 2015

I am seeing this issue as well on OSX with ssh-agent running, and a password protected ssh-key:
docker-machine create -d generic --generic-ssh-user root --generic-ip-address XXX.XXX.X.XXX foo

I've added the key to ssh-agent via ssh-add id_rsa, but the create command results in the ssh process asking for the passphrase several times in a row. Ticking the option to save the passphrase in the keychain does not help.
I really like the proposal in #1374.

jottr commented Aug 3, 2015

I am seeing this issue as well on OSX with ssh-agent running, and a password protected ssh-key:
docker-machine create -d generic --generic-ssh-user root --generic-ip-address XXX.XXX.X.XXX foo

I've added the key to ssh-agent via ssh-add id_rsa, but the create command results in the ssh process asking for the passphrase several times in a row. Ticking the option to save the passphrase in the keychain does not help.
I really like the proposal in #1374.

@jasonben

This comment has been minimized.

Show comment
Hide comment
@jasonben

jasonben Aug 24, 2015

Keeps prompting for passphrase when newly created key does not contain a passphrase.
+1 for more documentation

docker-machine -v
docker-machine version 0.4.1 (e2c88d6)

docker-machine create -d generic \
  --generic-ssh-user XXX \
  --generic-ssh-key /Users/XXX/.ssh/test_for_dm \
  --generic-ip-address XXX \
  --generic-ssh-port XXX \
  my-host

jasonben commented Aug 24, 2015

Keeps prompting for passphrase when newly created key does not contain a passphrase.
+1 for more documentation

docker-machine -v
docker-machine version 0.4.1 (e2c88d6)

docker-machine create -d generic \
  --generic-ssh-user XXX \
  --generic-ssh-key /Users/XXX/.ssh/test_for_dm \
  --generic-ip-address XXX \
  --generic-ssh-port XXX \
  my-host
@colinrgodsey

This comment has been minimized.

Show comment
Hide comment
@colinrgodsey

colinrgodsey Oct 27, 2015

so, this is a terrible work around but... keep entering your password.

its not asking again to retry the password, its asking again cause its running another ssh command.

EDIT: no way to trick ssh-agent. just keep entering the pass until it stops asking

docker-machine create --driver generic --generic-ip-address docker2 docker2
Importing SSH key...
Enter passphrase for key '/home/colin/.docker/machine/machines/docker2/id_rsa': 
Enter passphrase for key '/home/colin/.docker/machine/machines/docker2/id_rsa': 
Enter passphrase for key '/home/colin/.docker/machine/machines/docker2/id_rsa': 
Enter passphrase for key '/home/colin/.docker/machine/machines/docker2/id_rsa': 
Enter passphrase for key '/home/colin/.docker/machine/machines/docker2/id_rsa': 
Enter passphrase for key '/home/colin/.docker/machine/machines/docker2/id_rsa': 
Enter passphrase for key '/home/colin/.docker/machine/machines/docker2/id_rsa': 
Enter passphrase for key '/home/colin/.docker/machine/machines/docker2/id_rsa': 
Enter passphrase for key '/home/colin/.docker/machine/machines/docker2/id_rsa': 
Enter passphrase for key '/home/colin/.docker/machine/machines/docker2/id_rsa': 
Enter passphrase for key '/home/colin/.docker/machine/machines/docker2/id_rsa': 
Enter passphrase for key '/home/colin/.docker/machine/machines/docker2/id_rsa': 
Enter passphrase for key '/home/colin/.docker/machine/machines/docker2/id_rsa': 
Enter passphrase for key '/home/colin/.docker/machine/machines/docker2/id_rsa': 
Enter passphrase for key '/home/colin/.docker/machine/machines/docker2/id_rsa': 
To see how to connect Docker to this machine, run: docker-machine env docker2

colinrgodsey commented Oct 27, 2015

so, this is a terrible work around but... keep entering your password.

its not asking again to retry the password, its asking again cause its running another ssh command.

EDIT: no way to trick ssh-agent. just keep entering the pass until it stops asking

docker-machine create --driver generic --generic-ip-address docker2 docker2
Importing SSH key...
Enter passphrase for key '/home/colin/.docker/machine/machines/docker2/id_rsa': 
Enter passphrase for key '/home/colin/.docker/machine/machines/docker2/id_rsa': 
Enter passphrase for key '/home/colin/.docker/machine/machines/docker2/id_rsa': 
Enter passphrase for key '/home/colin/.docker/machine/machines/docker2/id_rsa': 
Enter passphrase for key '/home/colin/.docker/machine/machines/docker2/id_rsa': 
Enter passphrase for key '/home/colin/.docker/machine/machines/docker2/id_rsa': 
Enter passphrase for key '/home/colin/.docker/machine/machines/docker2/id_rsa': 
Enter passphrase for key '/home/colin/.docker/machine/machines/docker2/id_rsa': 
Enter passphrase for key '/home/colin/.docker/machine/machines/docker2/id_rsa': 
Enter passphrase for key '/home/colin/.docker/machine/machines/docker2/id_rsa': 
Enter passphrase for key '/home/colin/.docker/machine/machines/docker2/id_rsa': 
Enter passphrase for key '/home/colin/.docker/machine/machines/docker2/id_rsa': 
Enter passphrase for key '/home/colin/.docker/machine/machines/docker2/id_rsa': 
Enter passphrase for key '/home/colin/.docker/machine/machines/docker2/id_rsa': 
Enter passphrase for key '/home/colin/.docker/machine/machines/docker2/id_rsa': 
To see how to connect Docker to this machine, run: docker-machine env docker2
@thomasmodeneis

This comment has been minimized.

Show comment
Hide comment
@thomasmodeneis

thomasmodeneis Nov 21, 2015

same here, +1 for more detailed docs

thomasmodeneis commented Nov 21, 2015

same here, +1 for more detailed docs

@jaycyb

This comment has been minimized.

Show comment
Hide comment
@jaycyb

jaycyb Dec 10, 2015

same here and this worked for me on Mint/Ubuntu:
SSH_AUTH_SOCK=0 docker-machine create -d generic --generic-ip-address ...

a bug in gnome-keyring produces similar symptoms

https://chrisjean.com/ubuntu-ssh-fix-for-agent-admitted-failure-to-sign-using-the-key/
bug1
bug2

jaycyb commented Dec 10, 2015

same here and this worked for me on Mint/Ubuntu:
SSH_AUTH_SOCK=0 docker-machine create -d generic --generic-ip-address ...

a bug in gnome-keyring produces similar symptoms

https://chrisjean.com/ubuntu-ssh-fix-for-agent-admitted-failure-to-sign-using-the-key/
bug1
bug2

@marcellodesales

This comment has been minimized.

Show comment
Hide comment
@marcellodesales

marcellodesales Jan 1, 2016

I was able to make progress... Here's a summary of what I did today:

  • Sent my SSH keys to the host that's running Docker under my user
    • Including the ~/.ssh/authorized_keys including my key from my computer
  • Added my user to the group docker
    • Now my user can do docker ps
  • I tried creating the docker machine with my keys.

The result is that docker-machine now can get to my docker host, identify the operating system, etc... However, it gets stuck at the Provisioning with redhat..., since my host is running RHEL7, and nothing happens there...

$ docker-machine create
 --driver generic --generic-ip-address discovery.corp.bus.net 
--generic-ssh-user mdesales --generic-ssh-key ~/.ssh/id_rsa.pub consul-ui 
Running pre-create checks...
Creating machine...
(consul-ui) Importing SSH key...
Waiting for machine to be running, this may take a few minutes...
Machine is running, waiting for SSH to be available...
Detecting operating system of created instance...
Detecting the provisioner...
Provisioning with redhat...
^C

After I exited the process, I could see that docker-machine is trying to use the keys for the server. But I'm not sure if the failure during provisioning did not create those keys...

$ docker-machine ls
NAME                   ACTIVE   DRIVER    STATE     URL   
                                    SWARM   DOCKER    ERRORS
consul-ui              -        generic   Running   tcp://discovery.corp.bus.net:2376    
          Unknown   Unable to query docker version: Unable to read TLS config: open
 /home/mdesales/.docker/machine/machines/consul-ui/server.pem: no such file or directory

Questions

  • Are we supposed to place the server certificates there?
  • Is it a requirement that the existing machine use LTS in the process? It currently does NOT have any setup.

Running in Debug Mode

$ docker-machine -D create --driver generic 
--generic-ip-address discovery.corp.bus.net 
--generic-ssh-user mdesales --generic-ssh-key ~/.ssh/id_rsa.pub consul-ui 
Docker Machine Version:  0.5.4, build 6643d0e
Found binary path at /usr/bin/docker-machine
Launching plugin server for driver generic
Plugin server listening at address 127.0.0.1:56463
() Calling .GetVersion
Using API Version  1
() Calling .SetConfigRaw
() Calling .GetMachineName
(flag-lookup) Calling .GetCreateFlags
Making call to close connection to plugin binary
Making call to close driver server
(flag-lookup) Calling .Close
Successfully made call to close driver server
Found binary path at /usr/bin/docker-machine
Launching plugin server for driver generic
Plugin server listening at address 127.0.0.1:35153
() Calling .GetVersion
Using API Version  1
() Calling .SetConfigRaw
() Calling .GetMachineName
(consul-ui) Calling .GetMachineName
(consul-ui) Calling .DriverName
(consul-ui) Calling .GetCreateFlags
(consul-ui) Calling .SetConfigFromFlags
Running pre-create checks...
(consul-ui) Calling .PreCreateCheck
(consul-ui) Calling .GetConfigRaw
Creating machine...
(consul-ui) Calling .Create
(consul-ui) Importing SSH key...
(consul-ui) Calling .GetConfigRaw
(consul-ui) DBG | IP: discovery.corp.bus.net
(consul-ui) Calling .DriverName
Waiting for machine to be running, this may take a few minutes...
(consul-ui) Calling .GetState
Machine is running, waiting for SSH to be available...
Getting to WaitForSSH function...
(consul-ui) Calling .GetSSHHostname
(consul-ui) Calling .GetSSHPort
(consul-ui) Calling .GetSSHKeyPath
(consul-ui) Calling .GetSSHKeyPath
(consul-ui) Calling .GetSSHUsername
Using SSH client type: external
About to run SSH command:
exit 0
SSH cmd err, output: <nil>: 
Detecting operating system of created instance...
Detecting the provisioner...
(consul-ui) Calling .GetSSHHostname
(consul-ui) Calling .GetSSHPort
(consul-ui) Calling .GetSSHKeyPath
(consul-ui) Calling .GetSSHKeyPath
(consul-ui) Calling .GetSSHUsername
Using SSH client type: external
About to run SSH command:
cat /etc/os-release
SSH cmd err, output: <nil>: NAME="Red Hat Enterprise Linux Server"
VERSION="7.1 (Maipo)"
ID="rhel"
ID_LIKE="fedora"
VERSION_ID="7.1"
PRETTY_NAME="Red Hat Enterprise Linux Server 7.1 (Maipo)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:redhat:enterprise_linux:7.1:GA:server"
HOME_URL="https://www.redhat.com/"
BUG_REPORT_URL="https://bugzilla.redhat.com/"

REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 7"
REDHAT_BUGZILLA_PRODUCT_VERSION=7.1
REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
REDHAT_SUPPORT_PRODUCT_VERSION="7.1"

Couldn't set key CPE_NAME, no corresponding struct field found
Couldn't set key , no corresponding struct field found
Couldn't set key REDHAT_BUGZILLA_PRODUCT, no corresponding struct field found
Couldn't set key REDHAT_BUGZILLA_PRODUCT_VERSION, no corresponding struct field found
Couldn't set key REDHAT_SUPPORT_PRODUCT, no corresponding struct field found
Couldn't set key REDHAT_SUPPORT_PRODUCT_VERSION, no corresponding struct field found
found compatible host: rhel
Provisioning with redhat...
(consul-ui) Calling .GetMachineName
(consul-ui) Calling .GetSSHHostname
(consul-ui) Calling .GetSSHPort
(consul-ui) Calling .GetSSHKeyPath
(consul-ui) Calling .GetSSHKeyPath
(consul-ui) Calling .GetSSHUsername
Using SSH client type: external

marcellodesales commented Jan 1, 2016

I was able to make progress... Here's a summary of what I did today:

  • Sent my SSH keys to the host that's running Docker under my user
    • Including the ~/.ssh/authorized_keys including my key from my computer
  • Added my user to the group docker
    • Now my user can do docker ps
  • I tried creating the docker machine with my keys.

The result is that docker-machine now can get to my docker host, identify the operating system, etc... However, it gets stuck at the Provisioning with redhat..., since my host is running RHEL7, and nothing happens there...

$ docker-machine create
 --driver generic --generic-ip-address discovery.corp.bus.net 
--generic-ssh-user mdesales --generic-ssh-key ~/.ssh/id_rsa.pub consul-ui 
Running pre-create checks...
Creating machine...
(consul-ui) Importing SSH key...
Waiting for machine to be running, this may take a few minutes...
Machine is running, waiting for SSH to be available...
Detecting operating system of created instance...
Detecting the provisioner...
Provisioning with redhat...
^C

After I exited the process, I could see that docker-machine is trying to use the keys for the server. But I'm not sure if the failure during provisioning did not create those keys...

$ docker-machine ls
NAME                   ACTIVE   DRIVER    STATE     URL   
                                    SWARM   DOCKER    ERRORS
consul-ui              -        generic   Running   tcp://discovery.corp.bus.net:2376    
          Unknown   Unable to query docker version: Unable to read TLS config: open
 /home/mdesales/.docker/machine/machines/consul-ui/server.pem: no such file or directory

Questions

  • Are we supposed to place the server certificates there?
  • Is it a requirement that the existing machine use LTS in the process? It currently does NOT have any setup.

Running in Debug Mode

$ docker-machine -D create --driver generic 
--generic-ip-address discovery.corp.bus.net 
--generic-ssh-user mdesales --generic-ssh-key ~/.ssh/id_rsa.pub consul-ui 
Docker Machine Version:  0.5.4, build 6643d0e
Found binary path at /usr/bin/docker-machine
Launching plugin server for driver generic
Plugin server listening at address 127.0.0.1:56463
() Calling .GetVersion
Using API Version  1
() Calling .SetConfigRaw
() Calling .GetMachineName
(flag-lookup) Calling .GetCreateFlags
Making call to close connection to plugin binary
Making call to close driver server
(flag-lookup) Calling .Close
Successfully made call to close driver server
Found binary path at /usr/bin/docker-machine
Launching plugin server for driver generic
Plugin server listening at address 127.0.0.1:35153
() Calling .GetVersion
Using API Version  1
() Calling .SetConfigRaw
() Calling .GetMachineName
(consul-ui) Calling .GetMachineName
(consul-ui) Calling .DriverName
(consul-ui) Calling .GetCreateFlags
(consul-ui) Calling .SetConfigFromFlags
Running pre-create checks...
(consul-ui) Calling .PreCreateCheck
(consul-ui) Calling .GetConfigRaw
Creating machine...
(consul-ui) Calling .Create
(consul-ui) Importing SSH key...
(consul-ui) Calling .GetConfigRaw
(consul-ui) DBG | IP: discovery.corp.bus.net
(consul-ui) Calling .DriverName
Waiting for machine to be running, this may take a few minutes...
(consul-ui) Calling .GetState
Machine is running, waiting for SSH to be available...
Getting to WaitForSSH function...
(consul-ui) Calling .GetSSHHostname
(consul-ui) Calling .GetSSHPort
(consul-ui) Calling .GetSSHKeyPath
(consul-ui) Calling .GetSSHKeyPath
(consul-ui) Calling .GetSSHUsername
Using SSH client type: external
About to run SSH command:
exit 0
SSH cmd err, output: <nil>: 
Detecting operating system of created instance...
Detecting the provisioner...
(consul-ui) Calling .GetSSHHostname
(consul-ui) Calling .GetSSHPort
(consul-ui) Calling .GetSSHKeyPath
(consul-ui) Calling .GetSSHKeyPath
(consul-ui) Calling .GetSSHUsername
Using SSH client type: external
About to run SSH command:
cat /etc/os-release
SSH cmd err, output: <nil>: NAME="Red Hat Enterprise Linux Server"
VERSION="7.1 (Maipo)"
ID="rhel"
ID_LIKE="fedora"
VERSION_ID="7.1"
PRETTY_NAME="Red Hat Enterprise Linux Server 7.1 (Maipo)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:redhat:enterprise_linux:7.1:GA:server"
HOME_URL="https://www.redhat.com/"
BUG_REPORT_URL="https://bugzilla.redhat.com/"

REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 7"
REDHAT_BUGZILLA_PRODUCT_VERSION=7.1
REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
REDHAT_SUPPORT_PRODUCT_VERSION="7.1"

Couldn't set key CPE_NAME, no corresponding struct field found
Couldn't set key , no corresponding struct field found
Couldn't set key REDHAT_BUGZILLA_PRODUCT, no corresponding struct field found
Couldn't set key REDHAT_BUGZILLA_PRODUCT_VERSION, no corresponding struct field found
Couldn't set key REDHAT_SUPPORT_PRODUCT, no corresponding struct field found
Couldn't set key REDHAT_SUPPORT_PRODUCT_VERSION, no corresponding struct field found
found compatible host: rhel
Provisioning with redhat...
(consul-ui) Calling .GetMachineName
(consul-ui) Calling .GetSSHHostname
(consul-ui) Calling .GetSSHPort
(consul-ui) Calling .GetSSHKeyPath
(consul-ui) Calling .GetSSHKeyPath
(consul-ui) Calling .GetSSHUsername
Using SSH client type: external
@mefellows

This comment has been minimized.

Show comment
Hide comment
@mefellows

mefellows Jan 7, 2016

I had this issue also, and was able to resolve it by creating an SSH key for the user running docker-machine. e.g. to create a passwordless key:

ssh-keygen -f ~/.ssh/id_rsa -N ""

mefellows commented Jan 7, 2016

I had this issue also, and was able to resolve it by creating an SSH key for the user running docker-machine. e.g. to create a passwordless key:

ssh-keygen -f ~/.ssh/id_rsa -N ""
@marcellodesales

This comment has been minimized.

Show comment
Hide comment
@marcellodesales

marcellodesales Jan 9, 2016

@mefellows So, can you describe the process a bit more? Is it something like:

  1. Create password-less SSH keys
  2. Push to the host we wanna control
  3. Copy them to the expected location

Is that what we need?

marcellodesales commented Jan 9, 2016

@mefellows So, can you describe the process a bit more? Is it something like:

  1. Create password-less SSH keys
  2. Push to the host we wanna control
  3. Copy them to the expected location

Is that what we need?

@mefellows

This comment has been minimized.

Show comment
Hide comment
@mefellows

mefellows Jan 9, 2016

Actually @marcellodesales, I'm not sure that was a permanent fix either as I had troubles getting it to work later. The password-less SSH key was only installed on the docker machine client host, an nowhere else. What I ended up doing that worked reliably (and will revise once I have a better overall understanding) was creating the machine and providing ssh certs/keys whilst doing it i.e.

  docker-machine \
    --tls-ca-cert $LOCAL_CONFIG_PATH/ca.pem \
    --tls-ca-key $LOCAL_CONFIG_PATH/ca-key.pem \
    --tls-client-cert $LOCAL_CONFIG_PATH/cert.pem \
    --tls-client-key $LOCAL_CONFIG_PATH/key.pem \
    create \
    mymachinename \
    --driver generic  \
    --generic-ip-address 1.2.3.4 \
    --generic-ssh-key somekey.pem  \
    --generic-ssh-user ubuntu

This leads me to believe it's an issue with local SSH confguration, but further investigation is required.

mefellows commented Jan 9, 2016

Actually @marcellodesales, I'm not sure that was a permanent fix either as I had troubles getting it to work later. The password-less SSH key was only installed on the docker machine client host, an nowhere else. What I ended up doing that worked reliably (and will revise once I have a better overall understanding) was creating the machine and providing ssh certs/keys whilst doing it i.e.

  docker-machine \
    --tls-ca-cert $LOCAL_CONFIG_PATH/ca.pem \
    --tls-ca-key $LOCAL_CONFIG_PATH/ca-key.pem \
    --tls-client-cert $LOCAL_CONFIG_PATH/cert.pem \
    --tls-client-key $LOCAL_CONFIG_PATH/key.pem \
    create \
    mymachinename \
    --driver generic  \
    --generic-ip-address 1.2.3.4 \
    --generic-ssh-key somekey.pem  \
    --generic-ssh-user ubuntu

This leads me to believe it's an issue with local SSH confguration, but further investigation is required.

@marcellodesales

This comment has been minimized.

Show comment
Hide comment
@marcellodesales

marcellodesales Jan 11, 2016

@mefellows This looks more promising so far because I can see where all the client/server certs are being used, as well as the generic ssh key value is... I will try using it and report...

marcellodesales commented Jan 11, 2016

@mefellows This looks more promising so far because I can see where all the client/server certs are being used, as well as the generic ssh key value is... I will try using it and report...

@mefellows

This comment has been minimized.

Show comment
Hide comment
@mefellows

mefellows Jan 12, 2016

Excellent @marcellodesales, keen to hear how you go.

mefellows commented Jan 12, 2016

Excellent @marcellodesales, keen to hear how you go.

@RRAlex

This comment has been minimized.

Show comment
Hide comment
@RRAlex

RRAlex Jan 25, 2016

Looks like using a protected key with an ssh-agent loaded with it, but without specifying the key as an option to the generic driver, worked for me. Counter intuitive, but ...

Maybe the documentation should reflect this!

RRAlex commented Jan 25, 2016

Looks like using a protected key with an ssh-agent loaded with it, but without specifying the key as an option to the generic driver, worked for me. Counter intuitive, but ...

Maybe the documentation should reflect this!

@dgageot

This comment has been minimized.

Show comment
Hide comment
@dgageot

dgageot Feb 12, 2016

Collaborator

Closing. Generic driver supports password protected ssh keys through ssh-agent as specified here: https://github.com/docker/machine/blob/master/docs/drivers/generic.md#interaction-with-ssh-agents

Collaborator

dgageot commented Feb 12, 2016

Closing. Generic driver supports password protected ssh keys through ssh-agent as specified here: https://github.com/docker/machine/blob/master/docs/drivers/generic.md#interaction-with-ssh-agents

@dgageot dgageot closed this Feb 12, 2016

@RRAlex

This comment has been minimized.

Show comment
Hide comment
@RRAlex

RRAlex Feb 18, 2016

Agreed, but the error message right now is very cryptic and should give you something else than a number (ie: key not found / not unlocked / agent not found ... ?)

RRAlex commented Feb 18, 2016

Agreed, but the error message right now is very cryptic and should give you something else than a number (ie: key not found / not unlocked / agent not found ... ?)

@marcellodesales

This comment has been minimized.

Show comment
Hide comment
@marcellodesales

marcellodesales Jun 8, 2016

@dgageot Still having the same exact problem...

mdesales@Marcello-New2015 [06/08/201612:58:50] / $ docker-machine -D create -d generic --generic-ssh-key ~/.ssh/docker-id-rsa --generic-ssh-user dockeradmin  --generic-ip-address config-server.ttu.corp.intuit.net config-server
Docker Machine Version:  0.7.0, build a650a40
Found binary path at /usr/local/bin/docker-machine
Launching plugin server for driver generic
Plugin server listening at address 127.0.0.1:51096
() Calling .GetVersion
Using API Version  1
() Calling .SetConfigRaw
() Calling .GetMachineName
(flag-lookup) Calling .GetMachineName
(flag-lookup) Calling .DriverName
(flag-lookup) Calling .GetCreateFlags
Found binary path at /usr/local/bin/docker-machine
Launching plugin server for driver generic
Plugin server listening at address 127.0.0.1:51100
() Calling .GetVersion
Using API Version  1
() Calling .SetConfigRaw
() Calling .GetMachineName
(config-server) Calling .GetMachineName
(config-server) Calling .DriverName
(config-server) Calling .GetCreateFlags
(config-server) Calling .SetConfigFromFlags
Running pre-create checks...
(config-server) Calling .PreCreateCheck
(config-server) Calling .GetConfigRaw
Creating machine...
(config-server) Calling .Create
(config-server) Importing SSH key...
(config-server) Calling .GetConfigRaw
(config-server) DBG | IP: config-server.ttu.corp.intuit.net
(config-server) Calling .DriverName
(config-server) Calling .DriverName
Waiting for machine to be running, this may take a few minutes...
(config-server) Calling .GetState
Detecting operating system of created instance...
Waiting for SSH to be available...
Getting to WaitForSSH function...
(config-server) Calling .GetSSHHostname
(config-server) Calling .GetSSHPort
(config-server) Calling .GetSSHKeyPath
(config-server) Calling .GetSSHKeyPath
(config-server) Calling .GetSSHUsername
Using SSH client type: external
Using SSH private key: /Users/mdesales/.docker/machine/machines/config-server/id_rsa (-rw-------)
&{[-F /dev/null -o BatchMode=yes -o PasswordAuthentication=no -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=quiet -o ConnectionAttempts=3 -o ConnectTimeout=10 -o ControlMaster=no -o ControlPath=none dockeradmin@config-server.ttu.corp.intuit.net -o IdentitiesOnly=yes -i /Users/mdesales/.docker/machine/machines/config-server/id_rsa -p 22] /usr/bin/ssh <nil>}
About to run SSH command:
exit 0
SSH cmd err, output: <nil>:
Detecting the provisioner...
(config-server) Calling .GetSSHHostname
(config-server) Calling .GetSSHPort
(config-server) Calling .GetSSHKeyPath
(config-server) Calling .GetSSHKeyPath
(config-server) Calling .GetSSHUsername
Using SSH client type: external
Using SSH private key: /Users/mdesales/.docker/machine/machines/config-server/id_rsa (-rw-------)
&{[-F /dev/null -o BatchMode=yes -o PasswordAuthentication=no -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=quiet -o ConnectionAttempts=3 -o ConnectTimeout=10 -o ControlMaster=no -o ControlPath=none dockeradmin@config-server.ttu.corp.intuit.net -o IdentitiesOnly=yes -i /Users/mdesales/.docker/machine/machines/config-server/id_rsa -p 22] /usr/bin/ssh <nil>}
About to run SSH command:
cat /etc/os-release
SSH cmd err, output: <nil>: NAME="Red Hat Enterprise Linux Server"
VERSION="7.2 (Maipo)"
ID="rhel"
ID_LIKE="fedora"
VERSION_ID="7.2"
PRETTY_NAME="Red Hat Enterprise Linux Server 7.2 (Maipo)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:redhat:enterprise_linux:7.2:GA:server"
HOME_URL="https://www.redhat.com/"
BUG_REPORT_URL="https://bugzilla.redhat.com/"

REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 7"
REDHAT_BUGZILLA_PRODUCT_VERSION=7.2
REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
REDHAT_SUPPORT_PRODUCT_VERSION="7.2"

Couldn't set key CPE_NAME, no corresponding struct field found
Couldn't set key , no corresponding struct field found
Couldn't set key REDHAT_BUGZILLA_PRODUCT, no corresponding struct field found
Couldn't set key REDHAT_BUGZILLA_PRODUCT_VERSION, no corresponding struct field found
Couldn't set key REDHAT_SUPPORT_PRODUCT, no corresponding struct field found
Couldn't set key REDHAT_SUPPORT_PRODUCT_VERSION, no corresponding struct field found
found compatible host: rhel
Provisioning with redhat...
No storagedriver specified, using devicemapper

(config-server) Calling .GetMachineName
(config-server) Calling .GetSSHHostname
(config-server) Calling .GetSSHPort
(config-server) Calling .GetSSHKeyPath
(config-server) Calling .GetSSHKeyPath
(config-server) Calling .GetSSHUsername
Using SSH client type: external
Using SSH private key: /Users/mdesales/.docker/machine/machines/config-server/id_rsa (-rw-------)
&{[-F /dev/null -o BatchMode=yes -o PasswordAuthentication=no -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=quiet -o ConnectionAttempts=3 -o ConnectTimeout=10 -o ControlMaster=no -o ControlPath=none dockeradmin@config-server.ttu.corp.intuit.net -o IdentitiesOnly=yes -i /Users/mdesales/.docker/machine/machines/config-server/id_rsa -p 22] /usr/bin/ssh <nil>}
(config-server) Calling .GetSSHHostname
(config-server) Calling .GetSSHPort
(config-server) Calling .GetSSHKeyPath
(config-server) Calling .GetSSHKeyPath
(config-server) Calling .GetSSHUsername
Using SSH client type: external
Using SSH private key: /Users/mdesales/.docker/machine/machines/config-server/id_rsa (-rw-------)
&{[-F /dev/null -o BatchMode=yes -o PasswordAuthentication=no -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=quiet -o ConnectionAttempts=3 -o ConnectTimeout=10 -o ControlMaster=no -o ControlPath=none dockeradmin@config-server.ttu.corp.intuit.net -o IdentitiesOnly=yes -i /Users/mdesales/.docker/machine/machines/config-server/id_rsa -p 22] /usr/bin/ssh <nil>}
installing base package: name=curl
(config-server) Calling .GetSSHHostname
(config-server) Calling .GetSSHPort
(config-server) Calling .GetSSHKeyPath
(config-server) Calling .GetSSHKeyPath
(config-server) Calling .GetSSHUsername
Using SSH client type: external
Using SSH private key: /Users/mdesales/.docker/machine/machines/config-server/id_rsa (-rw-------)
&{[-F /dev/null -o BatchMode=yes -o PasswordAuthentication=no -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=quiet -o ConnectionAttempts=3 -o ConnectTimeout=10 -o ControlMaster=no -o ControlPath=none dockeradmin@config-server.ttu.corp.intuit.net -o IdentitiesOnly=yes -i /Users/mdesales/.docker/machine/machines/config-server/id_rsa -p 22] /usr/bin/ssh <nil>}

docker-machine ls showing server.pem: no such file or directory

The command docker-machine ls is showing that the PEM is missing...

mdesales@Marcello-New2015 [06/08/201613:17:26] / $ docker-machine ls
NAME            ACTIVE   DRIVER         STATE     URL                                            SWARM   DOCKER    ERRORS
config-server   -        generic        Running   tcp://config-server.ttu.corp.intuit.net:2376           Unknown   Unable to query docker version: Unable to read TLS config: open /Users/mdesales/.docker/machine/machines/config-server/server.pem: no such file or directory
default         -        virtualbox     Saved                                                            Unknown
dev             -        virtualbox     Saved                                                            Unknown
vmwf            *        vmwarefusion   Running   tcp://172.16.188.136:2376                              v1.11.2

In fact, the server.pem file was not created... Is it supposed to be created? I can confirm that the ssh key was copied from the the I requested during the command, but still missing the pem...

mdesales@Marcello-New2015 [06/08/201613:20:06] / $ ls -la /Users/mdesales/.docker/machine/machines/config-server/
total 16
drwx------  4 mdesales  staff   136 Jun  8 12:59 .
drwx------  8 mdesales  staff   272 Jun  8 12:59 ..
-rw-------  1 mdesales  staff  2378 Jun  8 12:59 config.json
-rw-------  1 mdesales  staff  1679 Jun  8 12:59 id_rsa

mdesales@Marcello-New2015 [06/08/201613:20:30] / $ diff /Users/mdesales/.docker/machine/machines/config-server/id_rsa ~/.ssh/docker-id-rsa
mdesales@Marcello-New2015 [06/08/201613:21:43] / $

Setting up the user with sudo

I setup the user dockeradmin and set it up properly as described at https://github.com/docker/machine/blob/master/docs/drivers/generic.md#interaction-with-ssh-agents.

LAPTOP $ ssh-add -L
AAAAB3NzaC1yc2EAAAADAQABAAABAQDBx+mtZOL7/Ou11YGGg6SWnL4LrPnmRQrCGaNryGoIB8V/MAxKYk98oQ2UpJTqeRL5qFezw0Z2nOmSf7jEFLvSvkXW3igfCZYjMih3czO4KMebHbBCDqLDUWHtpOv8aN+u7bKCNmGLn0J5YGnnwY6DqT0H02xoZ26JAqIbI3PqC2b0P/CyztXDGhOX3z3zyutOYirEojEY5jYpm1Bp2WervqfP2uzqysUuZVPGy5ZMaq7PiH5Tay5Udx3pqyAToCLXo35o1FNnG8l9NMgSiN6iebiddZSV8QS0xBjACT5ipJvVfCOlQ/uqQHTxanQtWxhv6e+bLtKDKpA4JnaY/bg7 /Users/mdesales/.ssh/docker-id-rsa

HOST $ visudo
dockeradmin     ALL=(ALL) NOPASSWD: ALL

$ ssh 'dockeradmin@config-server.ttu.corp.intuit.net'

 Last login: Wed Jun  8 12:54:03 2016 from 172.28.169.92
[dockeradmin@config-server ~]$ sudo su
[root@config-server dockeradmin]# exit
exit

Questions

  • What's missing?
  • I logged into the machine and I saw a command yum -y upgrade. Was that triggered from docker-machine?
    • If so, shouldn't that be shown in the docker-machine output while managing a host?
[root@config-server dockeradmin]# ps aux | grep yum
root     20982  0.0  0.0 213304  3624 ?        Ss   13:13   0:00 sudo yum -y update
root     20989  0.0  0.2 423552 23664 ?        S    13:13   0:00 /usr/bin/python /bin/yum -y update

marcellodesales commented Jun 8, 2016

@dgageot Still having the same exact problem...

mdesales@Marcello-New2015 [06/08/201612:58:50] / $ docker-machine -D create -d generic --generic-ssh-key ~/.ssh/docker-id-rsa --generic-ssh-user dockeradmin  --generic-ip-address config-server.ttu.corp.intuit.net config-server
Docker Machine Version:  0.7.0, build a650a40
Found binary path at /usr/local/bin/docker-machine
Launching plugin server for driver generic
Plugin server listening at address 127.0.0.1:51096
() Calling .GetVersion
Using API Version  1
() Calling .SetConfigRaw
() Calling .GetMachineName
(flag-lookup) Calling .GetMachineName
(flag-lookup) Calling .DriverName
(flag-lookup) Calling .GetCreateFlags
Found binary path at /usr/local/bin/docker-machine
Launching plugin server for driver generic
Plugin server listening at address 127.0.0.1:51100
() Calling .GetVersion
Using API Version  1
() Calling .SetConfigRaw
() Calling .GetMachineName
(config-server) Calling .GetMachineName
(config-server) Calling .DriverName
(config-server) Calling .GetCreateFlags
(config-server) Calling .SetConfigFromFlags
Running pre-create checks...
(config-server) Calling .PreCreateCheck
(config-server) Calling .GetConfigRaw
Creating machine...
(config-server) Calling .Create
(config-server) Importing SSH key...
(config-server) Calling .GetConfigRaw
(config-server) DBG | IP: config-server.ttu.corp.intuit.net
(config-server) Calling .DriverName
(config-server) Calling .DriverName
Waiting for machine to be running, this may take a few minutes...
(config-server) Calling .GetState
Detecting operating system of created instance...
Waiting for SSH to be available...
Getting to WaitForSSH function...
(config-server) Calling .GetSSHHostname
(config-server) Calling .GetSSHPort
(config-server) Calling .GetSSHKeyPath
(config-server) Calling .GetSSHKeyPath
(config-server) Calling .GetSSHUsername
Using SSH client type: external
Using SSH private key: /Users/mdesales/.docker/machine/machines/config-server/id_rsa (-rw-------)
&{[-F /dev/null -o BatchMode=yes -o PasswordAuthentication=no -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=quiet -o ConnectionAttempts=3 -o ConnectTimeout=10 -o ControlMaster=no -o ControlPath=none dockeradmin@config-server.ttu.corp.intuit.net -o IdentitiesOnly=yes -i /Users/mdesales/.docker/machine/machines/config-server/id_rsa -p 22] /usr/bin/ssh <nil>}
About to run SSH command:
exit 0
SSH cmd err, output: <nil>:
Detecting the provisioner...
(config-server) Calling .GetSSHHostname
(config-server) Calling .GetSSHPort
(config-server) Calling .GetSSHKeyPath
(config-server) Calling .GetSSHKeyPath
(config-server) Calling .GetSSHUsername
Using SSH client type: external
Using SSH private key: /Users/mdesales/.docker/machine/machines/config-server/id_rsa (-rw-------)
&{[-F /dev/null -o BatchMode=yes -o PasswordAuthentication=no -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=quiet -o ConnectionAttempts=3 -o ConnectTimeout=10 -o ControlMaster=no -o ControlPath=none dockeradmin@config-server.ttu.corp.intuit.net -o IdentitiesOnly=yes -i /Users/mdesales/.docker/machine/machines/config-server/id_rsa -p 22] /usr/bin/ssh <nil>}
About to run SSH command:
cat /etc/os-release
SSH cmd err, output: <nil>: NAME="Red Hat Enterprise Linux Server"
VERSION="7.2 (Maipo)"
ID="rhel"
ID_LIKE="fedora"
VERSION_ID="7.2"
PRETTY_NAME="Red Hat Enterprise Linux Server 7.2 (Maipo)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:redhat:enterprise_linux:7.2:GA:server"
HOME_URL="https://www.redhat.com/"
BUG_REPORT_URL="https://bugzilla.redhat.com/"

REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 7"
REDHAT_BUGZILLA_PRODUCT_VERSION=7.2
REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
REDHAT_SUPPORT_PRODUCT_VERSION="7.2"

Couldn't set key CPE_NAME, no corresponding struct field found
Couldn't set key , no corresponding struct field found
Couldn't set key REDHAT_BUGZILLA_PRODUCT, no corresponding struct field found
Couldn't set key REDHAT_BUGZILLA_PRODUCT_VERSION, no corresponding struct field found
Couldn't set key REDHAT_SUPPORT_PRODUCT, no corresponding struct field found
Couldn't set key REDHAT_SUPPORT_PRODUCT_VERSION, no corresponding struct field found
found compatible host: rhel
Provisioning with redhat...
No storagedriver specified, using devicemapper

(config-server) Calling .GetMachineName
(config-server) Calling .GetSSHHostname
(config-server) Calling .GetSSHPort
(config-server) Calling .GetSSHKeyPath
(config-server) Calling .GetSSHKeyPath
(config-server) Calling .GetSSHUsername
Using SSH client type: external
Using SSH private key: /Users/mdesales/.docker/machine/machines/config-server/id_rsa (-rw-------)
&{[-F /dev/null -o BatchMode=yes -o PasswordAuthentication=no -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=quiet -o ConnectionAttempts=3 -o ConnectTimeout=10 -o ControlMaster=no -o ControlPath=none dockeradmin@config-server.ttu.corp.intuit.net -o IdentitiesOnly=yes -i /Users/mdesales/.docker/machine/machines/config-server/id_rsa -p 22] /usr/bin/ssh <nil>}
(config-server) Calling .GetSSHHostname
(config-server) Calling .GetSSHPort
(config-server) Calling .GetSSHKeyPath
(config-server) Calling .GetSSHKeyPath
(config-server) Calling .GetSSHUsername
Using SSH client type: external
Using SSH private key: /Users/mdesales/.docker/machine/machines/config-server/id_rsa (-rw-------)
&{[-F /dev/null -o BatchMode=yes -o PasswordAuthentication=no -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=quiet -o ConnectionAttempts=3 -o ConnectTimeout=10 -o ControlMaster=no -o ControlPath=none dockeradmin@config-server.ttu.corp.intuit.net -o IdentitiesOnly=yes -i /Users/mdesales/.docker/machine/machines/config-server/id_rsa -p 22] /usr/bin/ssh <nil>}
installing base package: name=curl
(config-server) Calling .GetSSHHostname
(config-server) Calling .GetSSHPort
(config-server) Calling .GetSSHKeyPath
(config-server) Calling .GetSSHKeyPath
(config-server) Calling .GetSSHUsername
Using SSH client type: external
Using SSH private key: /Users/mdesales/.docker/machine/machines/config-server/id_rsa (-rw-------)
&{[-F /dev/null -o BatchMode=yes -o PasswordAuthentication=no -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=quiet -o ConnectionAttempts=3 -o ConnectTimeout=10 -o ControlMaster=no -o ControlPath=none dockeradmin@config-server.ttu.corp.intuit.net -o IdentitiesOnly=yes -i /Users/mdesales/.docker/machine/machines/config-server/id_rsa -p 22] /usr/bin/ssh <nil>}

docker-machine ls showing server.pem: no such file or directory

The command docker-machine ls is showing that the PEM is missing...

mdesales@Marcello-New2015 [06/08/201613:17:26] / $ docker-machine ls
NAME            ACTIVE   DRIVER         STATE     URL                                            SWARM   DOCKER    ERRORS
config-server   -        generic        Running   tcp://config-server.ttu.corp.intuit.net:2376           Unknown   Unable to query docker version: Unable to read TLS config: open /Users/mdesales/.docker/machine/machines/config-server/server.pem: no such file or directory
default         -        virtualbox     Saved                                                            Unknown
dev             -        virtualbox     Saved                                                            Unknown
vmwf            *        vmwarefusion   Running   tcp://172.16.188.136:2376                              v1.11.2

In fact, the server.pem file was not created... Is it supposed to be created? I can confirm that the ssh key was copied from the the I requested during the command, but still missing the pem...

mdesales@Marcello-New2015 [06/08/201613:20:06] / $ ls -la /Users/mdesales/.docker/machine/machines/config-server/
total 16
drwx------  4 mdesales  staff   136 Jun  8 12:59 .
drwx------  8 mdesales  staff   272 Jun  8 12:59 ..
-rw-------  1 mdesales  staff  2378 Jun  8 12:59 config.json
-rw-------  1 mdesales  staff  1679 Jun  8 12:59 id_rsa

mdesales@Marcello-New2015 [06/08/201613:20:30] / $ diff /Users/mdesales/.docker/machine/machines/config-server/id_rsa ~/.ssh/docker-id-rsa
mdesales@Marcello-New2015 [06/08/201613:21:43] / $

Setting up the user with sudo

I setup the user dockeradmin and set it up properly as described at https://github.com/docker/machine/blob/master/docs/drivers/generic.md#interaction-with-ssh-agents.

LAPTOP $ ssh-add -L
AAAAB3NzaC1yc2EAAAADAQABAAABAQDBx+mtZOL7/Ou11YGGg6SWnL4LrPnmRQrCGaNryGoIB8V/MAxKYk98oQ2UpJTqeRL5qFezw0Z2nOmSf7jEFLvSvkXW3igfCZYjMih3czO4KMebHbBCDqLDUWHtpOv8aN+u7bKCNmGLn0J5YGnnwY6DqT0H02xoZ26JAqIbI3PqC2b0P/CyztXDGhOX3z3zyutOYirEojEY5jYpm1Bp2WervqfP2uzqysUuZVPGy5ZMaq7PiH5Tay5Udx3pqyAToCLXo35o1FNnG8l9NMgSiN6iebiddZSV8QS0xBjACT5ipJvVfCOlQ/uqQHTxanQtWxhv6e+bLtKDKpA4JnaY/bg7 /Users/mdesales/.ssh/docker-id-rsa

HOST $ visudo
dockeradmin     ALL=(ALL) NOPASSWD: ALL

$ ssh 'dockeradmin@config-server.ttu.corp.intuit.net'

 Last login: Wed Jun  8 12:54:03 2016 from 172.28.169.92
[dockeradmin@config-server ~]$ sudo su
[root@config-server dockeradmin]# exit
exit

Questions

  • What's missing?
  • I logged into the machine and I saw a command yum -y upgrade. Was that triggered from docker-machine?
    • If so, shouldn't that be shown in the docker-machine output while managing a host?
[root@config-server dockeradmin]# ps aux | grep yum
root     20982  0.0  0.0 213304  3624 ?        Ss   13:13   0:00 sudo yum -y update
root     20989  0.0  0.2 423552 23664 ?        S    13:13   0:00 /usr/bin/python /bin/yum -y update
@marcellodesales

This comment has been minimized.

Show comment
Hide comment
@marcellodesales

marcellodesales Jun 8, 2016

I upgraded the machine manually to match my laptop's version... I re-ran the command and it failed with the following:

...
...
Using SSH client type: external
Using SSH private key: /Users/mdesales/.docker/machine/machines/config-server/id_rsa (-rw-------)
&{[-F /dev/null -o BatchMode=yes -o PasswordAuthentication=no -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=quiet -o ConnectionAttempts=3 -o ConnectTimeout=10 -o ControlMaster=no -o ControlPath=none dockeradmin@config-server.ttu.corp.intuit.net -o IdentitiesOnly=yes -i /Users/mdesales/.docker/machine/machines/config-server/id_rsa -p 22] /usr/bin/ssh <nil>}
Error creating machine: Error running provisioning: exit status 1
Opting out of crash reporting.
  • Does it require to create the PEM certs for the box?
  • For existing machines running Docker, would docker-machine create the PEM files and transfer them to the LAPTOP from the HOST?

Laptop and Host Docker on same docker version

mdesales@Marcello-New2015 [06/08/201613:39:33] / $ docker --version
Docker version 1.11.1, build 5604cbe

[root@config-server dockeradmin]# docker --version
Docker version 1.11.1, build 5604cbe

[root@config-server dockeradmin]# docker info
Containers: 12
 Running: 0
 Paused: 0
 Stopped: 12
Images: 15
Server Version: 1.11.1
Storage Driver: devicemapper
 Pool Name: docker-253:2-17045359-pool
 Pool Blocksize: 65.54 kB
 Base Device Size: 107.4 GB
 Backing Filesystem: ext4
 Data file: /dev/loop0
 Metadata file: /dev/loop1
 Data Space Used: 4.519 GB
 Data Space Total: 107.4 GB
 Data Space Available: 11.53 GB
 Metadata Space Used: 7.827 MB
 Metadata Space Total: 2.147 GB
 Metadata Space Available: 2.14 GB
 Udev Sync Supported: true
 Deferred Removal Enabled: false
 Deferred Deletion Enabled: false
 Deferred Deleted Device Count: 0
 Data loop file: /var/lib/docker/devicemapper/devicemapper/data
 WARNING: Usage of loopback devices is strongly discouraged for production use. Either use `--storage-opt dm.thinpooldev` or use `--storage-opt dm.no_warn_on_loop_devices=true` to suppress this warning.
 Metadata loop file: /var/lib/docker/devicemapper/devicemapper/metadata
 Library Version: 1.02.107-RHEL7 (2015-12-01)
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: null host bridge
Kernel Version: 3.10.0-327.13.1.el7.x86_64
Operating System: Red Hat Enterprise Linux Server 7.2 (Maipo)
OSType: linux
Architecture: x86_64
CPUs: 2
Total Memory: 7.64 GiB
Name: config-server
ID: 6JSO:7HLF:EZAA:6B2O:NZWJ:5KGT:N27Q:N2UQ:C2DE:62EO:6F6B:KMJQ
Docker Root Dir: /var/lib/docker
Debug mode (client): false
Debug mode (server): false
Http Proxy: http://qypprdproxy02.ie.intuit.net:80
Https Proxy: http://qypprdproxy02.ie.intuit.net:80
No Proxy: .intuit.net,.intuit.com,10.*.*.*,localhost,127.0.0.1
Registry: https://index.docker.io/v1/
WARNING: bridge-nf-call-iptables is disabled
WARNING: bridge-nf-call-ip6tables is disabled

marcellodesales commented Jun 8, 2016

I upgraded the machine manually to match my laptop's version... I re-ran the command and it failed with the following:

...
...
Using SSH client type: external
Using SSH private key: /Users/mdesales/.docker/machine/machines/config-server/id_rsa (-rw-------)
&{[-F /dev/null -o BatchMode=yes -o PasswordAuthentication=no -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=quiet -o ConnectionAttempts=3 -o ConnectTimeout=10 -o ControlMaster=no -o ControlPath=none dockeradmin@config-server.ttu.corp.intuit.net -o IdentitiesOnly=yes -i /Users/mdesales/.docker/machine/machines/config-server/id_rsa -p 22] /usr/bin/ssh <nil>}
Error creating machine: Error running provisioning: exit status 1
Opting out of crash reporting.
  • Does it require to create the PEM certs for the box?
  • For existing machines running Docker, would docker-machine create the PEM files and transfer them to the LAPTOP from the HOST?

Laptop and Host Docker on same docker version

mdesales@Marcello-New2015 [06/08/201613:39:33] / $ docker --version
Docker version 1.11.1, build 5604cbe

[root@config-server dockeradmin]# docker --version
Docker version 1.11.1, build 5604cbe

[root@config-server dockeradmin]# docker info
Containers: 12
 Running: 0
 Paused: 0
 Stopped: 12
Images: 15
Server Version: 1.11.1
Storage Driver: devicemapper
 Pool Name: docker-253:2-17045359-pool
 Pool Blocksize: 65.54 kB
 Base Device Size: 107.4 GB
 Backing Filesystem: ext4
 Data file: /dev/loop0
 Metadata file: /dev/loop1
 Data Space Used: 4.519 GB
 Data Space Total: 107.4 GB
 Data Space Available: 11.53 GB
 Metadata Space Used: 7.827 MB
 Metadata Space Total: 2.147 GB
 Metadata Space Available: 2.14 GB
 Udev Sync Supported: true
 Deferred Removal Enabled: false
 Deferred Deletion Enabled: false
 Deferred Deleted Device Count: 0
 Data loop file: /var/lib/docker/devicemapper/devicemapper/data
 WARNING: Usage of loopback devices is strongly discouraged for production use. Either use `--storage-opt dm.thinpooldev` or use `--storage-opt dm.no_warn_on_loop_devices=true` to suppress this warning.
 Metadata loop file: /var/lib/docker/devicemapper/devicemapper/metadata
 Library Version: 1.02.107-RHEL7 (2015-12-01)
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: null host bridge
Kernel Version: 3.10.0-327.13.1.el7.x86_64
Operating System: Red Hat Enterprise Linux Server 7.2 (Maipo)
OSType: linux
Architecture: x86_64
CPUs: 2
Total Memory: 7.64 GiB
Name: config-server
ID: 6JSO:7HLF:EZAA:6B2O:NZWJ:5KGT:N27Q:N2UQ:C2DE:62EO:6F6B:KMJQ
Docker Root Dir: /var/lib/docker
Debug mode (client): false
Debug mode (server): false
Http Proxy: http://qypprdproxy02.ie.intuit.net:80
Https Proxy: http://qypprdproxy02.ie.intuit.net:80
No Proxy: .intuit.net,.intuit.com,10.*.*.*,localhost,127.0.0.1
Registry: https://index.docker.io/v1/
WARNING: bridge-nf-call-iptables is disabled
WARNING: bridge-nf-call-ip6tables is disabled
@marcellodesales

This comment has been minimized.

Show comment
Hide comment
@marcellodesales

marcellodesales Jun 8, 2016

I finally got it to work...

$ docker-machine ls
NAME            ACTIVE   DRIVER         STATE     URL                                            SWARM   DOCKER    ERRORS
config-server   *        generic        Running   tcp://config-server.ttu.corp.intuit.net:2376           v1.11.1
default         -        virtualbox     Saved                                                            Unknown
dev             -        virtualbox     Saved                                                            Unknown
vmwf            -        vmwarefusion   Running   tcp://172.16.188.136:2376                              v1.11.2

Missing Pieces

  • Generate all the server keys and initiate the Docker Engine using TLS
  • The Generic Driver will only work on hosts that have been setup to run with the HTTPS interface
    • Does Docker Machine setup the host with the new keys?
    • What's the recommended way to setup those hosts with the new keys from Docker Machine?

Reviewing https://docs.docker.com/engine/security/https/, I noticed that the ca.pem is NOT retrieved the ca.pem. Also, as described by #3212 (comment), the keys could be copied from the original server. So, I copied all the generated keys from the host to the LAPTOP...

$ ~/.docker/machine/machines/config-server $ 
scp dockeradmin@config-server.ttu.corp.intuit.net:/tmp/pems/cert.pem server.pem
$ ~/.docker/machine/machines/config-server $ 
scp dockeradmin@config-server.ttu.corp.intuit.net:/tmp/pems/key.pem server-key.pem
$ ~/.docker/machine/machines/config-server $ 
scp dockeradmin@config-server.ttu.corp.intuit.net:/tmp/pems/ca.pem ca.pem

 $ ls -la
total 40
drwx------  7 mdesales  staff   238 Jun  8 15:34 .
drwx------  8 mdesales  staff   272 Jun  8 13:34 ..
-rwxr-xr-x  1 mdesales  staff  2183 Jun  8 15:31 ca.pem
-rw-------  1 mdesales  staff  2396 Jun  8 15:34 config.json
-rw-------  1 mdesales  staff  1679 Jun  8 13:34 id_rsa
-rwxr-xr-x  1 mdesales  staff  3243 Jun  8 15:28 server-key.pem
-rwxr-xr-x  1 mdesales  staff  1903 Jun  8 15:27 server.pem

The last change is the config.json file to change the location from ca.pem from the default machine ca.pem to the one we just copied...

      "CaCertPath": "/Users/mdesales/.docker/machine/machines/config-server/ca.pem",

Lastly, you need select the machine to use...

$ eval $(docker-machine env config-server)

The eval switches to the other machine... The environment variable is now switched.

$ env | grep -i 'docker'
DOCKER_HOST=tcp://config-server.ttu.corp.intuit.net:2376
DOCKER_MACHINE_NAME=config-server
DOCKER_TLS_VERIFY=1
PWD=/Users/mdesales/.docker/machine/machines/config-server
DOCKER_CERT_PATH=/Users/mdesales/.docker/machine/machines/config-server

The error about bad certificate can be solved with the following (#2270 (comment))

$ export DOCKER_OPTS="-H $DOCKER_HOST --tls --tlskey $DOCKER_CERT_PATH/server-key.pem 
   --tlscert $DOCKER_CERT_PATH/server.pem --tlsverify --tlscacert $DOCKER_CERT_PATH/ca.pem "

$ alias docker="docker $DOCKER_OPTS "

At this point, all the commands are being sent to the generic machine that's selected.

$ docker info
Containers: 12
 Running: 0
 Paused: 0
 Stopped: 12
Images: 15
Server Version: 1.11.1
Storage Driver: devicemapper
 Pool Name: docker-253:2-17045359-pool
 Pool Blocksize: 65.54 kB
 Base Device Size: 107.4 GB
 Backing Filesystem: ext4
 Data file: /dev/loop0
 Metadata file: /dev/loop1
 Data Space Used: 4.519 GB
 Data Space Total: 107.4 GB
 Data Space Available: 11.52 GB
 Metadata Space Used: 7.827 MB
 Metadata Space Total: 2.147 GB
 Metadata Space Available: 2.14 GB
 Udev Sync Supported: true
 Deferred Removal Enabled: false
 Deferred Deletion Enabled: false
 Deferred Deleted Device Count: 0
 Data loop file: /var/lib/docker/devicemapper/devicemapper/data
 WARNING: Usage of loopback devices is strongly discouraged for production use. Either use `--storage-opt dm.thinpooldev` or use `--storage-opt dm.no_warn_on_loop_devices=true` to suppress this warning.
 Metadata loop file: /var/lib/docker/devicemapper/devicemapper/metadata
 Library Version: 1.02.107-RHEL7 (2015-12-01)
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: bridge null host
Kernel Version: 3.10.0-327.13.1.el7.x86_64
Operating System: Red Hat Enterprise Linux Server 7.2 (Maipo)
OSType: linux
Architecture: x86_64
CPUs: 2
Total Memory: 7.64 GiB
Name: config-server
ID: 6JSO:7HLF:EZAA:6B2O:NZWJ:5KGT:N27Q:N2UQ:C2DE:62EO:6F6B:KMJQ
Docker Root Dir: /var/lib/docker
Debug mode (client): false
Debug mode (server): false
Http Proxy: http://qypprdproxy02.ie.intuit.net:80
Https Proxy: http://qypprdproxy02.ie.intuit.net:80
No Proxy: .intuit.net,.intuit.com,10.*.*.*,localhost,127.0.0.1
Registry: https://index.docker.io/v1/
WARNING: bridge-nf-call-iptables is disabled
WARNING: bridge-nf-call-ip6tables is disabled

At this point, everything started working...

marcellodesales commented Jun 8, 2016

I finally got it to work...

$ docker-machine ls
NAME            ACTIVE   DRIVER         STATE     URL                                            SWARM   DOCKER    ERRORS
config-server   *        generic        Running   tcp://config-server.ttu.corp.intuit.net:2376           v1.11.1
default         -        virtualbox     Saved                                                            Unknown
dev             -        virtualbox     Saved                                                            Unknown
vmwf            -        vmwarefusion   Running   tcp://172.16.188.136:2376                              v1.11.2

Missing Pieces

  • Generate all the server keys and initiate the Docker Engine using TLS
  • The Generic Driver will only work on hosts that have been setup to run with the HTTPS interface
    • Does Docker Machine setup the host with the new keys?
    • What's the recommended way to setup those hosts with the new keys from Docker Machine?

Reviewing https://docs.docker.com/engine/security/https/, I noticed that the ca.pem is NOT retrieved the ca.pem. Also, as described by #3212 (comment), the keys could be copied from the original server. So, I copied all the generated keys from the host to the LAPTOP...

$ ~/.docker/machine/machines/config-server $ 
scp dockeradmin@config-server.ttu.corp.intuit.net:/tmp/pems/cert.pem server.pem
$ ~/.docker/machine/machines/config-server $ 
scp dockeradmin@config-server.ttu.corp.intuit.net:/tmp/pems/key.pem server-key.pem
$ ~/.docker/machine/machines/config-server $ 
scp dockeradmin@config-server.ttu.corp.intuit.net:/tmp/pems/ca.pem ca.pem

 $ ls -la
total 40
drwx------  7 mdesales  staff   238 Jun  8 15:34 .
drwx------  8 mdesales  staff   272 Jun  8 13:34 ..
-rwxr-xr-x  1 mdesales  staff  2183 Jun  8 15:31 ca.pem
-rw-------  1 mdesales  staff  2396 Jun  8 15:34 config.json
-rw-------  1 mdesales  staff  1679 Jun  8 13:34 id_rsa
-rwxr-xr-x  1 mdesales  staff  3243 Jun  8 15:28 server-key.pem
-rwxr-xr-x  1 mdesales  staff  1903 Jun  8 15:27 server.pem

The last change is the config.json file to change the location from ca.pem from the default machine ca.pem to the one we just copied...

      "CaCertPath": "/Users/mdesales/.docker/machine/machines/config-server/ca.pem",

Lastly, you need select the machine to use...

$ eval $(docker-machine env config-server)

The eval switches to the other machine... The environment variable is now switched.

$ env | grep -i 'docker'
DOCKER_HOST=tcp://config-server.ttu.corp.intuit.net:2376
DOCKER_MACHINE_NAME=config-server
DOCKER_TLS_VERIFY=1
PWD=/Users/mdesales/.docker/machine/machines/config-server
DOCKER_CERT_PATH=/Users/mdesales/.docker/machine/machines/config-server

The error about bad certificate can be solved with the following (#2270 (comment))

$ export DOCKER_OPTS="-H $DOCKER_HOST --tls --tlskey $DOCKER_CERT_PATH/server-key.pem 
   --tlscert $DOCKER_CERT_PATH/server.pem --tlsverify --tlscacert $DOCKER_CERT_PATH/ca.pem "

$ alias docker="docker $DOCKER_OPTS "

At this point, all the commands are being sent to the generic machine that's selected.

$ docker info
Containers: 12
 Running: 0
 Paused: 0
 Stopped: 12
Images: 15
Server Version: 1.11.1
Storage Driver: devicemapper
 Pool Name: docker-253:2-17045359-pool
 Pool Blocksize: 65.54 kB
 Base Device Size: 107.4 GB
 Backing Filesystem: ext4
 Data file: /dev/loop0
 Metadata file: /dev/loop1
 Data Space Used: 4.519 GB
 Data Space Total: 107.4 GB
 Data Space Available: 11.52 GB
 Metadata Space Used: 7.827 MB
 Metadata Space Total: 2.147 GB
 Metadata Space Available: 2.14 GB
 Udev Sync Supported: true
 Deferred Removal Enabled: false
 Deferred Deletion Enabled: false
 Deferred Deleted Device Count: 0
 Data loop file: /var/lib/docker/devicemapper/devicemapper/data
 WARNING: Usage of loopback devices is strongly discouraged for production use. Either use `--storage-opt dm.thinpooldev` or use `--storage-opt dm.no_warn_on_loop_devices=true` to suppress this warning.
 Metadata loop file: /var/lib/docker/devicemapper/devicemapper/metadata
 Library Version: 1.02.107-RHEL7 (2015-12-01)
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: bridge null host
Kernel Version: 3.10.0-327.13.1.el7.x86_64
Operating System: Red Hat Enterprise Linux Server 7.2 (Maipo)
OSType: linux
Architecture: x86_64
CPUs: 2
Total Memory: 7.64 GiB
Name: config-server
ID: 6JSO:7HLF:EZAA:6B2O:NZWJ:5KGT:N27Q:N2UQ:C2DE:62EO:6F6B:KMJQ
Docker Root Dir: /var/lib/docker
Debug mode (client): false
Debug mode (server): false
Http Proxy: http://qypprdproxy02.ie.intuit.net:80
Https Proxy: http://qypprdproxy02.ie.intuit.net:80
No Proxy: .intuit.net,.intuit.com,10.*.*.*,localhost,127.0.0.1
Registry: https://index.docker.io/v1/
WARNING: bridge-nf-call-iptables is disabled
WARNING: bridge-nf-call-ip6tables is disabled

At this point, everything started working...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment