New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

boot2docker(v 1.4.1 and 1.5) bad cert issues on OSX 10.9.3 #824

Closed
meenarc opened this Issue Apr 10, 2015 · 106 comments

Comments

Projects
None yet
@meenarc

meenarc commented Apr 10, 2015

Keep running into the following error I attempt to run the hello-world app.
An error occurred trying to connect: Post https://192.168.59.103:2376/v1.16/containers/create: remote error: bad certificate

Have attempted to do boot2docker stop,delete and init multiple times. Also tried manually removing the .bootdocker directory and starting over

The following is what I see in the /var/log/docker.log file

time="2015-04-10T18:25:26Z" level="debug" msg="/usr/local/sbin/iptables, [-t nat -D OUTPUT -m addrtype --dst-type LOCAL ! --dst 127.0.0.0/8 -j DOCKER]"
time="2015-04-10T18:25:26Z" level="debug" msg="/usr/local/sbin/iptables, [-t nat -D OUTPUT -m addrtype --dst-type LOCAL -j DOCKER]"
time="2015-04-10T18:25:26Z" level="debug" msg="/usr/local/sbin/iptables, [-t nat -D PREROUTING -j DOCKER]"
time="2015-04-10T18:25:26Z" level="debug" msg="/usr/local/sbin/iptables, [-t nat -D OUTPUT -j DOCKER]"
time="2015-04-10T18:25:26Z" level="debug" msg="/usr/local/sbin/iptables, [-t nat -F DOCKER]"
time="2015-04-10T18:25:26Z" level="debug" msg="/usr/local/sbin/iptables, [-t nat -X DOCKER]"
time="2015-04-10T18:25:26Z" level="debug" msg="/usr/local/sbin/iptables, [-t nat -N DOCKER]"
time="2015-04-10T18:25:26Z" level="debug" msg="/usr/local/sbin/iptables, [-t nat -A PREROUTING -m addrtype --dst-type LOCAL -j DOCKER]"
time="2015-04-10T18:25:26Z" level="debug" msg="/usr/local/sbin/iptables, [-t nat -A OUTPUT -m addrtype --dst-type LOCAL ! --dst 127.0.0.0/8 -j DOCKER]"
time="2015-04-10T18:25:26Z" level="info" msg="-job init_networkdriver() = OK (0)"
time="2015-04-10T18:25:26Z" level="debug" msg="Restarting containers..."
time="2015-04-10T18:25:26Z" level="info" msg="docker daemon: 1.4.1 5bc2ff8; execdriver: native-0.2; graphdriver: aufs"
time="2015-04-10T18:25:26Z" level="info" msg="+job acceptconnections()"
time="2015-04-10T18:25:26Z" level="info" msg="-job acceptconnections() = OK (0)"
2015/04/10 18:08:43 http: TLS handshake error from 192.168.59.3:53358: tls: failed to verify client's certificate: x509: certificate has expired or is not yet valid

@phact

This comment has been minimized.

Show comment
Hide comment
@phact

phact May 6, 2015

I see this too sporadically. I run:
boot2docker stop
boot2docker delete
boot2docker init
boot2docker start

and it goes away. But keeps coming back...

phact commented May 6, 2015

I see this too sporadically. I run:
boot2docker stop
boot2docker delete
boot2docker init
boot2docker start

and it goes away. But keeps coming back...

@meenarc

This comment has been minimized.

Show comment
Hide comment
@meenarc

meenarc May 6, 2015

Yes, that's exactly what I have noticed too. Going over the stop/delete/init/start cycle fixes the issues but sooner or later the same issue shows up

meenarc commented May 6, 2015

Yes, that's exactly what I have noticed too. Going over the stop/delete/init/start cycle fixes the issues but sooner or later the same issue shows up

@zachaller

This comment has been minimized.

Show comment
Hide comment
@zachaller

zachaller May 28, 2015

I also have this same issue, the second the vm restarts i get the issue so

boot2docker init
boot2docker up
everything works fine now
boot2docker restart
now it just throws this error with only way to fix is to delete vm
FATA[0000] An error occurred trying to connect: Get https://192.168.59.107:2376/v1.18/containers/json?all=1: x509: certificate is valid for 127.0.0.1, 10.0.2.15, not 192.168.59.107

this is with the newest boot2docker

zachaller commented May 28, 2015

I also have this same issue, the second the vm restarts i get the issue so

boot2docker init
boot2docker up
everything works fine now
boot2docker restart
now it just throws this error with only way to fix is to delete vm
FATA[0000] An error occurred trying to connect: Get https://192.168.59.107:2376/v1.18/containers/json?all=1: x509: certificate is valid for 127.0.0.1, 10.0.2.15, not 192.168.59.107

this is with the newest boot2docker

@housebolt

This comment has been minimized.

Show comment
Hide comment
@housebolt

housebolt commented Jun 19, 2015

+1

@mkmik

This comment has been minimized.

Show comment
Hide comment
@mkmik

mkmik Jun 19, 2015

I had a similar problem after doing boot2docker update to version 1.7.0.

I solved the issue by removing the server cert and key files and restarted /etc/init.d/docker.

It regenerated the server certificate with the address 192.168.59.103 now bound to the eth1 interface. It might be that the interface wasn't correctly brought up when the certificate has been regenerated during boot2docker update.

$ rm /var/lib/boot2docker/tls/server*.pem

mkmik commented Jun 19, 2015

I had a similar problem after doing boot2docker update to version 1.7.0.

I solved the issue by removing the server cert and key files and restarted /etc/init.d/docker.

It regenerated the server certificate with the address 192.168.59.103 now bound to the eth1 interface. It might be that the interface wasn't correctly brought up when the certificate has been regenerated during boot2docker update.

$ rm /var/lib/boot2docker/tls/server*.pem
@rafi

This comment has been minimized.

Show comment
Hide comment
@rafi

rafi Jun 19, 2015

Deleting boot2docker's ssh keys and re-initialization boot2docker worked, thanks @mmikulicic

rafi commented Jun 19, 2015

Deleting boot2docker's ssh keys and re-initialization boot2docker worked, thanks @mmikulicic

@jmshelby

This comment has been minimized.

Show comment
Hide comment
@jmshelby

jmshelby Jun 19, 2015

I'm having the same problem after doing boot2docker update to version 1.7.0 also...
Performing the steps mentioned by @mmikulicic does not work; in fact, it breaks other things. (like boot2docker shellinit returning error in run: Error requesting socket: exit status 1)

The problem goes away if I destroy the image, and re-init; but comes back once I restart the vm (like 'restart' or 'start' and then 'stop).

Here is my specific message:

An error occurred trying to connect: Get https://192.168.59.103:2376/v1.19/containers/json: x509: certificate is valid for 127.0.0.1, 10.0.2.15, not 192.168.59.103

jmshelby commented Jun 19, 2015

I'm having the same problem after doing boot2docker update to version 1.7.0 also...
Performing the steps mentioned by @mmikulicic does not work; in fact, it breaks other things. (like boot2docker shellinit returning error in run: Error requesting socket: exit status 1)

The problem goes away if I destroy the image, and re-init; but comes back once I restart the vm (like 'restart' or 'start' and then 'stop).

Here is my specific message:

An error occurred trying to connect: Get https://192.168.59.103:2376/v1.19/containers/json: x509: certificate is valid for 127.0.0.1, 10.0.2.15, not 192.168.59.103
@rooty0

This comment has been minimized.

Show comment
Hide comment
@rooty0

rooty0 Jun 19, 2015

@jmshelby following should help

boot2docker ssh
sudo rm /var/lib/boot2docker/tls/server*.pem
sudo /etc/init.d/docker restart
exit
boot2docker shellinit

rooty0 commented Jun 19, 2015

@jmshelby following should help

boot2docker ssh
sudo rm /var/lib/boot2docker/tls/server*.pem
sudo /etc/init.d/docker restart
exit
boot2docker shellinit
@jmshelby

This comment has been minimized.

Show comment
Hide comment
@jmshelby

jmshelby Jun 19, 2015

@rooty0 as I explained, I already performed those steps (the ones mentioned by @mmikulicic), and they DO NOT WORK. After exiting from ssh'ing into boot2docker, and running boot2docker shellinit I get error in run: Error requesting socket: exit status 1

Even if that did work, it's something I would have to do every single time I restarted the b2d instance, it's not a solution

jmshelby commented Jun 19, 2015

@rooty0 as I explained, I already performed those steps (the ones mentioned by @mmikulicic), and they DO NOT WORK. After exiting from ssh'ing into boot2docker, and running boot2docker shellinit I get error in run: Error requesting socket: exit status 1

Even if that did work, it's something I would have to do every single time I restarted the b2d instance, it's not a solution

@talbright

This comment has been minimized.

Show comment
Hide comment
@talbright

talbright Jun 20, 2015

+1

The steps @mmikulicic mention did fix it for me.

talbright commented Jun 20, 2015

+1

The steps @mmikulicic mention did fix it for me.

@pdzilla

This comment has been minimized.

Show comment
Hide comment
@pdzilla

pdzilla Jun 20, 2015

+1, in the same boat as @jmshelby

pdzilla commented Jun 20, 2015

+1, in the same boat as @jmshelby

@mkmik

This comment has been minimized.

Show comment
Hide comment
@mkmik

mkmik Jun 20, 2015

@jmshelby, when you ssh on your boot2docker instance and issue ifconfig, on which interface is the incriminated ip?

(i.e. if the message is x509: certificate is valid for 127.0.0.1, 10.0.2.15, not 192.168.59.107 the ip I'm talking about would be 192.168.59.107).

The /etc/init.d/docker file lists the interfaces whose ips it will add to the generated certificate:

: ${CERT_INTERFACES:='docker0 eth0 eth1'}

When the certificate is missing, it will regenerate and then reuse the same cert at subsequent boots.

You can see the ips contained in the existing certificate with:

$ sudo openssl x509 -text -in /var/lib/boot2docker/tls/server.pem | grep DNS
  DNS:boot2docker, IP Address:127.0.0.1, IP Address:172.17.42.1, IP Address:10.0.2.15, IP Address:192.168.59.103

If the certificate is ok, then perhaps you have a different problem than what I had.

mkmik commented Jun 20, 2015

@jmshelby, when you ssh on your boot2docker instance and issue ifconfig, on which interface is the incriminated ip?

(i.e. if the message is x509: certificate is valid for 127.0.0.1, 10.0.2.15, not 192.168.59.107 the ip I'm talking about would be 192.168.59.107).

The /etc/init.d/docker file lists the interfaces whose ips it will add to the generated certificate:

: ${CERT_INTERFACES:='docker0 eth0 eth1'}

When the certificate is missing, it will regenerate and then reuse the same cert at subsequent boots.

You can see the ips contained in the existing certificate with:

$ sudo openssl x509 -text -in /var/lib/boot2docker/tls/server.pem | grep DNS
  DNS:boot2docker, IP Address:127.0.0.1, IP Address:172.17.42.1, IP Address:10.0.2.15, IP Address:192.168.59.103

If the certificate is ok, then perhaps you have a different problem than what I had.

@housebolt

This comment has been minimized.

Show comment
Hide comment
@housebolt

housebolt Jun 20, 2015

@mmikulicic I ran the following after a reset via boot2docker delete/init/up

sudo openssl x509 -text -in /var/lib/boot2docker/tls/server.pem | grep DNS
DNS:boot2docker, IP Address:127.0.0.1, IP Address:10.0.2.15, IP Address:192.168.59.107

The output was correct, with all three IP addresses.

However after I run boot2docker restart and try again I get this:

sudo openssl x509 -text -in /var/lib/boot2docker/tls/server.pem
DNS:boot2docker, IP Address:127.0.0.1, IP Address:10.0.2.15

The third IP is gone!

When I exit out of boot2docker and run docker version

docker version
Client version: 1.7.0
Client API version: 1.19
Go version (client): go1.4.2
Git commit (client): 0baf609
OS/Arch (client): darwin/amd64
An error occurred trying to connect: Get https://192.168.59.107:2376/v1.19/version: x509: certificate is valid for 127.0.0.1, 10.0.2.15, not 192.168.59.107

housebolt commented Jun 20, 2015

@mmikulicic I ran the following after a reset via boot2docker delete/init/up

sudo openssl x509 -text -in /var/lib/boot2docker/tls/server.pem | grep DNS
DNS:boot2docker, IP Address:127.0.0.1, IP Address:10.0.2.15, IP Address:192.168.59.107

The output was correct, with all three IP addresses.

However after I run boot2docker restart and try again I get this:

sudo openssl x509 -text -in /var/lib/boot2docker/tls/server.pem
DNS:boot2docker, IP Address:127.0.0.1, IP Address:10.0.2.15

The third IP is gone!

When I exit out of boot2docker and run docker version

docker version
Client version: 1.7.0
Client API version: 1.19
Go version (client): go1.4.2
Git commit (client): 0baf609
OS/Arch (client): darwin/amd64
An error occurred trying to connect: Get https://192.168.59.107:2376/v1.19/version: x509: certificate is valid for 127.0.0.1, 10.0.2.15, not 192.168.59.107
@scifisamurai

This comment has been minimized.

Show comment
Hide comment
@scifisamurai

scifisamurai Jun 21, 2015

I'm having the same issue as many above on a fresh install of boot2docker & docker I did just today (haven't used docker or boot2docker before on this computer). I can reproduce the error a similar fashion as the others above.

1 boot2docker stop
2 boot2docker delete
3 boot2docker init
4 boot2docker start
Waiting for VM and Docker daemon to start...
.........................oooooooooooooooooo
Started.
Writing /Users/< user here >/.boot2docker/certs/boot2docker-vm/ca.pem
Writing /Users/< user here >/.boot2docker/certs/boot2docker-vm/cert.pem
Writing /Users/< user here >/.boot2docker/certs/boot2docker-vm/key.pem

To connect the Docker client to the Docker daemon, please set:
export DOCKER_HOST=tcp://192.168.59.103:2376
export DOCKER_CERT_PATH=/Users//.boot2docker/certs/boot2docker-vm
export DOCKER_TLS_VERIFY=1

5 $(boot2docker shellinit)

Writing /Users/<user here>/.boot2docker/certs/boot2docker-vm/ca.pem
Writing /Users/<user here>/.boot2docker/certs/boot2docker-vm/cert.pem
Writing /Users/<user here>/.boot2docker/certs/boot2docker-vm/key.pem

6 docker run hello-world

Unable to find image 'hello-world:latest' locally
latest: Pulling from hello-world
a8219747be10: Pull complete 
91c95931e552: Already exists 
hello-world:latest: The image you are pulling has been verified. Important: image verification is a tech preview feature and should not be relied on to provide security.
Digest: sha256:aa03e5d0d5553b4c3473e89c8619cf79df368babd18681cf5daeb82aab55838d
Status: Downloaded newer image for hello-world:latest
Hello from Docker.
This message shows that your installation appears to be working correctly.

To generate this message, Docker took the following steps:
 1. The Docker client contacted the Docker daemon.
 2. The Docker daemon pulled the "hello-world" image from the Docker Hub.
    (Assuming it was not already locally available.)
 3. The Docker daemon created a new container from that image which runs the
    executable that produces the output you are currently reading.
 4. The Docker daemon streamed that output to the Docker client, which sent it
    to your terminal.

To try something more ambitious, you can run an Ubuntu container with:
 $ docker run -it ubuntu bash

For more examples and ideas, visit:
 http://docs.docker.com/userguide/

7 boot2docker stop
8 boot2docker start

Waiting for VM and Docker daemon to start...
......................ooooo
Started.
Writing /Users/<user here>/.boot2docker/certs/boot2docker-vm/ca.pem
Writing /Users/<user here>/.boot2docker/certs/boot2docker-vm/cert.pem
Writing /Users/<user here>/.boot2docker/certs/boot2docker-vm/key.pem
Your environment variables are already set correctly.

9 docker run hello-world

An error occurred trying to connect: Post https://192.168.59.103:2376/v1.19/containers/create: x509: certificate is valid for 127.0.0.1, 10.0.2.15, not 192.168.59.103

10 boot2docker ssh
11 docker@boot2docker:$ sudo openssl x509 -text -in /var/lib/boot2docker/tls/server.pem | grep DNS
DNS:boot2docker, IP Address:127.0.0.1, IP Address:10.0.2.15
12 docker@boot2docker:
$ cat /etc/init.d/docker |grep -i cert_inte

: ${CERT_INTERFACES:='eth0 eth1'}
        for interface in ${CERT_INTERFACES}; do

13 docker@boot2docker:~$ ifconfig eth1|grep -i inet\ addr
inet addr:192.168.59.103 Bcast:192.168.59.255 Mask:255.255.255.0
Hence the ip should be being brought up but it's not...

I tried deleting the server*.pem files & restarting docker while inside the ssh session, exiting and then doing 'boot2docker shellinit. and got the same error in run: Error requesting socket: exit status 1

14 boot2docker -v up

Boot2Docker-cli version: v1.6.2
Git commit: cb2c3bc
2015/06/20 23:23:23 executing: VBoxManage showvminfo boot2docker-vm --machinereadable
2015/06/20 23:23:23 executing: VBoxManage showvminfo boot2docker-vm --machinereadable
2015/06/20 23:23:23 executing: VBoxManage showvminfo boot2docker-vm --machinereadable
Waiting for VM and Docker daemon to start...
.Connecting to tcp://localhost:2022 (attempt #0)2015/06/20 23:23:23 executing: /usr/bin/ssh ssh -o IdentitiesOnly=yes -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=quiet -p 2022 -i /Users/<user here>/.ssh/id_boot2docker docker@localhost ip addr show dev eth1
SSH returned: 4: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 08:00:27:83:97:50 brd ff:ff:ff:ff:ff:ff
    inet 192.168.59.103/24 brd 192.168.59.255 scope global eth1
       valid_lft forever preferred_lft forever
    inet6 fe80::a00:27ff:fe83:9750/64 scope link 
       valid_lft forever preferred_lft forever

END SSH
VM Host-only IP address: 192.168.59.103
Waiting for Docker daemon to start...
o2015/06/20 23:23:24 executing: /usr/bin/ssh ssh -o IdentitiesOnly=yes -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=quiet -p 2022 -i /Users/dharris/.ssh/id_boot2docker docker@localhost grep tcp:// /proc/$(cat /var/run/docker.pid)/cmdline
cat: can't open '/var/run/docker.pid': No such file or directory
Error requesting socket: exit status 1
o2015/06/20 23:23:24 executing: /usr/bin/ssh ssh -o IdentitiesOnly=yes -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=quiet -p 2022 -i /Users/dharris/.ssh/id_boot2docker docker@localhost grep tcp:// /proc/$(cat /var/run/docker.pid)/cmdline
cat: can't open '/var/run/docker.pid': No such file or directory
Error requesting socket: exit status 1

so something very weird is going on....

scifisamurai commented Jun 21, 2015

I'm having the same issue as many above on a fresh install of boot2docker & docker I did just today (haven't used docker or boot2docker before on this computer). I can reproduce the error a similar fashion as the others above.

1 boot2docker stop
2 boot2docker delete
3 boot2docker init
4 boot2docker start
Waiting for VM and Docker daemon to start...
.........................oooooooooooooooooo
Started.
Writing /Users/< user here >/.boot2docker/certs/boot2docker-vm/ca.pem
Writing /Users/< user here >/.boot2docker/certs/boot2docker-vm/cert.pem
Writing /Users/< user here >/.boot2docker/certs/boot2docker-vm/key.pem

To connect the Docker client to the Docker daemon, please set:
export DOCKER_HOST=tcp://192.168.59.103:2376
export DOCKER_CERT_PATH=/Users//.boot2docker/certs/boot2docker-vm
export DOCKER_TLS_VERIFY=1

5 $(boot2docker shellinit)

Writing /Users/<user here>/.boot2docker/certs/boot2docker-vm/ca.pem
Writing /Users/<user here>/.boot2docker/certs/boot2docker-vm/cert.pem
Writing /Users/<user here>/.boot2docker/certs/boot2docker-vm/key.pem

6 docker run hello-world

Unable to find image 'hello-world:latest' locally
latest: Pulling from hello-world
a8219747be10: Pull complete 
91c95931e552: Already exists 
hello-world:latest: The image you are pulling has been verified. Important: image verification is a tech preview feature and should not be relied on to provide security.
Digest: sha256:aa03e5d0d5553b4c3473e89c8619cf79df368babd18681cf5daeb82aab55838d
Status: Downloaded newer image for hello-world:latest
Hello from Docker.
This message shows that your installation appears to be working correctly.

To generate this message, Docker took the following steps:
 1. The Docker client contacted the Docker daemon.
 2. The Docker daemon pulled the "hello-world" image from the Docker Hub.
    (Assuming it was not already locally available.)
 3. The Docker daemon created a new container from that image which runs the
    executable that produces the output you are currently reading.
 4. The Docker daemon streamed that output to the Docker client, which sent it
    to your terminal.

To try something more ambitious, you can run an Ubuntu container with:
 $ docker run -it ubuntu bash

For more examples and ideas, visit:
 http://docs.docker.com/userguide/

7 boot2docker stop
8 boot2docker start

Waiting for VM and Docker daemon to start...
......................ooooo
Started.
Writing /Users/<user here>/.boot2docker/certs/boot2docker-vm/ca.pem
Writing /Users/<user here>/.boot2docker/certs/boot2docker-vm/cert.pem
Writing /Users/<user here>/.boot2docker/certs/boot2docker-vm/key.pem
Your environment variables are already set correctly.

9 docker run hello-world

An error occurred trying to connect: Post https://192.168.59.103:2376/v1.19/containers/create: x509: certificate is valid for 127.0.0.1, 10.0.2.15, not 192.168.59.103

10 boot2docker ssh
11 docker@boot2docker:$ sudo openssl x509 -text -in /var/lib/boot2docker/tls/server.pem | grep DNS
DNS:boot2docker, IP Address:127.0.0.1, IP Address:10.0.2.15
12 docker@boot2docker:
$ cat /etc/init.d/docker |grep -i cert_inte

: ${CERT_INTERFACES:='eth0 eth1'}
        for interface in ${CERT_INTERFACES}; do

13 docker@boot2docker:~$ ifconfig eth1|grep -i inet\ addr
inet addr:192.168.59.103 Bcast:192.168.59.255 Mask:255.255.255.0
Hence the ip should be being brought up but it's not...

I tried deleting the server*.pem files & restarting docker while inside the ssh session, exiting and then doing 'boot2docker shellinit. and got the same error in run: Error requesting socket: exit status 1

14 boot2docker -v up

Boot2Docker-cli version: v1.6.2
Git commit: cb2c3bc
2015/06/20 23:23:23 executing: VBoxManage showvminfo boot2docker-vm --machinereadable
2015/06/20 23:23:23 executing: VBoxManage showvminfo boot2docker-vm --machinereadable
2015/06/20 23:23:23 executing: VBoxManage showvminfo boot2docker-vm --machinereadable
Waiting for VM and Docker daemon to start...
.Connecting to tcp://localhost:2022 (attempt #0)2015/06/20 23:23:23 executing: /usr/bin/ssh ssh -o IdentitiesOnly=yes -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=quiet -p 2022 -i /Users/<user here>/.ssh/id_boot2docker docker@localhost ip addr show dev eth1
SSH returned: 4: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 08:00:27:83:97:50 brd ff:ff:ff:ff:ff:ff
    inet 192.168.59.103/24 brd 192.168.59.255 scope global eth1
       valid_lft forever preferred_lft forever
    inet6 fe80::a00:27ff:fe83:9750/64 scope link 
       valid_lft forever preferred_lft forever

END SSH
VM Host-only IP address: 192.168.59.103
Waiting for Docker daemon to start...
o2015/06/20 23:23:24 executing: /usr/bin/ssh ssh -o IdentitiesOnly=yes -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=quiet -p 2022 -i /Users/dharris/.ssh/id_boot2docker docker@localhost grep tcp:// /proc/$(cat /var/run/docker.pid)/cmdline
cat: can't open '/var/run/docker.pid': No such file or directory
Error requesting socket: exit status 1
o2015/06/20 23:23:24 executing: /usr/bin/ssh ssh -o IdentitiesOnly=yes -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=quiet -p 2022 -i /Users/dharris/.ssh/id_boot2docker docker@localhost grep tcp:// /proc/$(cat /var/run/docker.pid)/cmdline
cat: can't open '/var/run/docker.pid': No such file or directory
Error requesting socket: exit status 1

so something very weird is going on....

@jmshelby

This comment has been minimized.

Show comment
Hide comment
@jmshelby

jmshelby Jun 21, 2015

I decided to just move on to kitematic (which seems to be a wrapper for docker-machine which seems to be a wrapper for b2d) so far I haven't had the same issues (which is strange because it seems to be the same thing undercover).

Looks like kitematic might be what they're moving toward - just takes a bit of time to figure out how to work things without that pesky gui.

jmshelby commented Jun 21, 2015

I decided to just move on to kitematic (which seems to be a wrapper for docker-machine which seems to be a wrapper for b2d) so far I haven't had the same issues (which is strange because it seems to be the same thing undercover).

Looks like kitematic might be what they're moving toward - just takes a bit of time to figure out how to work things without that pesky gui.

@lrafagnin

This comment has been minimized.

Show comment
Hide comment
@lrafagnin

lrafagnin Jun 21, 2015

Something weird happening on boot2docker start
"/etc/init.d/docker status" returns "Docker daemon is not running" even though docker is running
"sudo /etc/init.d/docker restart" doesn't work because it uses the method above to check is process is running, ends up removing the pid file and all goes pear shape
"sudo /etc/init.d/docker stop && sudo /etc/init.d/docker start" works though

boot2docker ssh
sudo /etc/init.d/docker stop && sudo /etc/init.d/docker start
exit

lrafagnin commented Jun 21, 2015

Something weird happening on boot2docker start
"/etc/init.d/docker status" returns "Docker daemon is not running" even though docker is running
"sudo /etc/init.d/docker restart" doesn't work because it uses the method above to check is process is running, ends up removing the pid file and all goes pear shape
"sudo /etc/init.d/docker stop && sudo /etc/init.d/docker start" works though

boot2docker ssh
sudo /etc/init.d/docker stop && sudo /etc/init.d/docker start
exit

@housebolt

This comment has been minimized.

Show comment
Hide comment
@housebolt

housebolt Jun 21, 2015

@lrafagnin I can confirm your comments on the pid file and /etc/init.d/docker start, I ran into this yesterday while trying to debug

housebolt commented Jun 21, 2015

@lrafagnin I can confirm your comments on the pid file and /etc/init.d/docker start, I ran into this yesterday while trying to debug

@gesellix

This comment has been minimized.

Show comment
Hide comment
@gesellix

gesellix Jun 21, 2015

Contributor

the /var/log/boot2docker.log shows the following output for me:

...

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: dummy0: <BROADCAST,NOARP> mtu 1500 qdisc noop state DOWN group default 
    link/ether 26:08:5d:72:12:81 brd ff:ff:ff:ff:ff:ff
3: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 08:00:27:5a:72:f2 brd ff:ff:ff:ff:ff:ff
    inet 10.0.2.15/24 brd 10.0.2.255 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::a00:27ff:fe5a:72f2/64 scope link tentative 
       valid_lft forever preferred_lft forever
4: eth1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 08:00:27:c7:75:1b brd ff:ff:ff:ff:ff:ff
-------------------
Need TLS certs for boot2docker,127.0.0.1,10.0.2.15
-------------------

Looks like eth1 is DOWN during the initial boot, so that its ip cannot be used for the certs creation step. Would it be enough to make the docker init script be dependent on all interface initializations?

Contributor

gesellix commented Jun 21, 2015

the /var/log/boot2docker.log shows the following output for me:

...

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: dummy0: <BROADCAST,NOARP> mtu 1500 qdisc noop state DOWN group default 
    link/ether 26:08:5d:72:12:81 brd ff:ff:ff:ff:ff:ff
3: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 08:00:27:5a:72:f2 brd ff:ff:ff:ff:ff:ff
    inet 10.0.2.15/24 brd 10.0.2.255 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::a00:27ff:fe5a:72f2/64 scope link tentative 
       valid_lft forever preferred_lft forever
4: eth1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 08:00:27:c7:75:1b brd ff:ff:ff:ff:ff:ff
-------------------
Need TLS certs for boot2docker,127.0.0.1,10.0.2.15
-------------------

Looks like eth1 is DOWN during the initial boot, so that its ip cannot be used for the certs creation step. Would it be enough to make the docker init script be dependent on all interface initializations?

@gesellix

This comment has been minimized.

Show comment
Hide comment
@gesellix

gesellix Jun 21, 2015

Contributor

I made it work on my machine by adding the following to /var/lib/boot2docker/profile:

wait4eth1() {
        CNT=0
        until ip a show eth1 | grep -q UP
        do
                [ $((CNT++)) -gt 60 ] && break || sleep 1
        done
        sleep 1
}
wait4eth1

credits go to @fayep for PR #411

Contributor

gesellix commented Jun 21, 2015

I made it work on my machine by adding the following to /var/lib/boot2docker/profile:

wait4eth1() {
        CNT=0
        until ip a show eth1 | grep -q UP
        do
                [ $((CNT++)) -gt 60 ] && break || sleep 1
        done
        sleep 1
}
wait4eth1

credits go to @fayep for PR #411

@caleb

This comment has been minimized.

Show comment
Hide comment
@caleb

caleb Jun 21, 2015

@gesellix Thank you! I've been running into this problem sporadically now, and I never had the time to dive into fixing it.

I've created a boot2docker image building repository to fix this issue (and to change the permissions of mounted volumes from your mac to your containers to 777 to allow containers to write to files and directories)

https://github.com/caleb/boot2docker-caleb

caleb commented Jun 21, 2015

@gesellix Thank you! I've been running into this problem sporadically now, and I never had the time to dive into fixing it.

I've created a boot2docker image building repository to fix this issue (and to change the permissions of mounted volumes from your mac to your containers to 777 to allow containers to write to files and directories)

https://github.com/caleb/boot2docker-caleb

@gesellix

This comment has been minimized.

Show comment
Hide comment
@gesellix

gesellix Jun 21, 2015

Contributor

the fix also applies for boot2docker-windows.

Contributor

gesellix commented Jun 21, 2015

the fix also applies for boot2docker-windows.

@dremonkey

This comment has been minimized.

Show comment
Hide comment
@dremonkey

dremonkey Jun 21, 2015

@gesellix Only solution that worked for me too. Thanks!

dremonkey commented Jun 21, 2015

@gesellix Only solution that worked for me too. Thanks!

@garthk

This comment has been minimized.

Show comment
Hide comment
@garthk

garthk Jun 22, 2015

Short-cut:

boot2docker ssh
sudo curl -o /var/lib/boot2docker/profile https://gist.githubusercontent.com/garthk/d5a17007c277aa5c76de/raw/3d09c77aae38b4f2809d504784965f5a16f2de4c/profile
sudo halt
boot2docker up

All credit to @fayep for PR #411 and @gesellix for pointing it out. Slower documentation in https://gist.github.com/garthk/d5a17007c277aa5c76de

WARNING: boot2docker didn't come up for me on the third run-through. I'm attributing it to the VirtualBox-inflicted kernel panics. Manual power-off and restart in VirtualBox yielded a working docker, after which boot2docker down and up did the same.

garthk commented Jun 22, 2015

Short-cut:

boot2docker ssh
sudo curl -o /var/lib/boot2docker/profile https://gist.githubusercontent.com/garthk/d5a17007c277aa5c76de/raw/3d09c77aae38b4f2809d504784965f5a16f2de4c/profile
sudo halt
boot2docker up

All credit to @fayep for PR #411 and @gesellix for pointing it out. Slower documentation in https://gist.github.com/garthk/d5a17007c277aa5c76de

WARNING: boot2docker didn't come up for me on the third run-through. I'm attributing it to the VirtualBox-inflicted kernel panics. Manual power-off and restart in VirtualBox yielded a working docker, after which boot2docker down and up did the same.

@warden

This comment has been minimized.

Show comment
Hide comment
@warden

warden Jun 22, 2015

Just upgraded to 1.7.0 and this problem surfaced.
The @garthk suggestion worked although the boot2docker up command hung. After forcing Virtualbox poweroff and starting up again all seemed to work fine. MacOS.

warden commented Jun 22, 2015

Just upgraded to 1.7.0 and this problem surfaced.
The @garthk suggestion worked although the boot2docker up command hung. After forcing Virtualbox poweroff and starting up again all seemed to work fine. MacOS.

@ringods

This comment has been minimized.

Show comment
Hide comment
@ringods

ringods Jun 22, 2015

@garthk thanks for the fix. Works like a charm.

ringods commented Jun 22, 2015

@garthk thanks for the fix. Works like a charm.

@gesellix

This comment has been minimized.

Show comment
Hide comment
@gesellix

gesellix Jun 22, 2015

Contributor

thanks @garthk for the convenient gist!
After applying the patch it was enough for me to restart the docker daemon with sudo /etc/init.d/docker restart (or from outside of the vm: boot2docker ssh sudo /etc/init.d/docker restart).

Contributor

gesellix commented Jun 22, 2015

thanks @garthk for the convenient gist!
After applying the patch it was enough for me to restart the docker daemon with sudo /etc/init.d/docker restart (or from outside of the vm: boot2docker ssh sudo /etc/init.d/docker restart).

@cjquinon

This comment has been minimized.

Show comment
Hide comment
@cjquinon

cjquinon Jul 2, 2015

@garthk, thanks for the solution. Worked like a charm!

cjquinon commented Jul 2, 2015

@garthk, thanks for the solution. Worked like a charm!

@maerten

This comment has been minimized.

Show comment
Hide comment
@maerten

maerten Jul 6, 2015

SvenDowiteit's solution worked for me:

boot2docker ssh sudo /etc/init.d/docker restart

maerten commented Jul 6, 2015

SvenDowiteit's solution worked for me:

boot2docker ssh sudo /etc/init.d/docker restart
@rupakg

This comment has been minimized.

Show comment
Hide comment
@rupakg

rupakg Jul 7, 2015

Solution stated by @garthk WORKS! Thanks to @fayep for PR #411 and @gesellix.

rupakg commented Jul 7, 2015

Solution stated by @garthk WORKS! Thanks to @fayep for PR #411 and @gesellix.

@dennybritz

This comment has been minimized.

Show comment
Hide comment
@dennybritz

dennybritz Jul 12, 2015

boot2docker ssh sudo /etc/init.d/docker restart

Worked for me.

dennybritz commented Jul 12, 2015

boot2docker ssh sudo /etc/init.d/docker restart

Worked for me.

@loulin

This comment has been minimized.

Show comment
Hide comment
@loulin

loulin Jul 14, 2015

@phact's solution temporarily worked.

loulin commented Jul 14, 2015

@phact's solution temporarily worked.

@mcobzarenco

This comment has been minimized.

Show comment
Hide comment
@mcobzarenco

mcobzarenco commented Jul 14, 2015

+1

@tianon

This comment has been minimized.

Show comment
Hide comment
@tianon

tianon Jul 14, 2015

Contributor

Can anyone who's able to reproduce this test with v1.7.1-rc3 (https://github.com/tianon/boot2docker/releases/tag/v1.7.1-rc3) and verify that #960 fixed it?

Contributor

tianon commented Jul 14, 2015

Can anyone who's able to reproduce this test with v1.7.1-rc3 (https://github.com/tianon/boot2docker/releases/tag/v1.7.1-rc3) and verify that #960 fixed it?

@tianon

This comment has been minimized.

Show comment
Hide comment
@tianon

tianon Jul 14, 2015

Contributor

Thanks to @nathanleclaire, we've verified that #960 is the fix. Let's build us an official 1.7.1 release and finally put this to bed! 👍

Contributor

tianon commented Jul 14, 2015

Thanks to @nathanleclaire, we've verified that #960 is the fix. Let's build us an official 1.7.1 release and finally put this to bed! 👍

@tianon tianon closed this Jul 14, 2015

@charford

This comment has been minimized.

Show comment
Hide comment
@charford

charford Jul 14, 2015

Finally this is being fixed. I verified this fix two weeks ago in the original PR #960. It's very disappointing to see it has taken over two weeks for this to be released. Considering how frequent this bug was encountered, I am surprised it was not caught before the release.

charford commented Jul 14, 2015

Finally this is being fixed. I verified this fix two weeks ago in the original PR #960. It's very disappointing to see it has taken over two weeks for this to be released. Considering how frequent this bug was encountered, I am surprised it was not caught before the release.

@nathanleclaire

This comment has been minimized.

Show comment
Hide comment
@nathanleclaire

nathanleclaire Jul 15, 2015

Contributor

Really sorry for any inconvenience encountered in the meantime, everyone. In our excitement to migrate to Docker Machine, unfortunately this one slipped through the cracks.

Contributor

nathanleclaire commented Jul 15, 2015

Really sorry for any inconvenience encountered in the meantime, everyone. In our excitement to migrate to Docker Machine, unfortunately this one slipped through the cracks.

@liusongsen

This comment has been minimized.

Show comment
Hide comment
@liusongsen

liusongsen Jul 16, 2015

我是这样解决问题的:
boot2docker ssh sudo /etc/init.d/docker restart
非常感谢dennybritz

liusongsen commented Jul 16, 2015

我是这样解决问题的:
boot2docker ssh sudo /etc/init.d/docker restart
非常感谢dennybritz

@jakirkham

This comment has been minimized.

Show comment
Hide comment
@jakirkham

jakirkham Jul 16, 2015

@liusongsen, just upgrade. The fix has been included in version 1.7.1.

jakirkham commented Jul 16, 2015

@liusongsen, just upgrade. The fix has been included in version 1.7.1.

@Laxman-SM

This comment has been minimized.

Show comment
Hide comment
@Laxman-SM

Laxman-SM Jul 18, 2015

@jakirkham Fix version work for me. Thank.

Laxman-SM commented Jul 18, 2015

@jakirkham Fix version work for me. Thank.

@mariowise

This comment has been minimized.

Show comment
Hide comment
@mariowise

mariowise Jul 18, 2015

I've got version 1.7.1 and the issue remains :( I try the wait4eth1() and nothing :S

mariowise commented Jul 18, 2015

I've got version 1.7.1 and the issue remains :( I try the wait4eth1() and nothing :S

@Laxman-SM

This comment has been minimized.

Show comment
Hide comment
@Laxman-SM

Laxman-SM Jul 19, 2015

@mariowise, My OS X version 10.10.4 and fix working with out any issue. may be you need to update your OS X version.

Kernel Version: 4.0.7-boot2docker
Operating System: Boot2Docker 1.7.1 (TCL 6.3); master : c202798 - Wed Jul 15 00:16:02 UTC 2015

Laxman-SM commented Jul 19, 2015

@mariowise, My OS X version 10.10.4 and fix working with out any issue. may be you need to update your OS X version.

Kernel Version: 4.0.7-boot2docker
Operating System: Boot2Docker 1.7.1 (TCL 6.3); master : c202798 - Wed Jul 15 00:16:02 UTC 2015

@jakirkham

This comment has been minimized.

Show comment
Hide comment
@jakirkham

jakirkham Jul 20, 2015

@mariowise, are you sure the VM is actually using this version. If not, could you try the following and see if it makes any difference?

boot2docker destory && boot2docker init && boot2docker up

Also, you may find that you need to reboot the VM when switching networks. That is unfortunately a different unresolved problem ( #945 ).

jakirkham commented Jul 20, 2015

@mariowise, are you sure the VM is actually using this version. If not, could you try the following and see if it makes any difference?

boot2docker destory && boot2docker init && boot2docker up

Also, you may find that you need to reboot the VM when switching networks. That is unfortunately a different unresolved problem ( #945 ).

@dennybritz

This comment has been minimized.

Show comment
Hide comment
@dennybritz

dennybritz Jul 20, 2015

Fix in 1.7.1 worked for me.

dennybritz commented Jul 20, 2015

Fix in 1.7.1 worked for me.

@mariowise

This comment has been minimized.

Show comment
Hide comment
@mariowise

mariowise Jul 20, 2015

Well it was important to get this to work in OSX 10.9, I reboot the Mac, reinstall and then use the this, and then it worked fine.
Thanks! (I'm on OSX 10.9.5 + boot2docker 1.7.1)

mariowise commented Jul 20, 2015

Well it was important to get this to work in OSX 10.9, I reboot the Mac, reinstall and then use the this, and then it worked fine.
Thanks! (I'm on OSX 10.9.5 + boot2docker 1.7.1)

@jakirkham

This comment has been minimized.

Show comment
Hide comment
@jakirkham

jakirkham Jul 20, 2015

@Laxman-SM, I have the same specs as @mariowise and the update works for me. I think something else is going on.

jakirkham commented Jul 20, 2015

@Laxman-SM, I have the same specs as @mariowise and the update works for me. I think something else is going on.

@liamdawson

This comment has been minimized.

Show comment
Hide comment
@liamdawson

liamdawson Jul 26, 2015

1.7.1 works perfectly for me.

liamdawson commented Jul 26, 2015

1.7.1 works perfectly for me.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment