v0.14.0 Builder Start Failure: No such id: deis/builder:v0.14.0

[jorihardman]

Just commented on #1924 since the symptoms are the same, but I think the cause is probably different at this point.

I installed deisctl 0.14.0 and ensured that my unit files were all refreshed to 0.14.0. I fired up a new cluster on digitalocean using a new discovery url and the provision script in the v0.14.0 tag. Install went fine, but starting the builder failed. Output is below.

Oct 23 18:36:35 deis-3 systemd[1]: Starting deis-builder...
Oct 23 18:36:35 deis-3 sh[967]: Unable to find image 'ubuntu-debootstrap:14.04' locally
Oct 23 18:36:35 deis-3 sh[967]: Repository ubuntu-debootstrap already being pulled by another client. Waiting.
Oct 23 18:36:44 deis-3 sh[1303]: 2014/10/23 18:36:44 Error response from daemon: No such id: deis/builder:v0.14.0
Oct 23 18:36:44 deis-3 sh[1303]: Pulling repository deis/builder
Oct 23 18:41:42 deis-3 sh[1303]: Status: Downloaded newer image for deis/builder:v0.14.0
Oct 23 18:41:42 deis-3 sh[4118]: Error: No such image or container: deis-builder
Oct 23 18:41:42 deis-3 sh[4130]: Waiting for builder on 2223/tcp...
Oct 23 18:41:54 deis-3 sh[4129]: 2014-10-23T18:41:54Z b1f9f53ea09d confd[77]: ERROR Config check failed: exit status 1
Oct 23 18:41:54 deis-3 sh[4129]: 2014-10-23T18:41:54Z b1f9f53ea09d confd[77]: ERROR Config check failed: exit status 1
Oct 23 18:41:54 deis-3 sh[4129]: builder: waiting for confd to write initial templates...
Oct 23 18:41:59 deis-3 sh[4129]: 2014-10-23T18:41:59Z b1f9f53ea09d confd[108]: ERROR Config check failed: exit status 1
Oct 23 18:41:59 deis-3 sh[4129]: 2014-10-23T18:41:59Z b1f9f53ea09d confd[108]: ERROR Config check failed: exit status 1
Oct 23 18:41:59 deis-3 sh[4129]: builder: waiting for confd to write initial templates...

[bacongobbler]

dupe of #2235. Can you please try deisctl config platform set domain=<domain>?

[jorihardman]

I thought I had checked the issues fairly well, but definitely missed that one. Have a working cluster now, thanks!

[jorihardman]

I thought this issue was fixed, but I guess not. I left a working 3-machine deis cluster on Friday, and when I came back this morning, all my deis client commands resulted in a controller not found error. I restarted the controller, and still no luck. I uninstalled the platform, and on reinstall, I'm once again stuck here.

deisctl config platform get domain shows that my domain is set properly, however all I get is the following over and over:

Oct 27 16:06:59 deis-2 sh[12696]: 2014-10-27T16:06:59Z aec84aaba8ba confd[1727]: ERROR Config check failed: exit status 1
Oct 27 16:06:59 deis-2 sh[12696]: builder: waiting for confd to write initial templates

[jorihardman]

A quick update on this. After using fleetctl list-units I noticed that the builder and database units were still running after the platform uninstall. I did a fleetctl destroy for both, performed another install, and am now back up again.

[carmstrong]

@jorihardman Your destroy issue is likely resolved by #2280 - we are now checking destroy state.

[adrianmoya]

I've just hit this issue from my second installation at #2300

systemctl status deis-builder
● deis-builder.service - deis-builder
   Loaded: loaded (/run/fleet/units/deis-builder.service; linked-runtime)
   Active: activating (start-post) since Thu 2014-10-30 00:10:11 UTC; 19min ago
  Process: 15659 ExecStopPost=/usr/bin/docker rm -f deis-builder (code=exited, status=0/SUCCESS)
  Process: 15742 ExecStartPre=/bin/sh -c docker inspect deis-builder >/dev/null && docker rm -f deis-builder || true (code=exited, status=0/SUCCESS)
  Process: 15717 ExecStartPre=/bin/sh -c IMAGE=`/run/deis/bin/get_image /deis/builder` && docker history $IMAGE >/dev/null || docker pull $IMAGE (code=exited, status=0/SUCCESS)
  Process: 15708 ExecStartPre=/bin/sh -c docker inspect deis-builder-data >/dev/null 2>&1 || docker run --name deis-builder-data -v /var/lib/docker ubuntu-debootstrap:14.04 /bin/true (code=exited, status=0/SUCCESS)
 Main PID: 15753 (sh);         : 15754 (sh)
   CGroup: /system.slice/deis-builder.service
           ├─15753 /bin/sh -c IMAGE=`/run/deis/bin/get_image /deis/builder` && docker run --name deis-builder --rm -p 2223:22 --volumes-from=deis-builder-data -e EXTERNAL_PORT=2223 -e HOST=$COREOS_PRIVAT...
           ├─15775 docker run --name deis-builder --rm -p 2223:22 --volumes-from=deis-builder-data -e EXTERNAL_PORT=2223 -e HOST=10.21.12.10 --privileged deis/builder:v0.15.0
           └─control
             ├─15754 /bin/sh -c echo 'Waiting for builder on 2223/tcp...' && until echo 'dummy-value' | ncat $COREOS_PRIVATE_IPV4 2223 >/dev/null 2>&1; do sleep 1; done
             └─28908 sleep 1

Oct 30 00:28:59 deis-2036302f-67e2-4c66-a609-9d0fb9b97bcf.novalocal sh[15753]: builder: waiting for confd to write initial templates...
Oct 30 00:29:04 deis-2036302f-67e2-4c66-a609-9d0fb9b97bcf.novalocal sh[15753]: 2014-10-30T00:29:04Z 1749321ebb67 confd[3330]: ERROR Config check failed: exit status 1
Oct 30 00:29:04 deis-2036302f-67e2-4c66-a609-9d0fb9b97bcf.novalocal sh[15753]: 2014-10-30T00:29:04Z 1749321ebb67 confd[3330]: ERROR Config check failed: exit status 1
Oct 30 00:29:04 deis-2036302f-67e2-4c66-a609-9d0fb9b97bcf.novalocal sh[15753]: builder: waiting for confd to write initial templates...
Oct 30 00:29:09 deis-2036302f-67e2-4c66-a609-9d0fb9b97bcf.novalocal sh[15753]: 2014-10-30T00:29:09Z 1749321ebb67 confd[3345]: ERROR Config check failed: exit status 1
Oct 30 00:29:09 deis-2036302f-67e2-4c66-a609-9d0fb9b97bcf.novalocal sh[15753]: 2014-10-30T00:29:09Z 1749321ebb67 confd[3345]: ERROR Config check failed: exit status 1
Oct 30 00:29:09 deis-2036302f-67e2-4c66-a609-9d0fb9b97bcf.novalocal sh[15753]: builder: waiting for confd to write initial templates...
Oct 30 00:29:14 deis-2036302f-67e2-4c66-a609-9d0fb9b97bcf.novalocal sh[15753]: 2014-10-30T00:29:14Z 1749321ebb67 confd[3360]: ERROR Config check failed: exit status 1
Oct 30 00:29:14 deis-2036302f-67e2-4c66-a609-9d0fb9b97bcf.novalocal sh[15753]: 2014-10-30T00:29:14Z 1749321ebb67 confd[3360]: ERROR Config check failed: exit status 1
Oct 30 00:29:14 deis-2036302f-67e2-4c66-a609-9d0fb9b97bcf.novalocal sh[15753]: builder: waiting for confd to write initial templates...

It may be because of the first uninstall. I'll try again uninstalling with the deisctl compiled from master and verify everything cleans up completly