Unable to Deploy on v2.8.0 ontop of Digitalocean K8 Cluster

[davidchua]

Server: Ubuntu 16.04
K8 Server Version: 1.4.1
Cluster Provider: Digitalocean
Cluster brought up by kubeadm
Deis Workflow version: v2.8.0

I’m trying to do a git push deis master on a deis cluster that lives ontop a kubeadm cluster hosted on DO.

whenever I do a push, I get the following error:

Counting objects: 105, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (61/61), done.
Writing objects: 100% (105/105), 23.25 KiB | 0 bytes/s, done.
Total 105 (delta 41), reused 100 (delta 39)
remote: Resolving deltas: 100% (41/41), done.
Starting build... but first, coffee!
remote: 2016/11/01 08:21:55 Error running git receive hook [attempting to stream logs (Get https://kube-minion-01:10250/containerLogs/deis/slugbuild-go-c3764da7-9c58a84e/deis-slugbuilder?follow=true: dial tcp: lookup kube-minion-01 on 8.8.8.8:53: no such host)]
To ssh://git@deis-builder<ip>.nip.io:2222/go.git
 ! [remote rejected] master -> master (pre-receive hook declined)
error: failed to push some refs to 'ssh://git@deis-builder.<ip>.nip.io:2222/go.git'

deis          slugbuild-go-c3764da7-9c58a84e           0/1       Completed          0          2m

When looking at the slugbuild logs, it looks like the build was successful

root@kube-minion-01:~# docker logs d79358bae23a
-----> Restoring cache...
       Done!
-----> Go app detected
-----> Checking Godeps/Godeps.json file.
-----> Using go1.7.1
-----> Running: go install -v -tags heroku .
       github.com/deis/example-go
-----> Discovering process types
       Procfile declares types -> web
-----> Checking for changes inside the cache directory...
       Cache unchanged, not updating
-----> Compiled slug size is 1.9M

deis-builder logs:

receiving git repo name: go.git, operation: git-receive-pack, fingerprint: a5:5e:37:7b:6e:91:54:56:11:95:9b:15:06:9f:0a:2d, user: admin
creating repo directory /home/git/go.git
writing pre-receive hook under /home/git/go.git
git-shell -c git-receive-pack 'go.git'
Waiting for git-receive to run.
Waiting for deploy.
Deploy complete.

deis-controller

10.36.0.4 "GET /v2/hooks/key/a5:5e:37:7b:6e:91:54:56:11:95:9b:15:06:9f:0a:2d HTTP/1.1" 200 50 "deis-builder"
10.36.0.4 "POST /v2/hooks/config/ HTTP/1.1" 200 214 "deis-builder"
10.36.0.4 "GET /v2/hooks/key/a5:5e:37:7b:6e:91:54:56:11:95:9b:15:06:9f:0a:2d HTTP/1.1" 200 50 "deis-builder"
10.36.0.4 "POST /v2/hooks/config/ HTTP/1.1" 200 214 "deis-builder”

The logs having issue the no such host error (due to kubernetes/kubernetes#33718) is a known issue. I used docker logs to get the above logs

[bacongobbler]

The builder follows the slugbuilder container's logs in order to display the build output similar to Heroku. What would be the expected output here in cases when the builder cannot fetch the logs from slugbuilder?

[bacongobbler]

If it's an upstream issue via kubernetes/kubernetes#33718 then personally I'm more prone to suggest upgrading to a release with that fix if there's no actionable items required otherwise. Anything we implement in Workflow will just be a workaround for this bug.

[davidchua]

The issue is that the deploy stops, and there's no pods created in the application namespace. Its as though a deploy didn't happen.

[bacongobbler]

That is intentional. If the builder cannot retrieve logs then it fails and backs out. If you feel up to it you can take a crack at a PR to work around the no such host issue, but it'd be a temporary thing until the upstream issue is resolved and a release is issued.

[bacongobbler]

Looks like the issue has been resolved upstream in 1.5.0-alpha.1, so there's nothing we need to do on our end. If you can test running Workflow on 1.5.0-beta.1 and confirm it works for you, then I think we can close this one out.

[bacongobbler]

I'm going to close this ticket due to inactivity, but please re-open if this still needs to be addressed. Thanks!