feat(charts): introduce kube-registry-proxy

[bacongobbler]

I didn't see a requirements.lock so I assume that's handled by CI.

closes #644

[krisnova]

Has anybody been able to manually test this yet?

[mboersma]

Has anybody been able to manually test this yet?

@knova I'll do so today.

[mboersma]

Wonder if there's a way to silence this warning:

$ helm install --namespace deis -n deis ./workflow/
Warning: kube-registry-proxy is in requirements.yaml but not in the charts/ directory!

[vdice]

I kicked off a workflow chart publish job to get a chart artifact from this PR. We can now helm repo up && helm install --namespace deis workflow-pr/workflow --version v2.11.1-20170228230834-sha.526682b for testing this chart/change. (e2e will run as well)

[vdice]

Looks like the e2e run encountered errors, specifically around git push deis master cases (https://ci.deis.io/job/workflow-chart-e2e/813/console); I'll test manually as well.

[vdice]

Looks like a correlating change to builder/dockerbuilder/other may be needed in tandem with this, perhaps specifically for Dockerfile-based apps. Hitting the following from the example-dockerfile-http app:

$ git push deis master
The authenticity of host '[deis-builder.130.211.223.241.nip.io]:2222 ([130.211.223.241]:2222)' can't be established.
ECDSA key fingerprint is SHA256:5tKZmwLiTG/qA4xmj4DExtyJljGbDfBwRT5sP5uXpE4.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '[deis-builder.130.211.223.241.nip.io]:2222,[130.211.223.241]:2222' (ECDSA) to the list of known hosts.
Counting objects: 90, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (58/58), done.
Writing objects: 100% (90/90), 32.05 KiB | 0 bytes/s, done.
Total 90 (delta 24), reused 81 (delta 21)
Starting build... but first, coffee!
Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/docker/client.py", line 170, in _raise_for_status
    response.raise_for_status()
  File "/usr/local/lib/python3.5/dist-packages/requests/models.py", line 909, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 500 Server Error: Internal Server Error for url: http+docker://localunixsocket/v1.23/build?nocache=False&t=127.0.0.1%3A%2Fedh%3Agit-d16a94de&forcerm=False&rm=True&q=False&pull=True

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/deploy.py", line 116, in <module>
    log_output(stream, True)
  File "/deploy.py", line 16, in log_output
    for chunk in stream:
  File "/usr/local/lib/python3.5/dist-packages/docker/client.py", line 268, in _stream_helper
    yield self._result(response, json=decode)
  File "/usr/local/lib/python3.5/dist-packages/docker/client.py", line 178, in _result
    self._raise_for_status(response)
  File "/usr/local/lib/python3.5/dist-packages/docker/client.py", line 174, in _raise_for_status
    raise errors.APIError(e, response, explanation=explanation)
docker.errors.APIError: 500 Server Error: Internal Server Error ("b'Error parsing reference: "127.0.0.1:/edh:git-d16a94de" is not a valid repository/tag'")
remote: 2017/03/01 17:26:42 Error running git receive hook [Build pod exited with code 1, stopping build.]
To ssh://git@deis-builder.130.211.223.241.nip.io:2222/edh.git
 ! [remote rejected] master -> master (pre-receive hook declined)
error: failed to push some refs to 'ssh://git@deis-builder.130.211.223.241.nip.io:2222/edh.git'

(Same git push deis master from Procfile-based app example-go is successful.)

[bacongobbler]

Hmm, that error makes me think that the service port is unavailable in dockerbuilder's environment when it hits https://github.com/deis/dockerbuilder/blob/master/rootfs/deploy.py#L62-L63.

Is there a way you can retrieve the available envvars from the dockerbuilder container by uploading a Dockerfile with a RUN sleep 600 in there? That should give you more than enough time to do a kubectl exec to check the running environment.

[vdice]

@bacongobbler it seems like it doesn't even get to processing the Dockerfile... I can't get the dockerbuilder pod to stay up with a minimal Dockerfile:

FROM alpine:3.5

RUN sleep 600

As immediately after git push master I see:

$ git push deis master
Counting objects: 99, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (66/66), done.
Writing objects: 100% (99/99), 33.84 KiB | 0 bytes/s, done.
Total 99 (delta 29), reused 81 (delta 21)
Starting build... but first, coffee!
remote: 2017/03/01 18:31:52 Error running git receive hook [watching events for builder pod startup (Giving up; pod went into failed status:
remote: []:)]
To ssh://git@deis-builder.130.211.223.241.nip.io:2222/edh.git
 ! [remote rejected] master -> master (pre-receive hook declined)
error: failed to push some refs to 'ssh://git@deis-builder.130.211.223.241.nip.io:2222/edh.git'

And dockerbuild pod logs show the aforementioned error:

$ kd logs dockerbuild-edh-7d2cfb0b-a02ecb29
Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/docker/client.py", line 170, in _raise_for_status
    response.raise_for_status()
  File "/usr/local/lib/python3.5/dist-packages/requests/models.py", line 909, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 500 Server Error: Internal Server Error for url: http+docker://localunixsocket/v1.23/build?q=False&forcerm=False&t=127.0.0.1%3A%2Fedh%3Agit-7d2cfb0b&nocache=False&pull=True&rm=True

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/deploy.py", line 116, in <module>
    log_output(stream, True)
  File "/deploy.py", line 16, in log_output
    for chunk in stream:
  File "/usr/local/lib/python3.5/dist-packages/docker/client.py", line 268, in _stream_helper
    yield self._result(response, json=decode)
  File "/usr/local/lib/python3.5/dist-packages/docker/client.py", line 178, in _result
    self._raise_for_status(response)
  File "/usr/local/lib/python3.5/dist-packages/docker/client.py", line 174, in _raise_for_status
    raise errors.APIError(e, response, explanation=explanation)
docker.errors.APIError: 500 Server Error: Internal Server Error ("b'Error parsing reference: "127.0.0.1:/edh:git-7d2cfb0b" is not a valid repository/tag'")

It definitely appears from "127.0.0.1:/edh:git-7d2cfb0b" that DEIS_REGISTRY_SERVICE_PORT used in dockerbuilder's deploy.py is not defined when we expect it to be.

Perhaps the crux lies in the values used here -- instead of getting values from env vars should they be hardcoded (but still overrideable)? Or, when/how are those two env vars expected to be populated when installing the workflow chart?

[vdice]

With latest update, no longer reproducing #734 (comment) -- pushing to the internal registry now works as intended (host and port now getting through to builder container/pod). LGTM.

[meonthewire]

Hello everyone, I did the update to the version 2.14.0 but I'm still getting this error when I run git push deis master

Counting objects: 1833, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (859/859), done.
Writing objects: 100% (1833/1833), 663.84 KiB | 0 bytes/s, done.
Total 1833 (delta 920), reused 1823 (delta 915)
remote: Resolving deltas: 100% (920/920), done.
Starting build... but first, coffee!
Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/docker/client.py", line 170, in _raise_for_status
    response.raise_for_status()
  File "/usr/local/lib/python3.5/dist-packages/requests/models.py", line 909, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 500 Server Error: Internal Server Error for url: http+docker://localunixsocket/v1.24/build?forcerm=False&rm=True&q=False&nocache=False&pull=True&t=%2Fknotty-vineyard%3Agit-6979c2e8

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/deploy.py", line 124, in <module>
    log_output(stream, True)
  File "/deploy.py", line 16, in log_output
    for chunk in stream:
  File "/usr/local/lib/python3.5/dist-packages/docker/client.py", line 268, in _stream_helper
    yield self._result(response, json=decode)
  File "/usr/local/lib/python3.5/dist-packages/docker/client.py", line 178, in _result
    self._raise_for_status(response)
  File "/usr/local/lib/python3.5/dist-packages/docker/client.py", line 174, in _raise_for_status
    raise errors.APIError(e, response, explanation=explanation)
docker.errors.APIError: 500 Server Error: Internal Server Error ("b'{"message":"Error parsing reference: \\"/knotty-vineyard:git-6979c2e8\\" is not a valid repository/tag"}'")
remote: 2017/05/09 17:38:20 Error running git receive hook [Build pod exited with code 1, stopping build.]

[bacongobbler]

Can you open a new issue for that? Looks unrelated to this pull request. Thanks!