New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor cluster/juju #22726

Merged
merged 11 commits into from Mar 23, 2016

Conversation

Projects
None yet
10 participants
@lazypower
Member

lazypower commented Mar 9, 2016

Apologies in advance for the XL PR

The Juju team has been hard at work refactoring the Kubernetes charms. A new charming pattern has emerged where charms are now assembled from layers to build a deployable artifact. Allowing our footprint in the Kubernetes upstream repository to remain light and only pertain to the concerns of Kubernetes itself.

A few things to note about this PR:

  • It requires Juju 2.0 for full validation
    • Juju 1.25 support is incoming - on this PR if its deemed necessary prior to landing
  • We make use of /tmp/${random} to warehouse the ssl key(s) and kubeconfig during cluster/kube-up.sh - this is not intended to stand up a long running kubernetes cluster, but instead to validate the charm deployment for devel purposes. (building and packing a custom hyperkube image is planned but unscheduled at the moment)
  • This also removes the compile from source option, and instead moves to delivery via hyperkube

Prereq work to address #9107

@googlebot googlebot added the cla: yes label Mar 9, 2016

@k8s-bot

This comment has been minimized.

Show comment
Hide comment
@k8s-bot

k8s-bot Mar 9, 2016

Can one of the admins verify that this patch is reasonable to test in $JOB_NAME? (reply "ok to test", or if you trust the user, reply "add to whitelist")

If this message is too spammy, please complain to ixdy.

k8s-bot commented Mar 9, 2016

Can one of the admins verify that this patch is reasonable to test in $JOB_NAME? (reply "ok to test", or if you trust the user, reply "add to whitelist")

If this message is too spammy, please complain to ixdy.

@k8s-bot

This comment has been minimized.

Show comment
Hide comment
@k8s-bot

k8s-bot Mar 9, 2016

Can one of the admins verify that this patch is reasonable to test in $JOB_NAME? (reply "ok to test", or if you trust the user, reply "add to whitelist")

If this message is too spammy, please complain to ixdy.

k8s-bot commented Mar 9, 2016

Can one of the admins verify that this patch is reasonable to test in $JOB_NAME? (reply "ok to test", or if you trust the user, reply "add to whitelist")

If this message is too spammy, please complain to ixdy.

@k8s-bot

This comment has been minimized.

Show comment
Hide comment
@k8s-bot

k8s-bot Mar 9, 2016

Can one of the admins verify that this patch is reasonable to test in $JOB_NAME? (reply "ok to test", or if you trust the user, reply "add to whitelist")

If this message is too spammy, please complain to ixdy.

k8s-bot commented Mar 9, 2016

Can one of the admins verify that this patch is reasonable to test in $JOB_NAME? (reply "ok to test", or if you trust the user, reply "add to whitelist")

If this message is too spammy, please complain to ixdy.

@k8s-merge-robot

This comment has been minimized.

Show comment
Hide comment
@k8s-merge-robot

k8s-merge-robot Mar 9, 2016

Contributor

Labelling this PR as size/XXL

Contributor

k8s-merge-robot commented Mar 9, 2016

Labelling this PR as size/XXL

Show outdated Hide outdated cluster/juju/layers/kubernetes/templates/skydns-rc.yml
args:
# command = "/kube2sky"
{% if dns_domain -%}- -domain={{ dns_domain }} {% else %} - -domain=cluster.local {% endif %}
- -kube_master_url=http://{{private_address}}:8080

This comment has been minimized.

@rapenchukd

rapenchukd Mar 9, 2016

This could be your issue. I believe you need it to be {{ private_address }} and not {{private_address}}

@rapenchukd

rapenchukd Mar 9, 2016

This could be your issue. I believe you need it to be {{ private_address }} and not {{private_address}}

@lazypower

This comment has been minimized.

Show comment
Hide comment
@lazypower

lazypower Mar 9, 2016

Member

Ok, I think i have this building cleanly now, and have updated the necessary Meta. Let me know if you'd like me to squash this before the review happens.

Member

lazypower commented Mar 9, 2016

Ok, I think i have this building cleanly now, and have updated the necessary Meta. Let me know if you'd like me to squash this before the review happens.

@lazypower

This comment has been minimized.

Show comment
Hide comment
@lazypower

lazypower Mar 14, 2016

Member

@zmerlynn - any update on getting a review? the travis and cla bot seem fine. submit-queue still however states its pending failing tests? is that the mesos smoke test bot that i need to satisfy before I get a review?

Member

lazypower commented Mar 14, 2016

@zmerlynn - any update on getting a review? the travis and cla bot seem fine. submit-queue still however states its pending failing tests? is that the mesos smoke test bot that i need to satisfy before I get a review?

@k8s-bot

This comment has been minimized.

Show comment
Hide comment
@k8s-bot

k8s-bot Mar 15, 2016

Can one of the admins verify that this patch is reasonable to test? (reply "ok to test", or if you trust the user, reply "add to whitelist")

This message may repeat a few times in short succession due to jenkinsci/ghprb-plugin#292. Sorry.

Otherwise, if this message is too spammy, please complain to ixdy.

k8s-bot commented Mar 15, 2016

Can one of the admins verify that this patch is reasonable to test? (reply "ok to test", or if you trust the user, reply "add to whitelist")

This message may repeat a few times in short succession due to jenkinsci/ghprb-plugin#292. Sorry.

Otherwise, if this message is too spammy, please complain to ixdy.

lazypower added some commits Mar 7, 2016

Rework `cluster/juju` to reflect current work
This commit imports the latest development focus from the Charmer team
working to deliver Kubernetes charms with Juju.

Notable Changes:

- The charm is now assembled from layers in $JUJU_ROOT/layers
- Prior, the juju provider would compile and fat-pack the charms, this
  new approach delivers the entirety of Kubernetes via hyperkube.
- Adds Kubedns as part of `cluster/kube-up.sh` and verification
- Removes the hard-coded port 8080 for the Kubernetes Master
- Includes TLS validation
- Validates kubernetes config from leader charm
- Targets Juju 2.0 commands
@k8s-teamcity-mesosphere

This comment has been minimized.

Show comment
Hide comment
@k8s-teamcity-mesosphere

k8s-teamcity-mesosphere Mar 16, 2016

TeamCity OSS :: Kubernetes Mesos :: 4 - Smoke Tests Build 19267 outcome was SUCCESS
Summary: Tests passed: 1, ignored: 265 Build time: 00:06:28

TeamCity OSS :: Kubernetes Mesos :: 4 - Smoke Tests Build 19267 outcome was SUCCESS
Summary: Tests passed: 1, ignored: 265 Build time: 00:06:28

@zmerlynn

This comment has been minimized.

Show comment
Hide comment
@zmerlynn

zmerlynn Mar 21, 2016

Member

@k8s-bot ok to test

Member

zmerlynn commented Mar 21, 2016

@k8s-bot ok to test

@zmerlynn

This comment has been minimized.

Show comment
Hide comment
@zmerlynn

zmerlynn Mar 21, 2016

Member

@chuckbutler: I'm sorry for the long delay, this went to the bottom due to size. Is there any other maintainer that can help review this as well? This is mammoth and I have little Juju context.

Member

zmerlynn commented Mar 21, 2016

@chuckbutler: I'm sorry for the long delay, this went to the bottom due to size. Is there any other maintainer that can help review this as well? This is mammoth and I have little Juju context.

@k8s-bot

This comment has been minimized.

Show comment
Hide comment
@k8s-bot

k8s-bot commented Mar 21, 2016

GCE e2e build/test passed for commit b8b54ac.

@k8s-merge-robot

This comment has been minimized.

Show comment
Hide comment
@k8s-merge-robot

k8s-merge-robot Mar 21, 2016

Contributor

The author of this PR is not in the whitelist for merge, can one of the admins add the 'ok-to-merge' label?

Contributor

k8s-merge-robot commented Mar 21, 2016

The author of this PR is not in the whitelist for merge, can one of the admins add the 'ok-to-merge' label?

@lazypower

This comment has been minimized.

Show comment
Hide comment
@lazypower

lazypower Mar 21, 2016

Member

@zmerlynn - you bet, theres a couple

If you're OK with canonical being the reviewers, @marcoceppi or @mbruzek is on our ticket.

In terms of googlers that have looked at our code in the past:

@eparis and @erictune both lent a hand in bringin us on board and looking over our initial PR's that landed in cluster. I apologize again for the size, it wasn't intended to be this large, but it is a big deletion of footprint and re-scoping of what needs to be here, vs supporting infra code that revs independently of our work here.

If there's anything else I can do to help expedite the request, such as test run logs or an environment stood up with the charms to poke at and validate I'm happy to lend a hand.

Member

lazypower commented Mar 21, 2016

@zmerlynn - you bet, theres a couple

If you're OK with canonical being the reviewers, @marcoceppi or @mbruzek is on our ticket.

In terms of googlers that have looked at our code in the past:

@eparis and @erictune both lent a hand in bringin us on board and looking over our initial PR's that landed in cluster. I apologize again for the size, it wasn't intended to be this large, but it is a big deletion of footprint and re-scoping of what needs to be here, vs supporting infra code that revs independently of our work here.

If there's anything else I can do to help expedite the request, such as test run logs or an environment stood up with the charms to poke at and validate I'm happy to lend a hand.

Show outdated Hide outdated cluster/juju/bundles/local.yaml
expose: true
num_units: 2
etcd:
charm: cs:~lazypower/trusty/etcd

This comment has been minimized.

@marcoceppi

marcoceppi Mar 21, 2016

Member

NIT: Would it be better to put etcd in ~containers?

@marcoceppi

marcoceppi Mar 21, 2016

Member

NIT: Would it be better to put etcd in ~containers?

This comment has been minimized.

@lazypower

lazypower Mar 21, 2016

Member

Ah good point!

@lazypower

lazypower Mar 21, 2016

Member

Ah good point!

This comment has been minimized.

@lazypower

lazypower Mar 21, 2016

Member

I'll bring this in on the next update w/ the docs. @zmerlynn - is that still going to be pointed here or have the docs migrated to their new home already?

@lazypower

lazypower Mar 21, 2016

Member

I'll bring this in on the next update w/ the docs. @zmerlynn - is that still going to be pointed here or have the docs migrated to their new home already?

@marcoceppi

This comment has been minimized.

Show comment
Hide comment
@marcoceppi

marcoceppi Mar 21, 2016

Member

LGTM

Member

marcoceppi commented Mar 21, 2016

LGTM

@mbruzek

This comment has been minimized.

Show comment
Hide comment
@mbruzek

mbruzek Mar 21, 2016

Member

I independently verified running ./kube-up.sh from the cluster directory. Everything came up OK. I will take a further look at the code.

... calling validate-cluster
Found 2 node(s).
NAME            LABELS                                 STATUS    AGE
172.31.10.125   kubernetes.io/hostname=172.31.10.125   Ready     8m
172.31.33.22    kubernetes.io/hostname=172.31.33.22    Ready     7m
Validate output:
NAME                 STATUS    MESSAGE              ERROR
etcd-0               Healthy   {"health": "true"}   nil
controller-manager   Healthy   ok                   nil
scheduler            Healthy   ok                   nil
Cluster validation succeeded
Done, listing cluster services:

Kubernetes master is running at https://52.91.207.25:6443
KubeDNS is running at https://52.91.207.25:6443/api/v1/proxy/namespaces/kube-system/services/kube-dns
Member

mbruzek commented Mar 21, 2016

I independently verified running ./kube-up.sh from the cluster directory. Everything came up OK. I will take a further look at the code.

... calling validate-cluster
Found 2 node(s).
NAME            LABELS                                 STATUS    AGE
172.31.10.125   kubernetes.io/hostname=172.31.10.125   Ready     8m
172.31.33.22    kubernetes.io/hostname=172.31.33.22    Ready     7m
Validate output:
NAME                 STATUS    MESSAGE              ERROR
etcd-0               Healthy   {"health": "true"}   nil
controller-manager   Healthy   ok                   nil
scheduler            Healthy   ok                   nil
Cluster validation succeeded
Done, listing cluster services:

Kubernetes master is running at https://52.91.207.25:6443
KubeDNS is running at https://52.91.207.25:6443/api/v1/proxy/namespaces/kube-system/services/kube-dns
@mbruzek

This comment has been minimized.

Show comment
Hide comment
@mbruzek

mbruzek Mar 21, 2016

Member

I know this looks like an XXL pull request, but looking at the lines changed: +915 −2,513 that is a net loss of 1598 lines of code in the cluster/juju/ directory that makes the Juju specific files much smaller.

The heart of this change is in the cluster/juju/util.sh and that also was reduced in size. I deployed this via kube-up.sh on amazon with KUBERNETES_REPOSITORY=juju and it worked for me.

+1 on the code and the change from my end (for what it is worth).

Member

mbruzek commented Mar 21, 2016

I know this looks like an XXL pull request, but looking at the lines changed: +915 −2,513 that is a net loss of 1598 lines of code in the cluster/juju/ directory that makes the Juju specific files much smaller.

The heart of this change is in the cluster/juju/util.sh and that also was reduced in size. I deployed this via kube-up.sh on amazon with KUBERNETES_REPOSITORY=juju and it worked for me.

+1 on the code and the change from my end (for what it is worth).

@erictune

This comment has been minimized.

Show comment
Hide comment
@erictune

erictune Mar 22, 2016

Member

@zmerlynn I'll pick this up.

Member

erictune commented Mar 22, 2016

@zmerlynn I'll pick this up.

@erictune erictune assigned erictune and unassigned zmerlynn Mar 22, 2016

Show outdated Hide outdated cluster/juju/layers/kubernetes/README.md
- [Kubernetes github project](https://github.com/kubernetes/kubernetes)
- [Kubernetes issue tracker](https://github.com/kubernetes/kubernetes/issues)
- [Kubernetes Documenation](https://github.com/kubernetes/kubernetes/tree/master/docs)

This comment has been minimized.

@erictune

erictune Mar 22, 2016

Member

We prefer that you link to http://kubernetes.io/docs/

@erictune

erictune Mar 22, 2016

Member

We prefer that you link to http://kubernetes.io/docs/

Show outdated Hide outdated cluster/juju/layers/kubernetes/actions/guestbook-example
#
# Lifted from github.com/kubernetes/kubernetes/examples/guestbook-example
kubectl create -f files/guestbook-example/redis-master-service.yaml

This comment has been minimized.

@erictune

erictune Mar 22, 2016

Member

Could this pull the files from git instead, so the files are not duplicated under the cluster/juju/... directory? e.g. curl https://rawgit.com/kubernetes/kubernetes/v1.2.0/examples/guestbook/frontend-controller.yaml

@erictune

erictune Mar 22, 2016

Member

Could this pull the files from git instead, so the files are not duplicated under the cluster/juju/... directory? e.g. curl https://rawgit.com/kubernetes/kubernetes/v1.2.0/examples/guestbook/frontend-controller.yaml

This comment has been minimized.

@marcoceppi

marcoceppi Mar 23, 2016

Member

We typically try to avoid too many remote lookups in charms as a lot of deployments are done so behind aggressive egress filtering, but given this is more or less a "hello world" action I don't see why not 👍

@marcoceppi

marcoceppi Mar 23, 2016

Member

We typically try to avoid too many remote lookups in charms as a lot of deployments are done so behind aggressive egress filtering, but given this is more or less a "hello world" action I don't see why not 👍

This comment has been minimized.

@lazypower

lazypower Mar 23, 2016

Member

Matt and I were talking about this. We're going to add a tactic to copy in paths from the repository which will be defined in the layer.yaml, so its building from the current revision of the tree. It makes more sense to put that in the charm artifact vs the layer.

Super perk to the location of the layer :)

@lazypower

lazypower Mar 23, 2016

Member

Matt and I were talking about this. We're going to add a tactic to copy in paths from the repository which will be defined in the layer.yaml, so its building from the current revision of the tree. It makes more sense to put that in the charm artifact vs the layer.

Super perk to the location of the layer :)

@erictune

This comment has been minimized.

Show comment
Hide comment
@erictune

erictune Mar 22, 2016

Member

two comments.

Member

erictune commented Mar 22, 2016

two comments.

@k8s-bot

This comment has been minimized.

Show comment
Hide comment
@k8s-bot

k8s-bot commented Mar 23, 2016

GCE e2e build/test passed for commit 326a51f.

@k8s-bot

This comment has been minimized.

Show comment
Hide comment
@k8s-bot

k8s-bot commented Mar 23, 2016

GCE e2e build/test passed for commit cc1fb15.

@k8s-teamcity-mesosphere

This comment has been minimized.

Show comment
Hide comment
@k8s-teamcity-mesosphere

k8s-teamcity-mesosphere Mar 23, 2016

TeamCity OSS :: Kubernetes Mesos :: 4 - Smoke Tests Build 19796 outcome was SUCCESS
Summary: Tests passed: 1, ignored: 265 Build time: 00:09:54

TeamCity OSS :: Kubernetes Mesos :: 4 - Smoke Tests Build 19796 outcome was SUCCESS
Summary: Tests passed: 1, ignored: 265 Build time: 00:09:54

@erictune

This comment has been minimized.

Show comment
Hide comment
@erictune

erictune Mar 23, 2016

Member

LGTM. Thanks!

Member

erictune commented Mar 23, 2016

LGTM. Thanks!

@erictune erictune added the lgtm label Mar 23, 2016

@k8s-merge-robot

This comment has been minimized.

Show comment
Hide comment
@k8s-merge-robot

k8s-merge-robot Mar 23, 2016

Contributor

@k8s-bot test this [submit-queue is verifying that this PR is safe to merge]

Contributor

k8s-merge-robot commented Mar 23, 2016

@k8s-bot test this [submit-queue is verifying that this PR is safe to merge]

@k8s-bot

This comment has been minimized.

Show comment
Hide comment
@k8s-bot

k8s-bot commented Mar 23, 2016

GCE e2e build/test passed for commit cc1fb15.

@k8s-merge-robot

This comment has been minimized.

Show comment
Hide comment
@k8s-merge-robot

k8s-merge-robot Mar 23, 2016

Contributor

@k8s-bot test this [submit-queue is verifying that this PR is safe to merge]

Contributor

k8s-merge-robot commented Mar 23, 2016

@k8s-bot test this [submit-queue is verifying that this PR is safe to merge]

@k8s-bot

This comment has been minimized.

Show comment
Hide comment
@k8s-bot

k8s-bot commented Mar 23, 2016

GCE e2e build/test passed for commit cc1fb15.

@k8s-merge-robot

This comment has been minimized.

Show comment
Hide comment
@k8s-merge-robot

k8s-merge-robot Mar 23, 2016

Contributor

Automatic merge from submit-queue

Contributor

k8s-merge-robot commented Mar 23, 2016

Automatic merge from submit-queue

@k8s-merge-robot k8s-merge-robot merged commit 69b3cb3 into kubernetes:master Mar 23, 2016

4 of 5 checks passed

Submit Queue Github CI tests are not green.
Details
Jenkins GCE e2e 268 tests run, 107 skipped, 0 failed.
Details
Jenkins unit/integration 5820 tests run, 18 skipped, 0 failed.
Details
cla/google All necessary CLAs are signed
continuous-integration/travis-ci/pr The Travis CI build passed
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment