Added function to create kubeconfig for addon-manager #75675

mwwolters · Mar 25, 2019

/kind cleanup

This is the first step toward migrating the addon manager off of the master insecure port.

k8s-ci-robot · Mar 25, 2019

@mwwolters: Adding the "do-not-merge/release-note-label-needed" label because no release-note block was detected, please follow our release note process to remove it.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

k8s-ci-robot · Mar 25, 2019

Hi @mwwolters. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

mwwolters · Mar 25, 2019

/assign dekkagaijin
/sig gcp

k8s-ci-robot · Mar 25, 2019

@mwwolters: GitHub didn't allow me to assign the following users: dekkagaijin.

Note that only kubernetes members and repo collaborators can be assigned and that issues/PRs can only have 10 assignees at the same time.
For more information please see the contributor guide

In response to this:

/assign dekkagaijin
/sig gcp

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

mwwolters · Mar 25, 2019

cc @dekkagaijin

dekkagaijin · Mar 26, 2019

dekkagaijin reviewed Mar 26, 2019

View changes

cluster/gce/manifests/kube-addon-manager.yaml

dekkagaijin · Mar 27, 2019

dekkagaijin reviewed Mar 27, 2019

View changes

cluster/gce/gci/configure-helper.sh

mwwolters · Apr 10, 2019

/assign @jszczepkowski

mwwolters · Apr 15, 2019

/assign MaciekPytel

MrHohn · Apr 18, 2019

/ok-to-test
/assign

MrHohn · Apr 18, 2019

MrHohn reviewed Apr 18, 2019

View changes

LGTM overall

MrHohn · Apr 18, 2019

cluster/gce/gci/configure-helper.sh

@@ -1096,6 +1097,30 @@ current-context: cluster-autoscaler
 EOF
 }

+function create-addonmanager-kubeconfig {


It feels like these create-*-kubeconfig funcs are mostly the same. Though not sure if it's worth it for refactoring.

Added the function and changed in most places a kubeconfig is made.

MrHohn · Apr 18, 2019

cluster/gce/manifests/kube-addon-manager.yaml

    env:
    - name: KUBECTL_EXTRA_PRUNE_WHITELIST
      value: {{kubectl_extra_prune_whitelist}}
+    - name: KUBECTL_OPTS
+      value: '--kubeconfig=/etc/srv/kubernetes/addon-manager/kubeconfig'


Does these apply to https://github.com/kubernetes/kubernetes/blob/master/test/kubemark/resources/manifests/kube-addon-manager.yaml as well?

I think they do, added and am testing.

MrHohn · Apr 18, 2019

cluster/gce/gci/configure-helper.sh

@@ -594,6 +594,7 @@ function create-master-auth {
  if [[ -n "${NODE_PROBLEM_DETECTOR_TOKEN:-}" ]]; then
    append_or_replace_prefixed_line "${known_tokens_csv}" "${NODE_PROBLEM_DETECTOR_TOKEN},"   "system:node-problem-detector,uid:node-problem-detector"
  fi
+  append_or_replace_prefixed_line "${known_tokens_csv}" "${ADDON_MANAGER_TOKEN},"   "system:addon-manager,uid:addon-manager"


Probably check if ${ADDON_MANAGER_TOKEN} is set like the others?

Does this need to match the system:addon-manager-role role we create in create-apply-addon-manager-rbac()?

Added the check.

In this case it needs to be the user which is bound to that role.

MrHohn · Apr 18, 2019

cluster/gce/gci/configure-helper.sh

@@ -2502,6 +2527,13 @@ function start-kube-addons {
  local -r src_dir="${KUBE_HOME}/kube-manifests/kubernetes/gci-trusty"
  local -r dst_dir="/etc/kubernetes/addons"

+  # need kube-apiserver to be ready
+  until kubectl get nodes; do


We already have two places (e.g. update-legacy-addon-node-labels()) that do this check. Maybe make a wait-til-apiserver-ready() func and put that under main?

Thanks! Can we run wait-till-apiserver-ready just once so individual step won't need to check explicitly?

start-kube-scheduler wait-till-apiserver-ready start-kube-addons

dekkagaijin · May 3, 2019

/priority important-soon

MrHohn · May 3, 2019

Thanks for making this working :) (I can't approve though.)
/lgtm
/approve

dekkagaijin · May 3, 2019

/test pull-kubernetes-e2e-gce
/test pull-kubernetes-integration

dekkagaijin · May 3, 2019

Need approval from cluster/gce/OWNERS
/assign cjcullen

dekkagaijin · May 3, 2019

/test pull-kubernetes-kubemark-e2e-gce-big

dekkagaijin · May 3, 2019

dekkagaijin reviewed May 3, 2019

View changes

dekkagaijin · May 3, 2019

cluster/gce/gci/configure-helper.sh

@@ -2765,6 +2700,7 @@ function start-lb-controller {
    prepare-log-file /var/log/glbc.log
    setup-addon-manifests "addons" "cluster-loadbalancing/glbc"
    setup-addon-manifests "addons" "rbac/cluster-loadbalancing/glbc"
+    create-kubeconfig "l7-lb-controller" ${GCE_GLBC_TOKEN}
    create-l7-lb-controller-kubeconfig


dekkagaijin · May 3, 2019

dekkagaijin approved these changes May 3, 2019

View changes

dekkagaijin · May 3, 2019

/lgtm

dekkagaijin · May 3, 2019

/test pull-kubernetes-kubemark-e2e-gce-big

dekkagaijin · May 3, 2019

🎉 🎉 🎉 🎉

cjcullen · May 3, 2019

cjcullen reviewed May 3, 2019

View changes

cjcullen · May 3, 2019

cluster/gce/gci/configure-helper.sh

@@ -1987,7 +1920,7 @@ function setup-etcd-encryption {

 # Updates node labels used by addons.
 function update-legacy-addon-node-labels() {
-  # need kube-apiserver to be ready
+  # need kube-api-server to be ready


Nit: it's usually just "kube-apiserver"

cjcullen · May 3, 2019

cluster/gce/gci/configure-helper.sh

@@ -597,6 +597,9 @@ function create-master-auth {
  if [[ -n "${GCE_GLBC_TOKEN:-}" ]]; then
    append_or_replace_prefixed_line "${known_tokens_csv}" "${GCE_GLBC_TOKEN},"                "system:controller:glbc,uid:system:controller:glbc"
  fi
+  if [[ -n "${ADDON_MANAGER_TOKEN:-}" ]]; then
+    append_or_replace_prefixed_line "${known_tokens_csv}" "${ADDON_MANAGER_TOKEN},"   "system:addon-manager,admin,system:masters"


The entry after the "Name" is the "UID." It's mostly not used, but setting it to "admin" is confusing. Make it uid:system:addon-manager.

dekkagaijin · May 3, 2019

/lgtm

cjcullen · May 4, 2019

/approve

k8s-ci-robot · May 4, 2019

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: cjcullen, MrHohn, mwwolters

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~cluster/gce/OWNERS~~ [cjcullen]
~~test/kubemark/OWNERS~~ [MrHohn]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

dekkagaijin · May 4, 2019

/test pull-kubernetes-e2e-gce

k8s-ci-robot added kind/cleanup size/S cncf-cla: yes do-not-merge/release-note-label-needed needs-sig needs-priority labels Mar 25, 2019

k8s-ci-robot added the needs-ok-to-test label Mar 25, 2019

k8s-ci-robot requested review from jingax10 and jszczepkowski Mar 25, 2019

k8s-ci-robot added sig/cluster-lifecycle and removed needs-sig labels Mar 25, 2019

k8s-ci-robot added the sig/gcp label Mar 25, 2019

k8s-ci-robot added size/M and removed size/S labels Mar 27, 2019

k8s-ci-robot assigned jszczepkowski Apr 10, 2019

k8s-ci-robot assigned MaciekPytel Apr 15, 2019

k8s-ci-robot assigned MrHohn Apr 18, 2019

k8s-ci-robot added ok-to-test and removed needs-ok-to-test labels Apr 18, 2019

k8s-ci-robot added release-note-none and removed do-not-merge/release-note-label-needed labels May 3, 2019

k8s-ci-robot added priority/important-soon and removed needs-priority labels May 3, 2019

k8s-ci-robot assigned cjcullen May 3, 2019

mwwolters force-pushed the mwwolters:addon-manager-kubeconfig branch from c75a9d7 to f584954 May 3, 2019

k8s-ci-robot removed the lgtm label May 3, 2019

k8s-ci-robot added the lgtm label May 3, 2019

Added function to create kubeconfig for addon-manager

Loading status checks…

1456979

mwwolters force-pushed the mwwolters:addon-manager-kubeconfig branch from f584954 to 1456979 May 3, 2019

k8s-ci-robot removed the lgtm label May 3, 2019

k8s-ci-robot added the lgtm label May 3, 2019

k8s-ci-robot added the approved label May 4, 2019

kubernetes/kubernetes

Join GitHub today

Added function to create kubeconfig for addon-manager #75675

Conversation

mwwolters commented Mar 25, 2019

This comment has been minimized.

k8s-ci-robot commented Mar 25, 2019

k8s-ci-robot added kind/cleanup size/S cncf-cla: yes do-not-merge/release-note-label-needed needs-sig needs-priority labels Mar 25, 2019

This comment has been minimized.

k8s-ci-robot commented Mar 25, 2019

k8s-ci-robot added the needs-ok-to-test label Mar 25, 2019

k8s-ci-robot requested review from jingax10 and jszczepkowski Mar 25, 2019

k8s-ci-robot added sig/cluster-lifecycle and removed needs-sig labels Mar 25, 2019

This comment has been minimized.

mwwolters commented Mar 25, 2019

k8s-ci-robot added the sig/gcp label Mar 25, 2019

This comment has been minimized.

k8s-ci-robot commented Mar 25, 2019

This comment has been minimized.

mwwolters commented Mar 25, 2019

dekkagaijin reviewed Mar 26, 2019 View changes

k8s-ci-robot added size/M and removed size/S labels Mar 27, 2019

dekkagaijin reviewed Mar 27, 2019 View changes

This comment has been minimized.

mwwolters commented Apr 10, 2019

k8s-ci-robot assigned jszczepkowski Apr 10, 2019

This comment has been minimized.

mwwolters commented Apr 15, 2019

k8s-ci-robot assigned MaciekPytel Apr 15, 2019

This comment has been minimized.

MrHohn commented Apr 18, 2019

k8s-ci-robot assigned MrHohn Apr 18, 2019

k8s-ci-robot added ok-to-test and removed needs-ok-to-test labels Apr 18, 2019

MrHohn reviewed Apr 18, 2019 View changes

MrHohn left a comment

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

k8s-ci-robot added release-note-none and removed do-not-merge/release-note-label-needed labels May 3, 2019

This comment has been minimized.

dekkagaijin commented May 3, 2019

k8s-ci-robot added priority/important-soon and removed needs-priority labels May 3, 2019

This comment has been minimized.

MrHohn commented May 3, 2019

This comment has been minimized.

dekkagaijin commented May 3, 2019

This comment has been minimized.

dekkagaijin commented May 3, 2019

k8s-ci-robot assigned cjcullen May 3, 2019

This comment has been minimized.

dekkagaijin commented May 3, 2019

dekkagaijin reviewed May 3, 2019 View changes

This comment has been minimized.

mwwolters force-pushed the mwwolters:addon-manager-kubeconfig branch from c75a9d7 to f584954 May 3, 2019

k8s-ci-robot removed the lgtm label May 3, 2019

dekkagaijin approved these changes May 3, 2019 View changes

This comment has been minimized.

dekkagaijin commented May 3, 2019

k8s-ci-robot added the lgtm label May 3, 2019

This comment has been minimized.

dekkagaijin commented May 3, 2019

This comment has been minimized.

dekkagaijin commented May 3, 2019

cjcullen reviewed May 3, 2019 View changes

This comment has been minimized.

This comment has been minimized.

mwwolters force-pushed the mwwolters:addon-manager-kubeconfig branch from f584954 to 1456979 May 3, 2019

k8s-ci-robot removed the lgtm label May 3, 2019

This comment has been minimized.

dekkagaijin commented May 3, 2019

k8s-ci-robot added the lgtm label May 3, 2019

This comment has been minimized.

cjcullen commented May 4, 2019

dekkagaijin reviewed Mar 26, 2019

View changes

dekkagaijin reviewed Mar 27, 2019

View changes

MrHohn reviewed Apr 18, 2019

View changes

dekkagaijin reviewed May 3, 2019

View changes

mwwolters force-pushed the mwwolters:addon-manager-kubeconfig branch from `c75a9d7` to `f584954` May 3, 2019

dekkagaijin approved these changes May 3, 2019

View changes

cjcullen reviewed May 3, 2019

View changes

mwwolters force-pushed the mwwolters:addon-manager-kubeconfig branch from `f584954` to `1456979` May 3, 2019

k8s-ci-robot merged commit `5f8d290` into kubernetes:master May 4, 2019
20 checks passed