New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

scheduler: enable secure port and authn/z #69663

Merged
merged 5 commits into from Nov 9, 2018

Conversation

@sttts
Copy link
Contributor

sttts commented Oct 11, 2018

Completes #67192.

This PR adds a secure port 10259 to the scheduler

  • enabled by default as for the controller manager
  • deprecates the old insecure port
  • adds exhaustive delegated authn/z tests.
Add secure port 10259 to the kube-scheduler (enabled by default) and deprecate old insecure port 10251. Without further flags self-signed certs are created on startup in memory.

Fixes #58983.

@sttts

This comment has been minimized.

Copy link
Contributor

sttts commented Oct 11, 2018

/retest

@sttts sttts force-pushed the sttts:sttts-scheduler-secure-serving branch from de64334 to 1588af4 Nov 8, 2018

@k8s-ci-robot k8s-ci-robot removed the lgtm label Nov 8, 2018

@k8s-ci-robot

This comment has been minimized.

Copy link
Contributor

k8s-ci-robot commented Nov 8, 2018

New changes are detected. LGTM label has been removed.

@sttts sttts added the lgtm label Nov 8, 2018

@aveshagarwal

This comment has been minimized.

Copy link
Member

aveshagarwal commented Nov 8, 2018

/approve

@k8s-ci-robot

This comment has been minimized.

Copy link
Contributor

k8s-ci-robot commented Nov 8, 2018

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: aveshagarwal, liggitt, sttts

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@AishSundar

This comment has been minimized.

Copy link
Contributor

AishSundar commented Nov 8, 2018

/cc @mengqiy

Adding @mengqiy who is the GKE oncall. @mengqiy this PR is failing on pull-kubernetes-e2e-gke presubmits. can you plz help @sttts get the right GLKE CI logs to investigate why. Thanks

@k8s-ci-robot k8s-ci-robot requested a review from mengqiy Nov 8, 2018

@sttts

This comment has been minimized.

Copy link
Contributor

sttts commented Nov 8, 2018

/retest

@sttts

This comment has been minimized.

Copy link
Contributor

sttts commented Nov 8, 2018

Adding @mengqiy who is the GKE oncall. @mengqiy this PR is failing on pull-kubernetes-e2e-gke presubmits. can you plz help @sttts get the right GLKE CI logs to investigate why. Thanks

Note that we moved the commit which breaks GKE into #70800

@k8s-ci-robot

This comment has been minimized.

Copy link
Contributor

k8s-ci-robot commented Nov 8, 2018

@sttts: The following test failed, say /retest to rerun them all:

Test name Commit Details Rerun command
pull-kubernetes-e2e-gke e56fd05 link /test pull-kubernetes-e2e-gke

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@fejta-bot

This comment has been minimized.

Copy link

fejta-bot commented Nov 8, 2018

/retest
This bot automatically retries jobs that failed/flaked on approved PRs (send feedback to fejta).

Review the full test history for this PR.

Silence the bot with an /lgtm cancel comment for consistent failures.

@k8s-ci-robot k8s-ci-robot merged commit be800e6 into kubernetes:master Nov 9, 2018

18 checks passed

cla/linuxfoundation sttts authorized
Details
pull-kubernetes-bazel-build Job succeeded.
Details
pull-kubernetes-bazel-test Job succeeded.
Details
pull-kubernetes-cross Skipped
pull-kubernetes-e2e-gce Job succeeded.
Details
pull-kubernetes-e2e-gce-100-performance Job succeeded.
Details
pull-kubernetes-e2e-gce-device-plugin-gpu Job succeeded.
Details
pull-kubernetes-e2e-gke Skipped
pull-kubernetes-e2e-kops-aws Job succeeded.
Details
pull-kubernetes-e2e-kubeadm-gce Skipped
pull-kubernetes-integration Job succeeded.
Details
pull-kubernetes-kubemark-e2e-gce-big Job succeeded.
Details
pull-kubernetes-local-e2e Skipped
pull-kubernetes-local-e2e-containerized Skipped
pull-kubernetes-node-e2e Job succeeded.
Details
pull-kubernetes-typecheck Job succeeded.
Details
pull-kubernetes-verify Job succeeded.
Details
tide In merge pool.
Details
@furkanmustafa

This comment has been minimized.

Copy link

furkanmustafa commented Dec 4, 2018

deprecates the old insecure port

@sttts I want to learn about the motivations behind deprecating the plain http (insecure!?) port. Can you point me to any related discussions about this?

Deprecated: &DeprecatedOptions{
UseLegacyPolicyConfig: false,
PolicyConfigMapNamespace: metav1.NamespaceSystem,
},
}

o.Authentication.RemoteKubeConfigFileOptional = true
o.Authorization.RemoteKubeConfigFileOptional = true

This comment has been minimized.

@liggitt

liggitt Dec 5, 2018

Member

in addition to tolerating missing kubeconfig, this needed to tolerate forbidden failures in lookupMissingConfigInCluster... nothing granted the scheduler delegated authentication permissions by default

xref #71743

joshrosso added a commit to joshrosso/kubernetes that referenced this pull request Dec 18, 2018

kubeadm: use secure port for liveness probe on controller-manager and…
… scheduler

This commit uses the secure port (tls) for liveness (/healthz) checks
against the kube-controller-manager and kube-scheduler. This makes the
livenessProbe protocol consistent between the kube-apiserver,
kube-controller-manager, and kube-scheduler.

Secure ports were introduced in 1.12 for the kube-controller-manager[0]
and 1.13 for the kube-scheduler[1]. The insecure ports have since been
deprecated.

[0]: kubernetes#67069
[1]: kubernetes#69663

joshrosso added a commit to joshrosso/kubernetes that referenced this pull request Dec 18, 2018

cmd/kubeadm: use secure port for liveness probe on controller-manager…
… and scheduler

This commit uses the secure port (tls) for liveness (/healthz) checks
against the kube-controller-manager and kube-scheduler. This makes the
livenessProbe protocol consistent between the kube-apiserver,
kube-controller-manager, and kube-scheduler.

Secure ports were introduced in 1.12 for the kube-controller-manager[0]
and 1.13 for the kube-scheduler[1]. The insecure ports have since been
deprecated.

[0]: kubernetes#67069
[1]: kubernetes#69663

joshrosso added a commit to joshrosso/kubernetes that referenced this pull request Dec 18, 2018

cmd/kubeadm: use secure port for liveness probe on controller-manager…
… and scheduler

This commit uses the secure port (tls) for liveness (/healthz) checks
against the kube-controller-manager and kube-scheduler. This makes the
livenessProbe protocol consistent between the kube-apiserver,
kube-controller-manager, and kube-scheduler.

Secure ports were introduced in 1.12 for the kube-controller-manager[0]
and 1.13 for the kube-scheduler[1]. The insecure ports have since been
deprecated.

[0]: kubernetes#67069
[1]: kubernetes#69663

joshrosso added a commit to joshrosso/kubernetes that referenced this pull request Dec 18, 2018

cmd/kubeadm: use secure port for liveness probe on controller-manager…
… and scheduler

This commit uses the secure port (tls) for liveness (/healthz) checks
against the kube-controller-manager and kube-scheduler. This makes the
livenessProbe protocol consistent between the kube-apiserver,
kube-controller-manager, and kube-scheduler.

Secure ports were introduced in 1.12 for the kube-controller-manager[0]
and 1.13 for the kube-scheduler[1]. The insecure ports have since been
deprecated.

[0]: kubernetes#67069
[1]: kubernetes#69663
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment