New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handle error responses from backends #71412

Merged
merged 1 commit into from Nov 26, 2018

Conversation

@liggitt
Copy link
Member

liggitt commented Nov 26, 2018

What type of PR is this?
/kind bug

What this PR does / why we need it:
Handles error responses from backends

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #71411

Does this PR introduce a user-facing change?:

Fixes an issue with stuck connections handling error responses

/sig api-machinery
/cc sttts

@k8s-ci-robot

This comment has been minimized.

Copy link
Contributor

k8s-ci-robot commented Nov 26, 2018

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: liggitt

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@liggitt

This comment has been minimized.

Copy link
Member Author

liggitt commented Nov 26, 2018

/retest

@sttts

This comment has been minimized.

Copy link
Contributor

sttts commented Nov 26, 2018

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm label Nov 26, 2018

@k8s-ci-robot k8s-ci-robot merged commit 2257c1e into kubernetes:master Nov 26, 2018

18 checks passed

cla/linuxfoundation liggitt authorized
Details
pull-kubernetes-bazel-build Job succeeded.
Details
pull-kubernetes-bazel-test Job succeeded.
Details
pull-kubernetes-cross Skipped
pull-kubernetes-e2e-gce Job succeeded.
Details
pull-kubernetes-e2e-gce-100-performance Job succeeded.
Details
pull-kubernetes-e2e-gce-device-plugin-gpu Job succeeded.
Details
pull-kubernetes-e2e-gke Skipped
pull-kubernetes-e2e-kops-aws Job succeeded.
Details
pull-kubernetes-e2e-kubeadm-gce Skipped
pull-kubernetes-integration Job succeeded.
Details
pull-kubernetes-kubemark-e2e-gce-big Job succeeded.
Details
pull-kubernetes-local-e2e Skipped
pull-kubernetes-local-e2e-containerized Skipped
pull-kubernetes-node-e2e Job succeeded.
Details
pull-kubernetes-typecheck Job succeeded.
Details
pull-kubernetes-verify Job succeeded.
Details
tide In merge pool.
Details

k8s-ci-robot added a commit that referenced this pull request Nov 26, 2018

Merge pull request #71414 from liggitt/backend-error-1.11
Cherry pick of #71412: Handle error responses from backends

k8s-ci-robot added a commit that referenced this pull request Nov 26, 2018

Merge pull request #71415 from liggitt/backend-error-1.10
Cherry pick of #71412: Handle error responses from backends

k8s-ci-robot added a commit that referenced this pull request Nov 26, 2018

Merge pull request #71413 from liggitt/backend-error-1.12
Cherry pick of #71412: Handle error responses from backends

@liggitt liggitt deleted the liggitt:backend-error branch Nov 26, 2018

@jennybuckley

This comment has been minimized.

Copy link
Contributor

jennybuckley commented Nov 26, 2018

@k8s-ci-robot k8s-ci-robot requested a review from wenjiaswe Nov 26, 2018

@tossmilestone

This comment has been minimized.

Copy link
Contributor

tossmilestone commented Dec 4, 2018

@liggitt Can I cherry-pick your change to our 1.9.x k8s to fix the issue?

@liggitt

This comment has been minimized.

Copy link
Member Author

liggitt commented Dec 4, 2018

The 1.10 commit (#71415) would be a better one to pick to 1.9, just to avoid file drift issues (the change itself is fundamentally the same)

@cizixs

This comment has been minimized.

Copy link

cizixs commented Dec 4, 2018

@liggitt Why does this MR fixes #71411.

Seems to me this only handles connection upgrade error. And only the connection is established, client can still send any request skipping authorization.

@pdhung

This comment has been minimized.

Copy link

pdhung commented Dec 5, 2018

cizisx:
@liggitt Why does this MR fixes #71411.

Seems to me this only handles connection upgrade error. And only the connection is established, client can still send any request skipping authorization.

As far as I understand, in the buggy version, the connection upgrade error was not handled correctly when it happens, and the backend connection is left open, which leaves subsequent request to pass through to kubelet unchecked/unauthorized. Since API server authenticates to kubelet as cluster-admin, it allows attacker to run any API on kubelet.
This PR rectifies the problem by closing the backend connection.

waynr added a commit to rcbops/kubernetes that referenced this pull request Dec 5, 2018

Merge pull request kubernetes#71415 from liggitt/backend-error-1.10
Cherry pick of kubernetes#71412: Handle error responses from backends

waynr added a commit to rcbops/kubernetes that referenced this pull request Dec 5, 2018

Merge pull request kubernetes#71415 from liggitt/backend-error-1.10
Cherry pick of kubernetes#71412: Handle error responses from backends
@mgalgs

This comment has been minimized.

Copy link

mgalgs commented on b84e3dd Dec 6, 2018

Backend upgraded connection: Not verified ✔️

@moonek

This comment has been minimized.

Copy link
Contributor

moonek commented Dec 10, 2018

We noticed that this change only applies to the following releases.
v1.10.11
v1.11.5
v1.12.3
v1.13.0-rc.1

Our old environment is using k8s 1.8.4.
We are considering an upgrade k8s version, but it takes time.
So I want to hotfix this commit until then.

Does the k8s 1.8.4 source rebuild solve #71411 vulnerability by only reflecting this commit?

When checked with the test utility below, it appears to be resolved.
https://github.com/gravitational/cve-2018-1002105

@liggitt

This comment has been minimized.

Copy link
Member Author

liggitt commented Dec 11, 2018

@moonek yes, this commit addresses #71411

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment