Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Automated cherry pick of #121201: Prevent rapid reset http2 DOS on API server #121228: Register UnauthenticatedHTTP2DOSMitigation into kube #121934

Closed
wants to merge 4 commits into from
Closed

Automated cherry pick of #121201: Prevent rapid reset http2 DOS on API server #121228: Register UnauthenticatedHTTP2DOSMitigation into kube #121934

wants to merge 4 commits into from
+297 −7

Conversation

enj
Copy link
Member

@enj enj commented Nov 16, 2023

Cherry pick of #121201 #121228 on release-1.24.

#121201: Prevent rapid reset http2 DOS on API server
#121228: Register UnauthenticatedHTTP2DOSMitigation into kube

For details on the cherry pick process, see the cherry pick requests page.

enj and others added 4 commits November 16, 2023 16:08
@enj
Prevent rapid reset http2 DOS on API server
12bd309 
This change fully addresses CVE-2023-44487 and CVE-2023-39325 for
the API server when the client is unauthenticated.

The changes to util/runtime are required because otherwise a large
number of requests can get blocked on the time.Sleep calls.

For unauthenticated clients (either via 401 or the anonymous user),
we simply no longer allow such clients to hold open http2
connections.  They can use http2, but with the performance of http1
(with keep-alive disabled).

Since this change has the potential to cause issues, the
UnauthenticatedHTTP2DOSMitigation feature gate can be disabled to
remove this protection (it is enabled by default).  For example,
when the API server is fronted by an L7 load balancer that is set up
to mitigate http2 attacks, unauthenticated clients could force
disable connection reuse between the load balancer and the API
server (many incoming connections could share the same backend
connection).  An API server that is on a private network may opt to
disable this protection to prevent performance regressions for
unauthenticated clients.

For all other clients, we rely on the golang.org/x/net fix in
golang/net@b225e7c
That change is not sufficient to adequately protect against a
motivated client - future changes to Kube and/or golang.org/x/net
will be explored to address this gap.

The Kube API server now uses a max stream of 100 instead of 250
(this matches the Go http2 client default).  This lowers the abuse
limit from 1000 to 400.

Signed-off-by: Monis Khan <mok@microsoft.com>
@enj
Disable UnauthenticatedHTTP2DOSMitigation by default
8b667cc 
This makes backports safer by not changing any default behavior.

Signed-off-by: Monis Khan <mok@microsoft.com>
@enj
Skip TestUnauthenticatedHTTP2ClientConnectionClose http1 tests
794f743 
These occasionally flake on CI:

https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/121200/pull-kubernetes-unit-go-compatibility/1712589824344461312

=== Failed
=== FAIL: vendor/k8s.io/apiserver/pkg/endpoints/filters TestUnauthenticatedHTTP2ClientConnectionClose/other_skip=true/http/1.1 (0.19s)
    authentication_test.go:653: expect TCP connection: 1, actual: 2
        --- FAIL: TestUnauthenticatedHTTP2ClientConnectionClose/other_skip=true/http/1.1 (0.19s)

=== FAIL: vendor/k8s.io/apiserver/pkg/endpoints/filters TestUnauthenticatedHTTP2ClientConnectionClose/other_skip=true (0.23s)
    --- FAIL: TestUnauthenticatedHTTP2ClientConnectionClose/other_skip=true (0.23s)

=== FAIL: vendor/k8s.io/apiserver/pkg/endpoints/filters TestUnauthenticatedHTTP2ClientConnectionClose (2.30s)

Signed-off-by: Monis Khan <mok@microsoft.com>
@liggitt @enj
Register UnauthenticatedHTTP2DOSMitigation into kube components
bb9bb79
@k8s-ci-robot
Copy link
Contributor

Adding the "do-not-merge/release-note-label-needed" label because no release-note block was detected, please follow our release note process to remove it.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@k8s-ci-robot k8s-ci-robot added the do-not-merge/cherry-pick-not-approved Indicates that a PR is not yet approved to merge into a release branch. label Nov 16, 2023
@k8s-ci-robot k8s-ci-robot added this to the v1.24 milestone Nov 16, 2023
@k8s-ci-robot k8s-ci-robot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Nov 16, 2023
@k8s-ci-robot
Copy link
Contributor

This cherry pick PR is for a release branch and has not yet been approved by Release Managers.
Adding the do-not-merge/cherry-pick-not-approved label.

To merge this cherry pick, it must first be approved (/lgtm + /approve) by the relevant OWNERS.

If you didn't cherry-pick this change to all supported release branches, please leave a comment describing why other cherry-picks are not needed to speed up the review process.

If you're not sure is it required to cherry-pick this change to all supported release branches, please consult the cherry-pick guidelines document.

AFTER it has been approved by code owners, please leave the following comment on a line by itself, with no leading whitespace: /cc kubernetes/release-managers

(This command will request a cherry pick review from Release Managers and should work for all GitHub users, whether they are members of the Kubernetes GitHub organization or not.)

For details on the patch release process and schedule, see the Patch Releases page.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@k8s-ci-robot k8s-ci-robot added do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. do-not-merge/needs-kind Indicates a PR lacks a `kind/foo` label and requires one. do-not-merge/needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. labels Nov 16, 2023
@k8s-ci-robot
Copy link
Contributor

This issue is currently awaiting triage.

If a SIG or subproject determines this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@k8s-ci-robot k8s-ci-robot added the needs-priority Indicates a PR lacks a `priority/foo` label and requires one. label Nov 16, 2023
@enj enj closed this Nov 16, 2023
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: enj
Once this PR has been reviewed and has the lgtm label, please assign deads2k for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added sig/api-machinery Categorizes an issue or PR as relevant to SIG API Machinery. and removed do-not-merge/needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. labels Nov 16, 2023
@cici37 cici37 added triage/accepted Indicates an issue or PR is ready to be actively worked on. and removed needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. labels Dec 5, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dchen1107 dchen1107 Awaiting requested review from dchen1107

@deads2k deads2k Awaiting requested review from deads2k

Assignees
No one assigned
Labels
area/apiserver cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. do-not-merge/cherry-pick-not-approved Indicates that a PR is not yet approved to merge into a release branch. do-not-merge/needs-kind Indicates a PR lacks a `kind/foo` label and requires one. do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. sig/api-machinery Categorizes an issue or PR as relevant to SIG API Machinery. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. triage/accepted Indicates an issue or PR is ready to be actively worked on.
Projects
None yet
Milestone
v1.24
Development

Successfully merging this pull request may close these issues.

None yet
4 participants
@enj @k8s-ci-robot @cici37 @liggitt