Remove MaxConcurrentStreams in the proxy peering gRPC server.
#39218
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jentfoo
approved these changes
Mar 11, 2024
rosstimothy
approved these changes
Mar 11, 2024
|
Nice find @espadolini! |
zmb3
approved these changes
Mar 11, 2024
|
@espadolini See the table below for backport results.
|
This was referenced Mar 12, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The mitigation for CVE-2023-44487 added a limit of 1000 concurrent streams to all gRPC servers used in Teleport; this PR removes that limit for the proxy peering gRPC server.
The proxy peering server uses transport authentication to check that the client is using valid credentials for a builtin
Proxyrole in the same cluster (i.e. it's another proxy server), and it's very unique in that the single gRPC service handled by the server consists of a single bidirectional rpc used to forward connections - with potentially unbounded duration - between proxy servers. With this limit in place, new connections across any given pair of proxy servers will start timing out once there's 1000 already active connections. In Teleport Cloud, because there's two proxies in each region by default, this results in timeouts at 3900 or so connections, which is lower than what the Cloud platform was previously tested to be able to handle.changelog: raised concurrent connection limits between Teleport Cloud regions and in clusters that use proxy peering