carbonserver: introducing request-timeout, heavy-glob-query-rate-limiters and api-per-path-rate-limiters for read traffic regulation #469
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
bom-d-van
force-pushed
the
carbonserver/resource-control
branch
from
8cc5262
to
9a56947
Compare
bom-d-van
changed the title
bom-d-van
force-pushed
the
carbonserver/resource-control
branch
3 times, most recently
from
b506f11
to
9496f7b
Compare
timtofan
reviewed
Jun 3, 2022
…ters and api-per-path-rate-limiters for read traffic regulation Three new types of read path/traffic control configs are introduced in this commit: * request-timeout: it is designed to control how long each api call in carbonserver can run. The existing timeouts like read, write and idle are for http.Server, but this new timeout is for each API call in carbonserver. * heavy-glob-query-rate-limiters are relatively narrow controls against queries that might causes high cpu and memory consumption due to matching over too many metrics or nodes at the same time, queries like: "*.*.*.*.*.*.*.*.*keyword*". For these types of queries, trigram might be able to handle it better, but for trie and filesystem glob, it's might be too expensive when the index tree is large. * api-per-path-rate-limiters are used for strict api call rate limiting. All registered API paths (see carbonserver.Listen for a full list) can be controlled separately with it.
bom-d-van
force-pushed
the
carbonserver/resource-control
branch
from
9496f7b
to
80d596f
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Three new types of read path/traffic control configs are introduced in this commit:
request-timeout: it is designed to control how long each api call in
carbonserver can run. The existing timeouts like read, write and idle are for
http.Server, but this new timeout is for each API call in carbonserver.
heavy-glob-query-rate-limiters are relatively narrow controls against queries
that might causes high cpu and memory consumption due to matching over too
many metrics or nodes at the same time, queries
like:
*.*.*.*.*.*.*.*.*keyword*. For these types of queries, trigram mightbe able to handle it better, but for trie and filesystem glob, it's might be too
expensive when the index tree is large.
api-per-path-rate-limiters are used for strict api call rate limiting. All
registered API paths (see carbonserver.Listen for a full list) can be
controlled separately with it.