Using subring on getShardedRules when shuffle sharding #4466
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
alanprot
force-pushed
the
enhancement/using-subring-on-rulers-getShardedRules
branch
6 times, most recently
from
b7a0438
to
4b4204e
Compare
alvinlin123
reviewed
Sep 8, 2021
alanprot
force-pushed
the
enhancement/using-subring-on-rulers-getShardedRules
branch
from
1ed3818
to
d3681b6
Compare
Signed-off-by: Alan Protasio <approtas@amazon.com>
Signed-off-by: Alan Protasio <approtas@amazon.com>
alanprot
force-pushed
the
enhancement/using-subring-on-rulers-getShardedRules
branch
from
d3681b6
to
25280c8
Compare
|
Hi... Can i get some attention on this PR? Situation like #4459 makes 100% of the APIs fail right now. |
pracucci
approved these changes
Sep 15, 2021
srijan55
pushed a commit
to srijan55/cortex
that referenced
this pull request
Nov 26, 2021
…#4466) * Using subring on getShardedRules when shuffle sharding Signed-off-by: Alan Protasio <approtas@amazon.com> Signed-off-by: Manish Kumar Gupta <manishkg@microsoft.com>
alvinlin123
pushed a commit
to ac1214/cortex
that referenced
this pull request
Jan 14, 2022
…#4466) * Using subring on getShardedRules when shuffle sharding Signed-off-by: Alan Protasio <approtas@amazon.com> Signed-off-by: Alvin Lin <alvinlin@amazon.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What this PR does:
GetRules API should call only the rulers on the tenant's subring when sharding strategy is set to
ShardingStrategyShuffle.Right now we are calling all rulers in the
GetRulesapi even if the sharding strategy is set toShardingStrategyShuffle. In this case, we should only call the rules in the tenant subring as those rules are the only ones that can load rules from this tenant. This change is specially important as right now we need all the rulers in the ring to be available or cortex will return an error - this means that if we have shardSize = 3 but 100 rulers in total, we will return 5xx if any of the 100 rules are down (for instance during deployment), even though the callers rules are spread only on 3 of them.Which issue(s) this PR fixes:
Fixes #
Checklist
CHANGELOG.mdupdated - the order of entries should be[CHANGE],[FEATURE],[ENHANCEMENT],[BUGFIX]