Implement RequiredNumberLabels query limit
#8918
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
github-actions
bot
added
the
type/docs
salvacorts
changed the title
RequiredNumberLabels limit
salvacorts
changed the title
RequiredNumberLabels limitRequiredNumberLabels query limit
owen-d
approved these changes
Mar 28, 2023
| original := l.CombinedLimits.RequiredNumberLabels(ctx, userID) | ||
| requestLimits := ExtractQueryLimitsContext(ctx) | ||
| if requestLimits == nil || requestLimits.RequiredNumberLabels == 0 || requestLimits.RequiredNumberLabels > original { | ||
| level.Debug(logutil.WithContext(ctx, l.logger)).Log("msg", "using original limit") |
There was a problem hiding this comment.
This should log the tenant, original & requested limits we compared.
edit: I think all the other limits in this file should do the same, but I'm fine if that's all left for another PR.
There was a problem hiding this comment.
Agreed with doing that on a separate PR.
DylanGuedes
requested changes
Mar 28, 2023
pkg/util/querylimits/propagation.go
Outdated
| MaxEntriesLimitPerQuery int `json:"maxEntriesLimitPerQuery,omitempty"` | ||
| QueryTimeout model.Duration `json:"queryTimeout,omitempty"` | ||
| RequiredLabels []string `json:"requiredLabels,omitempty"` | ||
| RequiredNumberLabels int `json:"requiredNumberLabels,omitempty"` |
There was a problem hiding this comment.
double checking: shouldn't the json here be "requiredNumberLabelMatchers"?
owen-d
reviewed
Mar 28, 2023
Co-authored-by: Dylan Guedes <djmgguedes@gmail.com>
|
@DylanGuedes @owen-d thanks for the review! I believe I've addressed all the pending comments. |
DylanGuedes
requested changes
Mar 29, 2023
Co-authored-by: Dylan Guedes <djmgguedes@gmail.com>
jeschkies
requested changes
Mar 29, 2023
| @@ -119,13 +128,17 @@ func TestLimiter_RejectHighLimits(t *testing.T) { | |||
| MaxEntriesLimitPerQuery: 10, | |||
| QueryTimeout: model.Duration(30 * time.Second), | |||
| MaxQueryBytesRead: 10, | |||
| // In this case, the higher it is, the more restrictive. | |||
| // Therefore, we return the query-time limit as it's more restrictive than the original. | |||
There was a problem hiding this comment.
Is this comment correct here?
There was a problem hiding this comment.
Yes. See #8918 (comment)
| MaxEntriesLimitPerQuery int `json:"maxEntriesLimitPerQuery,omitempty"` | ||
| QueryTimeout model.Duration `json:"queryTimeout,omitempty"` | ||
| RequiredLabels []string `json:"requiredLabels,omitempty"` | ||
| RequiredNumberLabels int `json:"requiredNumberLabelMatchers,omitempty"` |
There was a problem hiding this comment.
I've discussed the banking with Steven. We should omit "Matchers" and use minimumLabelsCount or minimumLabelsNumner.
5 tasks
salvacorts
added a commit
that referenced
this pull request
Mar 30, 2023
**What this PR does / why we need it**: Followup PR for #8918 renaming config. See https://github.com/grafana/loki/pull/8918/files#r1151820792. **Which issue(s) this PR fixes**: Fixes grafana/loki-private#699 **Special notes for your reviewer**: **Checklist** - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [x] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` --------- Co-authored-by: Dylan Guedes <djmgguedes@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What this PR does / why we need it:
As pointed out in #8851, some queries can impose a great workload on a cluster by selecting too many streams.
Similarly to the
RequiredLabelslimit introduced at #8851, here we add a new limitRequiredNumberLabelsto require queries to specify at least N label. For example, if the limit is set to 2, then the query should contain at least 2 label matchers.This limit can be configured per tenant and at query time.
Which issue(s) this PR fixes:
Fixes https://github.com/grafana/loki-private/issues/699
Special notes for your reviewer:
Checklist
CONTRIBUTING.mdguide (required)CHANGELOG.mdupdateddocs/sources/upgrading/_index.md