-
Notifications
You must be signed in to change notification settings - Fork 1.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
app/vmagent: added global aggregator #6268
Conversation
83e72d9
to
26e09dc
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should outline in docs and flag description the main purpose of the global aggregation config: to fan-out aggregated data to multiple URLs.
Have you tried comparing resource usage of aggregating data globally and per-url before sending to 2 destinations?
@@ -491,13 +479,28 @@ func tryPush(at *auth.Token, wr *prompbmarshal.WriteRequest, forceDropSamplesOnF | |||
} | |||
sortLabelsIfNeeded(tssBlock) | |||
tssBlock = limitSeriesCardinality(tssBlock) | |||
sas := sasGlobal.Load() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should we retrieve sas before for len(tss) > 0 {
on L:448?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
moved it
yes, tested it and there was a significant difference in resource usage |
39caea2
to
0dfb022
Compare
0dfb022
to
a3b6974
Compare
* clarify the deprecation note in Changelog * explain the purpose of the stream aggregation change in Changelog * remove mentions of `global` aggregation from docs. Use it as usual and per-url aggregation in wording, so it comes as default aggregation option (wide case) and then user can apply it per-url (narrow case) * add #Routing section to stream aggregation docs * update vmagent's list of flags Signed-off-by: hagen1778 <roman@victoriametrics.com>
@AndrewChubatiuk please see changes at 8fac4a6 |
yes, let's merge it |
Describe Your Changes
Added global stream aggregation for VMAgent
Checklist
The following checks are mandatory: