Resync upstream prometheus #641

pracucci · 2024-06-07T06:32:48Z

What has changed upstream

In this analysis I'm going to exclude:

Vendor updates PRs
Changes to tests

Impacting Mimir:

promql: ensure series in matrix values returned for instant queries are sorted (PR)
[ENHANCEMENT] TSDB: Faster exemplars (PR)
tsdb: add details to duplicate sample error (PR)
Improve contains check done by FastRegexMatcher (PR)
Implement histogram statistics decoder (PR)
Fix Group.Equals() to take in account the new queryOffset too (PR)

Not impacting Mimir:

Linode: bugfix, resolves partial fetch problem in 2.52 when service discovery would return more than exceeds 500 elements (PR)
tsdb: expose hook in tsdb to allow customizing compactor (PR)
Add configuration option for GOGC, default to 50 (PR)
RuleQueryOffset: Add omitempty for the global configuration (PR)

Changes I've done after merging upstream

40c9c28

Signed-off-by: tyltr <tylitianrui@126.com>

Bumps [@forevolve/bootstrap-dark](https://github.com/ForEvolve/bootstrap-dark) from 2.1.1 to 4.0.2. - [Release notes](https://github.com/ForEvolve/bootstrap-dark/releases) - [Commits](ForEvolve/bootstrap-dark@v2.1.1...v4.0.2) --- updated-dependencies: - dependency-name: "@forevolve/bootstrap-dark" dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>

Prepare v2.52.0 release

Signed-off-by: Charles Korn <charles.korn@grafana.com>

…them for pagination Signed-off-by: David Andruczyk <dandrucz@akamai.com>

Linode: bugfix, resolves partial fetch problem in 2.52 when service discovery would return more than exceeds 500 elements

Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>

Test with different amounts of capacity and exemplars, so that sometimes new exemplars are evicting older exemplars. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>

One exemplar per series is not a typical workload. Make it the same as `BenchmarkAddExemplar`. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>

In many cases we already have a pointer to the entry. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>

Saves memory and effort. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>

This saves having to zero the buffer every time. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>

Goes faster. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>

Prepare v2.52.1 release

[ENHANCEMENT] TSDB: Faster exemplars

# Conflicts: # promql/engine_test.go

promql: ensure series in matrix values returned for instant queries are sorted

Bumps [github.com/prometheus/client_golang](https://github.com/prometheus/client_golang) from 1.19.0 to 1.19.1. - [Release notes](https://github.com/prometheus/client_golang/releases) - [Changelog](https://github.com/prometheus/client_golang/blob/main/CHANGELOG.md) - [Commits](prometheus/client_golang@v1.19.0...v1.19.1) --- updated-dependencies: - dependency-name: github.com/prometheus/client_golang dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com>

Bumps [actions/checkout](https://github.com/actions/checkout) from 4.1.4 to 4.1.6. - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](actions/checkout@v4.1.4...a5ac7e5) --- updated-dependencies: - dependency-name: actions/checkout dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com>

Bumps [ossf/scorecard-action](https://github.com/ossf/scorecard-action) from 2.3.1 to 2.3.3. - [Release notes](https://github.com/ossf/scorecard-action/releases) - [Changelog](https://github.com/ossf/scorecard-action/blob/main/RELEASE.md) - [Commits](ossf/scorecard-action@0864cf1...dc50aa9) --- updated-dependencies: - dependency-name: ossf/scorecard-action dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com>

Bumps [github.com/hetznercloud/hcloud-go/v2](https://github.com/hetznercloud/hcloud-go) from 2.7.2 to 2.9.0. - [Release notes](https://github.com/hetznercloud/hcloud-go/releases) - [Changelog](https://github.com/hetznercloud/hcloud-go/blob/main/CHANGELOG.md) - [Commits](hetznercloud/hcloud-go@v2.7.2...v2.9.0) --- updated-dependencies: - dependency-name: github.com/hetznercloud/hcloud-go/v2 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com>

…er/compute/armcompute/v5 Bumps [github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/compute/armcompute/v5](https://github.com/Azure/azure-sdk-for-go) from 5.6.0 to 5.7.0. - [Release notes](https://github.com/Azure/azure-sdk-for-go/releases) - [Changelog](https://github.com/Azure/azure-sdk-for-go/blob/main/documentation/release.md) - [Commits](Azure/azure-sdk-for-go@sdk/resourcemanager/compute/armcompute/v5.6.0...sdk/resourcemanager/compute/armcompute/v5.7.0) --- updated-dependencies: - dependency-name: github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/compute/armcompute/v5 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com>

Bumps [actions/setup-go](https://github.com/actions/setup-go) from 5.0.0 to 5.0.1. - [Release notes](https://github.com/actions/setup-go/releases) - [Commits](actions/setup-go@0c52d54...cdcb360) --- updated-dependencies: - dependency-name: actions/setup-go dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com>

…ctions/checkout-4.1.6 build(deps): bump actions/checkout from 4.1.4 to 4.1.6

…ssf/scorecard-action-2.3.3 build(deps): bump ossf/scorecard-action from 2.3.1 to 2.3.3

…b.com/hetznercloud/hcloud-go/v2-2.9.0 build(deps): bump github.com/hetznercloud/hcloud-go/v2 from 2.7.2 to 2.9.0

…b.com/Azure/azure-sdk-for-go/sdk/resourcemanager/compute/armcompute/v5-5.7.0 build(deps): bump github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/compute/armcompute/v5 from 5.6.0 to 5.7.0

…/ui/nock-13.5.4 build(deps-dev): bump nock from 13.4.0 to 13.5.4 in /web/ui

…/ui/codemirror/language-6.10.2 build(deps): bump @codemirror/language from 6.9.3 to 6.10.2 in /web/ui

…/ui/fortawesome/free-solid-svg-icons-6.5.2 build(deps): bump @fortawesome/free-solid-svg-icons from 6.5.1 to 6.5.2 in /web/ui

…/ui/lezer/generator-1.7.0 build(deps-dev): bump @lezer/generator from 1.5.1 to 1.7.0 in /web/ui

…/ui/moment-timezone-0.5.45 build(deps): bump moment-timezone from 0.5.43 to 0.5.45 in /web/ui

…/ui/downshift-9.0.6 build(deps): bump downshift from 7.6.2 to 9.0.6 in /web/ui

…/ui/types/jquery-3.5.30 build(deps-dev): bump @types/jquery from 3.5.29 to 3.5.30 in /web/ui

Bumps [http-proxy-middleware](https://github.com/chimurai/http-proxy-middleware) from 2.0.6 to 3.0.0. - [Release notes](https://github.com/chimurai/http-proxy-middleware/releases) - [Changelog](https://github.com/chimurai/http-proxy-middleware/blob/master/CHANGELOG.md) - [Commits](chimurai/http-proxy-middleware@v2.0.6...v3.0.0) --- updated-dependencies: - dependency-name: http-proxy-middleware dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com>

Implement histogram statistics decoder This commit speeds up histogram_count and histogram_sum functions on native histograms. The idea is to have separate decoders which can be used by the engine to only read count/sum values from histogram objects. This should help with reducing allocations when decoding histograms, as well as with speeding up aggregations like sum since they will be done on floats and not on histogram objects. Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com> --------- Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com> Co-authored-by: Anthony Mirabella <a9@aneurysm9.com>

…ed error message (#14038) Add ability to assert that a query fails with a particular error message This also adds documentation for the test scripting language in general, including the new feature. Signed-off-by: Charles Korn <charles.korn@grafana.com> --------- Signed-off-by: Charles Korn <charles.korn@grafana.com>

…/ui/types/node-20.14.2 build(deps-dev): bump @types/node from 20.10.4 to 20.14.2 in /web/ui

Bumps [@types/sinon](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/sinon) from 10.0.20 to 17.0.3. - [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases) - [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/sinon) --- updated-dependencies: - dependency-name: "@types/sinon" dependency-type: direct:development update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com>

Bumps [@codemirror/commands](https://github.com/codemirror/commands) from 6.3.2 to 6.6.0. - [Changelog](https://github.com/codemirror/commands/blob/main/CHANGELOG.md) - [Commits](codemirror/commands@6.3.2...6.6.0) --- updated-dependencies: - dependency-name: "@codemirror/commands" dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com>

Bumps [eslint-config-prettier](https://github.com/prettier/eslint-config-prettier) from 8.10.0 to 9.1.0. - [Changelog](https://github.com/prettier/eslint-config-prettier/blob/main/CHANGELOG.md) - [Commits](prettier/eslint-config-prettier@v8.10.0...v9.1.0) --- updated-dependencies: - dependency-name: eslint-config-prettier dependency-type: direct:development update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com>

…tains Improve contains check done by FastRegexMatcher

…/ui/codemirror/commands-6.6.0 build(deps): bump @codemirror/commands from 6.3.2 to 6.6.0 in /web/ui

…/ui/eslint-config-prettier-9.1.0 build(deps-dev): bump eslint-config-prettier from 8.10.0 to 9.1.0 in /web/ui

…/ui/types/sinon-17.0.3 build(deps-dev): bump @types/sinon from 10.0.20 to 17.0.3 in /web/ui

…/ui/http-proxy-middleware-3.0.0 build(deps): bump http-proxy-middleware from 2.0.6 to 3.0.0 in /web/ui

Signed-off-by: Marco Pracucci <marco@pracucci.com>

…stream

CLAassistant · 2024-06-07T06:32:57Z

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
3 out of 5 committers have signed the CLA.

✅ charleskorn
✅ bboreham
✅ pracucci
❌ fpetkovski
❌ SuperQ
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

Signed-off-by: Marco Pracucci <marco@pracucci.com>

pracucci · 2024-06-07T06:54:44Z

promql/parser/ast.go

-	LabelMatchers []*labels.Matcher
+	Offset               time.Duration
+	Timestamp            *int64
+	SkipHistogramBuckets bool     // Set when decoding native histogram buckets is not needed for query evaluation.


Reminder for me and @krajorama. When updating mimir-prometheus in Mimir, we should check if there's anything to do about this (e.g. query sharding).

pracucci · 2024-06-07T06:55:41Z

rules/manager_test.go

@@ -1523,181 +1523,6 @@ func TestRuleHealthUpdates(t *testing.T) {
 	require.Equal(t, HealthBad, rules.Health())
 }

-func TestGroup_Equals(t *testing.T) {


Note to reviewers: merged with the new upstream test defined in group_test.go.

krajorama · 2024-06-07T07:56:29Z

promql/histogram_stats_iterator.go

+
+func (f *histogramStatsIterator) setLastH(h *histogram.Histogram) {
+	if f.lastH == nil {
+		f.lastH = h.Copy()


These copies might introduce some overhead in mimir query frontend. Since it's easy to turn off this optimization, we can do that if it's noticeable, related to prometheus/prometheus#13679

Do you think we should preemptively turn it off?

No. There isn't much native histogram usage yet that we know of, so in absolute terms it's fine. I hope to get back to migration sometime and introduce more usage at which point I'll be able to do profiling and check where we are

krajorama

LGTM

tylitianrui and others added 30 commits February 15, 2024 20:40

chore:fix typo

084420b

Signed-off-by: tyltr <tylitianrui@126.com>

Prepare v2.52.0 release

df05132

Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>

Merge pull request #14063 from prometheus/prepare-2.52

879d809

Prepare v2.52.0 release

Capture timing information while sorting

0e934db

Signed-off-by: Charles Korn <charles.korn@grafana.com>

Document sorting behaviour

76b1237

Signed-off-by: Charles Korn <charles.korn@grafana.com>

BUGFIX: Need seperate listOptions structs since linodego writes into …

851f68d

…them for pagination Signed-off-by: David Andruczyk <dandrucz@akamai.com>

Merge pull request #14141 from dandrucz/LinodeListOptsFix

296dd12

Linode: bugfix, resolves partial fetch problem in 2.52 when service discovery would return more than exceeds 500 elements

Prepare v2.52.1 release

3c1aadd

Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>

[Test] TSDB: let BenchmarkAddExemplar reuse slots

929fbf8

Test with different amounts of capacity and exemplars, so that sometimes new exemplars are evicting older exemplars. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>

[Test] TSDB: BenchmarkResizeExemplar multiple per series

f0c50b5

One exemplar per series is not a typical workload. Make it the same as `BenchmarkAddExemplar`. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>

[ENHANCEMENT] TSDB: Reduce map lookups on exemplar index

3eb5581

In many cases we already have a pointer to the entry. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>

[ENHANCEMENT] TSDB: Eliminate pointer when storing exemplars

c0bb156

Saves memory and effort. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>

[ENHANCEMENT] TSDB: let Resize re-use buffer

7d98487

This saves having to zero the buffer every time. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>

[ENHANCEMENT] TSDB: Save map lookup on validation

3ee52ab

Goes faster. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>

Merge pull request #14164 from prometheus/prepare-v2.52.1

3ac5dfb

Prepare v2.52.1 release

Merge pull request #14080 from bboreham/faster-exemplars

e47474d

[ENHANCEMENT] TSDB: Faster exemplars

Merge branch 'main' into sort-matrix-series

24ef000

# Conflicts: # promql/engine_test.go

Merge branch 'main' into resolveconflicts-2.52.1

d73537c

Merge pull request #14083 from charleskorn/sort-matrix-series

bfdca40

promql: ensure series in matrix values returned for instant queries are sorted

Merge pull request #14179 from prometheus/dependabot/github_actions/a…

0520e86

…ctions/checkout-4.1.6 build(deps): bump actions/checkout from 4.1.4 to 4.1.6

Merge pull request #14181 from prometheus/dependabot/github_actions/o…

2de6c5b

…ssf/scorecard-action-2.3.3 build(deps): bump ossf/scorecard-action from 2.3.1 to 2.3.3

Merge pull request #14186 from prometheus/dependabot/go_modules/githu…

27f2dde

…b.com/hetznercloud/hcloud-go/v2-2.9.0 build(deps): bump github.com/hetznercloud/hcloud-go/v2 from 2.7.2 to 2.9.0

Merge pull request #14188 from prometheus/dependabot/go_modules/githu…

5b32f1f

…b.com/Azure/azure-sdk-for-go/sdk/resourcemanager/compute/armcompute/v5-5.7.0 build(deps): bump github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/compute/armcompute/v5 from 5.6.0 to 5.7.0

SuperQ and others added 21 commits June 6, 2024 15:54

Merge pull request #14266 from prometheus/dependabot/npm_and_yarn/web…

50e3712

…/ui/nock-13.5.4 build(deps-dev): bump nock from 13.4.0 to 13.5.4 in /web/ui

Merge pull request #14263 from prometheus/dependabot/npm_and_yarn/web…

d433d20

…/ui/codemirror/language-6.10.2 build(deps): bump @codemirror/language from 6.9.3 to 6.10.2 in /web/ui

Merge pull request #14233 from prometheus/dependabot/npm_and_yarn/web…

fed911e

…/ui/fortawesome/free-solid-svg-icons-6.5.2 build(deps): bump @fortawesome/free-solid-svg-icons from 6.5.1 to 6.5.2 in /web/ui

Merge pull request #14264 from prometheus/dependabot/npm_and_yarn/web…

afd26e5

…/ui/lezer/generator-1.7.0 build(deps-dev): bump @lezer/generator from 1.5.1 to 1.7.0 in /web/ui

Merge pull request #14257 from prometheus/dependabot/npm_and_yarn/web…

5f12cf5

…/ui/moment-timezone-0.5.45 build(deps): bump moment-timezone from 0.5.43 to 0.5.45 in /web/ui

Merge pull request #14265 from prometheus/dependabot/npm_and_yarn/web…

89ab04b

…/ui/downshift-9.0.6 build(deps): bump downshift from 7.6.2 to 9.0.6 in /web/ui

Merge pull request #14260 from prometheus/dependabot/npm_and_yarn/web…

ee0527e

…/ui/types/jquery-3.5.30 build(deps-dev): bump @types/jquery from 3.5.29 to 3.5.30 in /web/ui

Merge pull request #14262 from prometheus/dependabot/npm_and_yarn/web…

687b9c2

…/ui/types/node-20.14.2 build(deps-dev): bump @types/node from 20.10.4 to 20.14.2 in /web/ui

Merge pull request #14173 from pracucci/fastregexmatcher-optimize-con…

6fb738a

…tains Improve contains check done by FastRegexMatcher

Merge pull request #14271 from prometheus/dependabot/npm_and_yarn/web…

77f7820

…/ui/codemirror/commands-6.6.0 build(deps): bump @codemirror/commands from 6.3.2 to 6.6.0 in /web/ui

Merge pull request #14272 from prometheus/dependabot/npm_and_yarn/web…

3ca4000

…/ui/eslint-config-prettier-9.1.0 build(deps-dev): bump eslint-config-prettier from 8.10.0 to 9.1.0 in /web/ui

Merge pull request #14270 from prometheus/dependabot/npm_and_yarn/web…

be0d512

…/ui/types/sinon-17.0.3 build(deps-dev): bump @types/sinon from 10.0.20 to 17.0.3 in /web/ui

Merge pull request #14244 from prometheus/dependabot/npm_and_yarn/web…

51e303d

…/ui/http-proxy-middleware-3.0.0 build(deps): bump http-proxy-middleware from 2.0.6 to 3.0.0 in /web/ui

Fix Group.Equals() to take in account the new queryOffset too (#14273)

edd5588

Signed-off-by: Marco Pracucci <marco@pracucci.com>

Merge remote-tracking branch 'remotes/prometheus/main' into resync-up…

9663c47

…stream

Merge TestGroup_Equals() with upstream one

40c9c28

Signed-off-by: Marco Pracucci <marco@pracucci.com>

pracucci commented Jun 7, 2024

View reviewed changes

krajorama reviewed Jun 7, 2024

View reviewed changes

krajorama approved these changes Jun 7, 2024

View reviewed changes

pracucci merged commit fce8e33 into main Jun 7, 2024
8 of 9 checks passed

pracucci deleted the resync-upstream branch June 7, 2024 08:01

pracucci mentioned this pull request Jun 7, 2024

Update vendored mimir-prometheus grafana/mimir#8305

Merged

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Resync upstream prometheus #641

Resync upstream prometheus #641

pracucci commented Jun 7, 2024 •

edited

Loading

CLAassistant commented Jun 7, 2024 •

edited

Loading

pracucci Jun 7, 2024

pracucci Jun 7, 2024

krajorama Jun 7, 2024

pracucci Jun 7, 2024

krajorama Jun 10, 2024

krajorama left a comment

Resync upstream prometheus #641

Resync upstream prometheus #641

Conversation

pracucci commented Jun 7, 2024 • edited Loading

What has changed upstream

Changes I've done after merging upstream

CLAassistant commented Jun 7, 2024 • edited Loading

pracucci Jun 7, 2024

Choose a reason for hiding this comment

pracucci Jun 7, 2024

Choose a reason for hiding this comment

krajorama Jun 7, 2024

Choose a reason for hiding this comment

pracucci Jun 7, 2024

Choose a reason for hiding this comment

krajorama Jun 10, 2024

Choose a reason for hiding this comment

krajorama left a comment

Choose a reason for hiding this comment

pracucci commented Jun 7, 2024 •

edited

Loading

CLAassistant commented Jun 7, 2024 •

edited

Loading