Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

store-gateway: add support for partial expanded postings #4667

Merged
merged 18 commits into from
Apr 11, 2023

Conversation

dimitarvdimitrov
Copy link
Contributor

@dimitarvdimitrov dimitarvdimitrov commented Apr 5, 2023

What this PR does

  • Adds total size in bytes of a posting group; this is the sum of all posting lists as they appear in the index
  • Changes the internal implementation of bucketIndexReader.ExpandedPostings to be able to handle partially resolved expanded postings. A partially resolved expanded posting list for a query is a posting list where not all the matchers from the query have been applied
Total size of posting groups

This PR uses the LabelValuesOffsets (instead of LabelValues) and PostingsOffset methods of the index header reader to calculate the size of each posting list and sum them to give the size of the whole posting group.

This is expected to increase the memory footprint of ExpandedPostings, although not by a lot. I am in the process of running benchmarks now.

Partially resolved expanded postings lists

This PR introduces a pluggable postingsSelectionStrategy interface which the bucketIndexReader uses to select only some posting groups. Currently, this interface has only one implementation selectAllStrategy, which just selects all posting groups.
The plan is to make the strategy selectable via a configuration option.

After running the strategy, the index reader fetches and then intersects/subtracts only the selected posting groups. The reader returns the original matchers of the posting groups that weren't selected - called "unapplied matchers" (naming suggestions welcome).

Caching

The unapplied matchers are stored in the cache item for expanded postings, so they can be applied in each cache fetch. Currently the code doesn't attempt to deal with applying the unapplied matchers, so ExpandedPostings returns an error if it finds any in a cache item. But since selectAllStrategy never returns unapplied matchers, this case shouldn't be reached.

This PR also changes the cache key of expanded postings in order to be able to run different strategies concurrently by different store-gateway replicas. The cache key now contains the name of the strategy that was used to derive the partial postings and the unapplied matchers. This has the side effect that the expanded postings cache will be "invalidated" after rolling out this change.

Which issue(s) this PR fixes or relates to

Related to #4593

Checklist

  • Tests updated
  • [n/a] Documentation added
  • [n/a] CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX]

Signed-off-by: Dimitar Dimitrov <dimitar.dimitrov@grafana.com>
Signed-off-by: Dimitar Dimitrov <dimitar.dimitrov@grafana.com>
@dimitarvdimitrov dimitarvdimitrov changed the title Cache unapplied matchers in ExpandedPostings store-gateway: add support for partial expanded postings Apr 5, 2023
Signed-off-by: Dimitar Dimitrov <dimitar.dimitrov@grafana.com>
@dimitarvdimitrov
Copy link
Contributor Author

I think it will make sense to also add a test case to make sure that the postings selection strategy is respected and the posting groups aren't mangled.

@colega
Copy link
Contributor

colega commented Apr 5, 2023

I would consider caching the final filtered list, and trying to retrieve that one from the expanded postings promise/cache. Is there any reason why caching the partial list + matchers would be more useful/efficient? Otherwise, if this approach is suboptimal for some reason, we'll repeat the suboptimal work multiple times.

Edit: if we do that, we don't even have to change the expanded postings cache key: the expanded postings don't depend on the strategy how we got them.

@dimitarvdimitrov
Copy link
Contributor Author

dimitarvdimitrov commented Apr 5, 2023

I ran the benchmarks Benchmarks.txt

The most notable point for me is that on all benchmarks, which involve the i=~"<unique_prefix>" matcher, the current implementation is slower (<= 60%) and allocates more memory (<= 100%). I'll be investigating this. I think the rest of the differences are not significant.

@dimitarvdimitrov
Copy link
Contributor Author

we discussed Oleg's comment offline and this is the TL;DR

The proposed implementation in this PR is less efficient and shouldn't have any theoretical benefits over caching the already filtered postings. However, caching the filtered list is non-trivial - it involves

  • solving sharding - should we filter even series that we know are outside our shard (via the series hash cache)?
  • deduplicating the SETs - do we want to do that - probably, but it's less easy
  • collecting the filtered postings through the iterations of loadingSeriesChunkRefsSetIterator (or the iterator that ends up filtering the series).

We agreed that it's ok to have the first iteration like this and then think about improving it. I hope that the added cost of filtering is dominated by the saved cost of processing large (and uncacheable) postings lists.

I've added this item to the parent issue #4593

Copy link
Collaborator

@pracucci pracucci left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great job! I haven't found any issue, so LGTM. I left some minor comments. I only have a small concern about the usage of YOLO string in decodeMatchers() for which I've left a question.

pkg/storegateway/bucket_index_postings.go Outdated Show resolved Hide resolved
pkg/storegateway/bucket_index_postings.go Outdated Show resolved Hide resolved
pkg/storegateway/bucket_index_reader.go Outdated Show resolved Hide resolved
pkg/storegateway/bucket_index_reader.go Outdated Show resolved Hide resolved
promise, loaded = r.expandedPostingsPromise(ctx, ms, stats)
returnRefs, cached, returnErr = promise(ctx)
returnRefs, unappliedMatchers, cached, returnErr = promise(ctx)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Naming: have you considered "lazyMatchers" given they get applied later? I don't feel strong about it, cause there's a risk of confusing them with the lazy posting groups.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm also not sure about mixing them with the lazy posting groups. Especially because lazy means different things in the two usages (used later vs resolved later).

What do you think about deferredMatchers, postponedMatchers, delayedMatchers?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

deferredMatchers

LGTM!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do feel pretty strong against deferredMatchers. The description of this PR talks about unapplied matchers, and IMO that doesn't require extra knowledge of the topic to understand. I could also buy postponedMatchers, or as I suggested in a separate comment, pendingMatchers. But I think deferred introduces a new term that increases congnitive load here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think pendingMatchers sounds good too. I'll wait for Marco to take a look too before renaming again.

pkg/storegateway/postings_codec.go Outdated Show resolved Hide resolved
pkg/storegateway/postings_codec.go Outdated Show resolved Hide resolved
pkg/storegateway/postings_codec.go Outdated Show resolved Hide resolved
pkg/storegateway/postings_codec.go Outdated Show resolved Hide resolved
pkg/storegateway/postings_codec.go Outdated Show resolved Hide resolved
Signed-off-by: Dimitar Dimitrov <dimitar.dimitrov@grafana.com>
Signed-off-by: Dimitar Dimitrov <dimitar.dimitrov@grafana.com>
Signed-off-by: Dimitar Dimitrov <dimitar.dimitrov@grafana.com>
Signed-off-by: Dimitar Dimitrov <dimitar.dimitrov@grafana.com>
Signed-off-by: Dimitar Dimitrov <dimitar.dimitrov@grafana.com>
Signed-off-by: Dimitar Dimitrov <dimitar.dimitrov@grafana.com>
Signed-off-by: Dimitar Dimitrov <dimitar.dimitrov@grafana.com>
Signed-off-by: Dimitar Dimitrov <dimitar.dimitrov@grafana.com>
Signed-off-by: Dimitar Dimitrov <dimitar.dimitrov@grafana.com>
Signed-off-by: Dimitar Dimitrov <dimitar.dimitrov@grafana.com>
Signed-off-by: Dimitar Dimitrov <dimitar.dimitrov@grafana.com>
Copy link
Collaborator

@pracucci pracucci left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good job, LGTM!

CHANGELOG.md Outdated Show resolved Hide resolved
dimitarvdimitrov and others added 2 commits April 6, 2023 12:02
Co-authored-by: Marco Pracucci <marco@pracucci.com>
Signed-off-by: Dimitar Dimitrov <dimitar.dimitrov@grafana.com>
@dimitarvdimitrov
Copy link
Contributor Author

dimitarvdimitrov commented Apr 6, 2023

i looked into the increased memory and CPU of the benchmarks with i=~"<unique_prefix>.*"

So what these test cases do is select a very small number of all label values for the i label. The i label has 100K different values across 5M series. The benchmark selects only one single value. But the way we preallocate the slice of label values is that we assume the query will select the majority of the values.

So in these benchmarks the memory was dominated by the wrong estimation of the number of values that will be selected. Now we increased the memory footprint of the item of each label value item (it was only string, now it's struct{ string, struct { int64, int64 }})

So the benchmarks showed almost a 100% increase. I'll see if we can be better in doing these estimations

  • maybe run the filters once to only count the matches and then iterate again to collect them?
  • or use the prefix to tune the estionation - we can only look at the groups of 32 label values that match the prefix; if there's no prefix, then fallback to the "all values" estimation

This wasn't caused by this PR, but this PR makes it worse.

@dimitarvdimitrov
Copy link
Contributor Author

dimitarvdimitrov commented Apr 6, 2023

so i tried running the filter twice, and counting the matches before running it again to collect the matches. This showed some good improvement in some benchmarks but also regression on others (ebfe653)

BucketIndexReader_ExpandedPostings with two passes
goos: darwin
goarch: arm64
pkg: github.com/grafana/mimir/pkg/storegateway
                                                                                         │ before-cab54ae516ae69b9334b9c779c6225f4dd1e3d13.txt │ after-ebfe653ef1fd8518de24f9fd8ad44016fbc80643.txt │
                                                                                         │                       sec/op                        │           sec/op             vs base               │
BucketIndexReader_ExpandedPostings/stream_reader/n="X"-10                                                                         10.10µ ± ∞ ¹                  10.37µ ± ∞ ¹        ~ (p=0.057 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="X",j="foo"-10                                                                 20.61µ ± ∞ ¹                  20.80µ ± ∞ ¹        ~ (p=0.200 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="X",j!="foo"-10                                                                20.55µ ± ∞ ¹                  21.03µ ± ∞ ¹   +2.33% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/j=~"XXX|YYY"-10                                                                  911.6n ± ∞ ¹                  959.6n ± ∞ ¹   +5.27% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/j=~"X.+"-10                                                                      904.2n ± ∞ ¹                  920.2n ± ∞ ¹        ~ (p=0.343 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/i=~"X|Y|Z"-10                                                                    41.22µ ± ∞ ¹                  28.38µ ± ∞ ¹  -31.15% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1"-10                                                                         4.652m ± ∞ ¹                  4.573m ± ∞ ¹   -1.70% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",j="foo"-10                                                                 46.23m ± ∞ ¹                  44.60m ± ∞ ¹   -3.52% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/j="foo",n="1"-10                                                                 46.10m ± ∞ ¹                  44.68m ± ∞ ¹   -3.08% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",j!="foo"-10                                                                35.91m ± ∞ ¹                  35.22m ± ∞ ¹   -1.92% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/i=~".*"-10                                                                       87.58m ± ∞ ¹                  87.21m ± ∞ ¹        ~ (p=0.686 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/i=~".+"-10                                                                        1.315 ± ∞ ¹                   1.318 ± ∞ ¹        ~ (p=0.886 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/i=~"^.+$",j=~"X.+"-10                                                            8.415m ± ∞ ¹                 13.942m ± ∞ ¹  +65.68% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/i=~""-10                                                                          1.414 ± ∞ ¹                   1.445 ± ∞ ¹   +2.14% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/i!=""-10                                                                          1.315 ± ∞ ¹                   1.334 ± ∞ ¹   +1.44% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i=~".*",j="foo"-10                                                         50.66m ± ∞ ¹                  50.57m ± ∞ ¹        ~ (p=0.343 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="X",i=~"^.+$",j="foo"-10                                                       20.86µ ± ∞ ¹                  20.89µ ± ∞ ¹        ~ (p=1.000 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i=~".*",i!="2",j="foo"-10                                                  59.00m ± ∞ ¹                  58.20m ± ∞ ¹   -1.36% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i!=""-10                                                                    1.188 ± ∞ ¹                   1.198 ± ∞ ¹        ~ (p=0.057 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i!="",j="foo"-10                                                            1.214 ± ∞ ¹                   1.224 ± ∞ ¹        ~ (p=0.886 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i!="",j=~"X.+"-10                                                          7.658m ± ∞ ¹                 12.464m ± ∞ ¹  +62.77% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i!="",j=~"XXX|YYY"-10                                                      13.13µ ± ∞ ¹                  13.15µ ± ∞ ¹        ~ (p=0.486 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i=~"X|Y|Z",j="foo"-10                                                      50.22µ ± ∞ ¹                  51.17µ ± ∞ ¹        ~ (p=1.000 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i=~".+",j="foo"-10                                                          1.219 ± ∞ ¹                   1.701 ± ∞ ¹  +39.53% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i=~"1.+",j="foo"-10                                                        148.7m ± ∞ ¹                  196.7m ± ∞ ¹  +32.29% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i=~".+",i!="2",j="foo"-10                                                   1.229 ± ∞ ¹                   1.723 ± ∞ ¹  +40.24% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i=~".+",i!~"2.*",j="foo"-10                                                 1.362 ± ∞ ¹                   1.870 ± ∞ ¹  +37.38% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="X",i=~"^.+$",i!~"^.*2.*$",j="foo"-10                                          21.07µ ± ∞ ¹                  30.26µ ± ∞ ¹  +43.61% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/i=~"0xxx|1xxx|2xxx"-10                                                           128.7µ ± ∞ ¹                  122.7µ ± ∞ ¹   -4.63% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/i=~"(0|1|2)xxx"-10                                                               128.8µ ± ∞ ¹                  125.4µ ± ∞ ¹   -2.68% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/i=~"[0-2]xxx"-10                                                                 129.3µ ± ∞ ¹                  123.7µ ± ∞ ¹   -4.30% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/i!~[0-2]xxx-10                                                                   96.23m ± ∞ ¹                  95.88m ± ∞ ¹        ~ (p=0.486 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/i=~".*",_i!~[0-2]xxx-10                                                          102.0m ± ∞ ¹                  101.3m ± ∞ ¹        ~ (p=0.114 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/i=~"<unique_prefix>.+"-10                                                       194.28µ ± ∞ ¹                  51.45µ ± ∞ ¹  -73.52% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i=~"<unique_prefix>.+"-10                                                  1.497m ± ∞ ¹                  1.293m ± ∞ ¹  -13.66% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i!~"<unique_prefix>.+"-10                                                  5.236m ± ∞ ¹                  5.061m ± ∞ ¹        ~ (p=0.486 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/p!=""-10                                                                         20.15m ± ∞ ¹                  19.52m ± ∞ ¹   -3.12% (p=0.029 n=4)
geomean                                                                                                                           4.248m                        4.324m         +1.79%
¹ need >= 6 samples for confidence interval at level 0.95

                                                                                         │ before-cab54ae516ae69b9334b9c779c6225f4dd1e3d13.txt │ after-ebfe653ef1fd8518de24f9fd8ad44016fbc80643.txt │
                                                                                         │                        B/op                         │            B/op              vs base               │
BucketIndexReader_ExpandedPostings/stream_reader/n="X"-10                                                                        1.332Ki ± ∞ ¹                 1.386Ki ± ∞ ¹   +4.11% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="X",j="foo"-10                                                                2.004Ki ± ∞ ¹                 2.075Ki ± ∞ ¹   +3.53% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="X",j!="foo"-10                                                               2.004Ki ± ∞ ¹                 2.074Ki ± ∞ ¹   +3.51% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/j=~"XXX|YYY"-10                                                                  1008.0 ± ∞ ¹                  1064.0 ± ∞ ¹   +5.56% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/j=~"X.+"-10                                                                       952.0 ± ∞ ¹                  1008.0 ± ∞ ¹   +5.88% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/i=~"X|Y|Z"-10                                                                   2.050Ki ± ∞ ¹                 2.104Ki ± ∞ ¹   +2.67% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1"-10                                                                        12.13Mi ± ∞ ¹                 12.13Mi ± ∞ ¹   +0.00% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",j="foo"-10                                                                18.88Mi ± ∞ ¹                 18.88Mi ± ∞ ¹        ~ (p=0.686 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/j="foo",n="1"-10                                                                18.88Mi ± ∞ ¹                 18.88Mi ± ∞ ¹        ~ (p=0.486 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",j!="foo"-10                                                               18.88Mi ± ∞ ¹                 18.88Mi ± ∞ ¹        ~ (p=0.343 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/i=~".*"-10                                                                      273.6Mi ± ∞ ¹                 272.1Mi ± ∞ ¹   -0.56% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/i=~".+"-10                                                                      422.4Mi ± ∞ ¹                 423.9Mi ± ∞ ¹   +0.36% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/i=~"^.+$",j=~"X.+"-10                                                           9.167Mi ± ∞ ¹                10.692Mi ± ∞ ¹  +16.63% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/i=~""-10                                                                        218.6Mi ± ∞ ¹                 220.1Mi ± ∞ ¹   +0.70% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/i!=""-10                                                                        422.4Mi ± ∞ ¹                 423.9Mi ± ∞ ¹   +0.36% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i=~".*",j="foo"-10                                                        20.42Mi ± ∞ ¹                 18.88Mi ± ∞ ¹   -7.50% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="X",i=~"^.+$",j="foo"-10                                                      2.191Ki ± ∞ ¹                 2.246Ki ± ∞ ¹   +2.50% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i=~".*",i!="2",j="foo"-10                                                 21.65Mi ± ∞ ¹                 20.12Mi ± ∞ ¹   -7.07% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i!=""-10                                                                  193.1Mi ± ∞ ¹                 194.6Mi ± ∞ ¹   +0.79% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i!="",j="foo"-10                                                          199.9Mi ± ∞ ¹                 201.4Mi ± ∞ ¹   +0.76% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i!="",j=~"X.+"-10                                                         9.170Mi ± ∞ ¹                10.693Mi ± ∞ ¹  +16.61% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i!="",j=~"XXX|YYY"-10                                                     2.896Ki ± ∞ ¹                 2.966Ki ± ∞ ¹   +2.43% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i=~"X|Y|Z",j="foo"-10                                                     4.319Ki ± ∞ ¹                 4.406Ki ± ∞ ¹   +2.01% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i=~".+",j="foo"-10                                                        199.8Mi ± ∞ ¹                 201.4Mi ± ∞ ¹   +0.76% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i=~"1.+",j="foo"-10                                                       35.06Mi ± ∞ ¹                 33.87Mi ± ∞ ¹   -3.39% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i=~".+",i!="2",j="foo"-10                                                 201.1Mi ± ∞ ¹                 202.6Mi ± ∞ ¹   +0.76% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i=~".+",i!~"2.*",j="foo"-10                                               217.7Mi ± ∞ ¹                 218.2Mi ± ∞ ¹   +0.23% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="X",i=~"^.+$",i!~"^.*2.*$",j="foo"-10                                         2.379Ki ± ∞ ¹                 2.434Ki ± ∞ ¹   +2.32% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/i=~"0xxx|1xxx|2xxx"-10                                                          18.11Ki ± ∞ ¹                 18.16Ki ± ∞ ¹   +0.30% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/i=~"(0|1|2)xxx"-10                                                              18.03Ki ± ∞ ¹                 18.08Ki ± ∞ ¹   +0.28% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/i=~"[0-2]xxx"-10                                                                18.03Ki ± ∞ ¹                 18.08Ki ± ∞ ¹   +0.27% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/i!~[0-2]xxx-10                                                                  272.1Mi ± ∞ ¹                 272.1Mi ± ∞ ¹        ~ (p=0.057 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/i=~".*",_i!~[0-2]xxx-10                                                         273.6Mi ± ∞ ¹                 272.1Mi ± ∞ ¹   -0.56% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/i=~"<unique_prefix>.+"-10                                                     879.472Ki ± ∞ ¹                 7.113Ki ± ∞ ¹  -99.19% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i=~"<unique_prefix>.+"-10                                                 2.097Mi ± ∞ ¹                 1.245Mi ± ∞ ¹  -40.63% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i!~"<unique_prefix>.+"-10                                                 12.99Mi ± ∞ ¹                 12.14Mi ± ∞ ¹   -6.56% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/p!=""-10                                                                        56.05Mi ± ∞ ¹                 56.05Mi ± ∞ ¹   -0.00% (p=0.029 n=4)
geomean                                                                                                                          1.648Mi                       1.443Mi        -12.41%
¹ need >= 6 samples for confidence interval at level 0.95

                                                                                         │ before-cab54ae516ae69b9334b9c779c6225f4dd1e3d13.txt │ after-ebfe653ef1fd8518de24f9fd8ad44016fbc80643.txt │
                                                                                         │                      allocs/op                      │         allocs/op           vs base                │
BucketIndexReader_ExpandedPostings/stream_reader/n="X"-10                                                                          27.00 ± ∞ ¹                  28.00 ± ∞ ¹  +3.70% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="X",j="foo"-10                                                                  35.00 ± ∞ ¹                  36.00 ± ∞ ¹  +2.86% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="X",j!="foo"-10                                                                 35.00 ± ∞ ¹                  36.00 ± ∞ ¹  +2.86% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/j=~"XXX|YYY"-10                                                                   22.00 ± ∞ ¹                  23.00 ± ∞ ¹  +4.55% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/j=~"X.+"-10                                                                       22.00 ± ∞ ¹                  23.00 ± ∞ ¹  +4.55% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/i=~"X|Y|Z"-10                                                                     37.00 ± ∞ ¹                  38.00 ± ∞ ¹  +2.70% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1"-10                                                                          147.0 ± ∞ ¹                  148.0 ± ∞ ¹  +0.68% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",j="foo"-10                                                                  181.0 ± ∞ ¹                  182.0 ± ∞ ¹       ~ (p=0.114 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/j="foo",n="1"-10                                                                  181.0 ± ∞ ¹                  182.0 ± ∞ ¹  +0.55% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",j!="foo"-10                                                                 179.0 ± ∞ ¹                  180.0 ± ∞ ¹       ~ (p=0.114 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/i=~".*"-10                                                                        119.0 ± ∞ ¹                  119.0 ± ∞ ¹       ~ (p=1.000 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/i=~".+"-10                                                                       2.650M ± ∞ ¹                 2.650M ± ∞ ¹  +0.00% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/i=~"^.+$",j=~"X.+"-10                                                            100.0k ± ∞ ¹                 100.0k ± ∞ ¹  +0.00% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/i=~""-10                                                                         2.650M ± ∞ ¹                 2.650M ± ∞ ¹       ~ (p=0.743 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/i!=""-10                                                                         2.650M ± ∞ ¹                 2.650M ± ∞ ¹       ~ (p=0.886 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i=~".*",j="foo"-10                                                          190.0 ± ∞ ¹                  188.5 ± ∞ ¹       ~ (p=0.086 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="X",i=~"^.+$",j="foo"-10                                                        37.00 ± ∞ ¹                  36.00 ± ∞ ¹  -2.70% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i=~".*",i!="2",j="foo"-10                                                   222.0 ± ∞ ¹                  222.0 ± ∞ ¹       ~ (p=1.000 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i!=""-10                                                                   2.650M ± ∞ ¹                 2.650M ± ∞ ¹       ~ (p=0.514 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i!="",j="foo"-10                                                           2.650M ± ∞ ¹                 2.650M ± ∞ ¹       ~ (p=0.886 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i!="",j=~"X.+"-10                                                          100.1k ± ∞ ¹                 100.1k ± ∞ ¹       ~ (p=0.057 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i!="",j=~"XXX|YYY"-10                                                       53.00 ± ∞ ¹                  53.00 ± ∞ ¹       ~ (p=1.000 n=4) ²
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i=~"X|Y|Z",j="foo"-10                                                       73.00 ± ∞ ¹                  74.00 ± ∞ ¹  +1.37% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i=~".+",j="foo"-10                                                         2.650M ± ∞ ¹                 2.650M ± ∞ ¹       ~ (p=0.114 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i=~"1.+",j="foo"-10                                                        294.6k ± ∞ ¹                 294.6k ± ∞ ¹       ~ (p=0.571 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i=~".+",i!="2",j="foo"-10                                                  2.650M ± ∞ ¹                 2.650M ± ∞ ¹       ~ (p=0.914 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i=~".+",i!~"2.*",j="foo"-10                                                2.945M ± ∞ ¹                 2.945M ± ∞ ¹       ~ (p=0.343 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="X",i=~"^.+$",i!~"^.*2.*$",j="foo"-10                                           38.00 ± ∞ ¹                  36.00 ± ∞ ¹  -5.26% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/i=~"0xxx|1xxx|2xxx"-10                                                            184.0 ± ∞ ¹                  185.0 ± ∞ ¹  +0.54% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/i=~"(0|1|2)xxx"-10                                                                184.0 ± ∞ ¹                  185.0 ± ∞ ¹  +0.54% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/i=~"[0-2]xxx"-10                                                                  184.0 ± ∞ ¹                  185.0 ± ∞ ¹  +0.54% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/i!~[0-2]xxx-10                                                                    238.0 ± ∞ ¹                  239.0 ± ∞ ¹  +0.42% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/i=~".*",_i!~[0-2]xxx-10                                                           248.0 ± ∞ ¹                  248.5 ± ∞ ¹       ~ (p=0.429 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/i=~"<unique_prefix>.+"-10                                                         105.0 ± ∞ ¹                  106.0 ± ∞ ¹  +0.95% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i=~"<unique_prefix>.+"-10                                                   183.0 ± ∞ ¹                  183.0 ± ∞ ¹       ~ (p=1.000 n=4) ²
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i!~"<unique_prefix>.+"-10                                                   216.0 ± ∞ ¹                  217.0 ± ∞ ¹  +0.46% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/p!=""-10                                                                          112.0 ± ∞ ¹                  113.0 ± ∞ ¹  +0.89% (p=0.029 n=4)
geomean                                                                                                                           1.573k                       1.582k        +0.55%

then i tried using the prefix to estimate where the last match will be (387b07f). This showed better results overall but still some regressions (e.g. when the prefix is also the value, since we now binary search twice; maybe we should do linear search?).

BucketIndexReader_ExpandedPostings with end prefix estimation too
goos: darwin
goarch: arm64
pkg: github.com/grafana/mimir/pkg/storegateway
                                                                                         │ before-cab54ae516ae69b9334b9c779c6225f4dd1e3d13.txt │ after-387b07f83fd27c0f40ff8722e95c5eb42b689c2d.txt │
                                                                                         │                       sec/op                        │          sec/op            vs base                 │
BucketIndexReader_ExpandedPostings/stream_reader/n="X"-10                                                                         10.10µ ± ∞ ¹                14.33µ ± ∞ ¹  +41.89% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="X",j="foo"-10                                                                 20.61µ ± ∞ ¹                31.46µ ± ∞ ¹  +52.63% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="X",j!="foo"-10                                                                20.55µ ± ∞ ¹                20.70µ ± ∞ ¹   +0.76% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/j=~"XXX|YYY"-10                                                                  911.6n ± ∞ ¹                960.4n ± ∞ ¹   +5.36% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/j=~"X.+"-10                                                                      904.2n ± ∞ ¹                932.2n ± ∞ ¹        ~ (p=0.343 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/i=~"X|Y|Z"-10                                                                    41.22µ ± ∞ ¹                28.42µ ± ∞ ¹  -31.05% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1"-10                                                                         4.652m ± ∞ ¹                4.553m ± ∞ ¹        ~ (p=0.114 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",j="foo"-10                                                                 46.23m ± ∞ ¹                44.91m ± ∞ ¹   -2.85% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/j="foo",n="1"-10                                                                 46.10m ± ∞ ¹                44.58m ± ∞ ¹   -3.31% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",j!="foo"-10                                                                35.91m ± ∞ ¹                35.11m ± ∞ ¹   -2.22% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/i=~".*"-10                                                                       87.58m ± ∞ ¹                87.38m ± ∞ ¹        ~ (p=1.000 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/i=~".+"-10                                                                        1.315 ± ∞ ¹                 1.322 ± ∞ ¹        ~ (p=0.486 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/i=~"^.+$",j=~"X.+"-10                                                            8.415m ± ∞ ¹                8.987m ± ∞ ¹   +6.80% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/i=~""-10                                                                          1.414 ± ∞ ¹                 1.433 ± ∞ ¹        ~ (p=0.114 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/i!=""-10                                                                          1.315 ± ∞ ¹                 1.310 ± ∞ ¹        ~ (p=0.343 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i=~".*",j="foo"-10                                                         50.66m ± ∞ ¹                50.64m ± ∞ ¹        ~ (p=1.000 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="X",i=~"^.+$",j="foo"-10                                                       20.86µ ± ∞ ¹                20.81µ ± ∞ ¹        ~ (p=0.486 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i=~".*",i!="2",j="foo"-10                                                  59.00m ± ∞ ¹                59.32m ± ∞ ¹        ~ (p=0.200 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i!=""-10                                                                    1.188 ± ∞ ¹                 1.182 ± ∞ ¹        ~ (p=0.343 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i!="",j="foo"-10                                                            1.214 ± ∞ ¹                 1.217 ± ∞ ¹        ~ (p=0.686 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i!="",j=~"X.+"-10                                                          7.658m ± ∞ ¹                8.201m ± ∞ ¹   +7.09% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i!="",j=~"XXX|YYY"-10                                                      13.13µ ± ∞ ¹                13.16µ ± ∞ ¹        ~ (p=0.343 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i=~"X|Y|Z",j="foo"-10                                                      50.22µ ± ∞ ¹                50.09µ ± ∞ ¹        ~ (p=0.486 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i=~".+",j="foo"-10                                                          1.219 ± ∞ ¹                 1.232 ± ∞ ¹        ~ (p=0.486 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i=~"1.+",j="foo"-10                                                        148.7m ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i=~".+",i!="2",j="foo"-10                                                   1.229 ± ∞ ¹                 1.226 ± ∞ ¹        ~ (p=0.343 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i=~".+",i!~"2.*",j="foo"-10                                                 1.362 ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/n="X",i=~"^.+$",i!~"^.*2.*$",j="foo"-10                                          21.07µ ± ∞ ¹                21.08µ ± ∞ ¹        ~ (p=0.686 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/i=~"0xxx|1xxx|2xxx"-10                                                           128.7µ ± ∞ ¹                129.9µ ± ∞ ¹        ~ (p=0.200 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/i=~"(0|1|2)xxx"-10                                                               128.8µ ± ∞ ¹                129.5µ ± ∞ ¹        ~ (p=0.886 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/i=~"[0-2]xxx"-10                                                                 129.3µ ± ∞ ¹                128.3µ ± ∞ ¹   -0.81% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/i!~[0-2]xxx-10                                                                   96.23m ± ∞ ¹                95.48m ± ∞ ¹        ~ (p=0.343 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/i=~".*",_i!~[0-2]xxx-10                                                          102.0m ± ∞ ¹                101.3m ± ∞ ¹        ~ (p=0.343 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/i=~"<unique_prefix>.+"-10                                                       194.28µ ± ∞ ¹                48.25µ ± ∞ ¹  -75.16% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i=~"<unique_prefix>.+"-10                                                  1.497m ± ∞ ¹                1.283m ± ∞ ¹  -14.33% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i!~"<unique_prefix>.+"-10                                                  5.236m ± ∞ ¹                5.028m ± ∞ ¹        ~ (p=0.114 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/p!=""-10                                                                         20.15m ± ∞ ¹                19.55m ± ∞ ¹        ~ (p=0.114 n=4)
geomean                                                                                                                           4.248m                      3.155m         -3.06%               ²
¹ need >= 6 samples for confidence interval at level 0.95
² benchmark set differs from baseline; geomeans may not be comparable

                                                                                         │ before-cab54ae516ae69b9334b9c779c6225f4dd1e3d13.txt │ after-387b07f83fd27c0f40ff8722e95c5eb42b689c2d.txt │
                                                                                         │                        B/op                         │           B/op             vs base                 │
BucketIndexReader_ExpandedPostings/stream_reader/n="X"-10                                                                        1.332Ki ± ∞ ¹               1.386Ki ± ∞ ¹   +4.11% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="X",j="foo"-10                                                                2.004Ki ± ∞ ¹               2.074Ki ± ∞ ¹   +3.48% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="X",j!="foo"-10                                                               2.004Ki ± ∞ ¹               2.074Ki ± ∞ ¹   +3.51% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/j=~"XXX|YYY"-10                                                                  1008.0 ± ∞ ¹                1064.0 ± ∞ ¹   +5.56% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/j=~"X.+"-10                                                                       952.0 ± ∞ ¹                1008.0 ± ∞ ¹   +5.88% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/i=~"X|Y|Z"-10                                                                   2.050Ki ± ∞ ¹               2.104Ki ± ∞ ¹   +2.67% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1"-10                                                                        12.13Mi ± ∞ ¹               12.13Mi ± ∞ ¹   +0.00% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",j="foo"-10                                                                18.88Mi ± ∞ ¹               18.88Mi ± ∞ ¹        ~ (p=0.686 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/j="foo",n="1"-10                                                                18.88Mi ± ∞ ¹               18.88Mi ± ∞ ¹        ~ (p=0.200 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",j!="foo"-10                                                               18.88Mi ± ∞ ¹               18.88Mi ± ∞ ¹        ~ (p=0.343 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/i=~".*"-10                                                                      273.6Mi ± ∞ ¹               275.1Mi ± ∞ ¹   +0.56% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/i=~".+"-10                                                                      422.4Mi ± ∞ ¹               423.9Mi ± ∞ ¹   +0.36% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/i=~"^.+$",j=~"X.+"-10                                                           9.167Mi ± ∞ ¹              10.691Mi ± ∞ ¹  +16.62% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/i=~""-10                                                                        218.6Mi ± ∞ ¹               220.1Mi ± ∞ ¹   +0.70% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/i!=""-10                                                                        422.4Mi ± ∞ ¹               423.9Mi ± ∞ ¹   +0.36% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i=~".*",j="foo"-10                                                        20.42Mi ± ∞ ¹               21.94Mi ± ∞ ¹   +7.47% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="X",i=~"^.+$",j="foo"-10                                                      2.191Ki ± ∞ ¹               2.246Ki ± ∞ ¹   +2.50% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i=~".*",i!="2",j="foo"-10                                                 21.65Mi ± ∞ ¹               23.18Mi ± ∞ ¹   +7.04% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i!=""-10                                                                  193.1Mi ± ∞ ¹               194.6Mi ± ∞ ¹   +0.79% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i!="",j="foo"-10                                                          199.9Mi ± ∞ ¹               201.4Mi ± ∞ ¹   +0.76% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i!="",j=~"X.+"-10                                                         9.170Mi ± ∞ ¹              10.694Mi ± ∞ ¹  +16.63% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i!="",j=~"XXX|YYY"-10                                                     2.896Ki ± ∞ ¹               2.966Ki ± ∞ ¹   +2.43% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i=~"X|Y|Z",j="foo"-10                                                     4.319Ki ± ∞ ¹               4.405Ki ± ∞ ¹   +1.99% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i=~".+",j="foo"-10                                                        199.8Mi ± ∞ ¹               201.4Mi ± ∞ ¹   +0.76% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i=~"1.+",j="foo"-10                                                       35.06Mi ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i=~".+",i!="2",j="foo"-10                                                 201.1Mi ± ∞ ¹               202.6Mi ± ∞ ¹   +0.76% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i=~".+",i!~"2.*",j="foo"-10                                               217.7Mi ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/n="X",i=~"^.+$",i!~"^.*2.*$",j="foo"-10                                         2.379Ki ± ∞ ¹               2.434Ki ± ∞ ¹   +2.32% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/i=~"0xxx|1xxx|2xxx"-10                                                          18.11Ki ± ∞ ¹               18.17Ki ± ∞ ¹   +0.31% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/i=~"(0|1|2)xxx"-10                                                              18.03Ki ± ∞ ¹               18.09Ki ± ∞ ¹   +0.31% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/i=~"[0-2]xxx"-10                                                                18.03Ki ± ∞ ¹               18.09Ki ± ∞ ¹   +0.31% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/i!~[0-2]xxx-10                                                                  272.1Mi ± ∞ ¹               272.1Mi ± ∞ ¹        ~ (p=1.000 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/i=~".*",_i!~[0-2]xxx-10                                                         273.6Mi ± ∞ ¹               275.1Mi ± ∞ ¹   +0.56% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/i=~"<unique_prefix>.+"-10                                                     879.472Ki ± ∞ ¹               8.085Ki ± ∞ ¹  -99.08% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i=~"<unique_prefix>.+"-10                                                 2.097Mi ± ∞ ¹               1.246Mi ± ∞ ¹  -40.58% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i!~"<unique_prefix>.+"-10                                                 12.99Mi ± ∞ ¹               12.14Mi ± ∞ ¹   -6.55% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/p!=""-10                                                                        56.05Mi ± ∞ ¹               56.05Mi ± ∞ ¹        ~ (p=0.200 n=4)
geomean                                                                                                                          1.648Mi                     1.157Mi        -11.88%               ²
¹ need >= 6 samples for confidence interval at level 0.95
² benchmark set differs from baseline; geomeans may not be comparable

                                                                                         │ before-cab54ae516ae69b9334b9c779c6225f4dd1e3d13.txt │ after-387b07f83fd27c0f40ff8722e95c5eb42b689c2d.txt │
                                                                                         │                      allocs/op                      │         allocs/op           vs base                │
BucketIndexReader_ExpandedPostings/stream_reader/n="X"-10                                                                          27.00 ± ∞ ¹                  28.00 ± ∞ ¹  +3.70% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="X",j="foo"-10                                                                  35.00 ± ∞ ¹                  36.00 ± ∞ ¹  +2.86% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="X",j!="foo"-10                                                                 35.00 ± ∞ ¹                  36.00 ± ∞ ¹  +2.86% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/j=~"XXX|YYY"-10                                                                   22.00 ± ∞ ¹                  23.00 ± ∞ ¹  +4.55% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/j=~"X.+"-10                                                                       22.00 ± ∞ ¹                  23.00 ± ∞ ¹  +4.55% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/i=~"X|Y|Z"-10                                                                     37.00 ± ∞ ¹                  38.00 ± ∞ ¹  +2.70% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1"-10                                                                          147.0 ± ∞ ¹                  148.0 ± ∞ ¹  +0.68% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",j="foo"-10                                                                  181.0 ± ∞ ¹                  182.0 ± ∞ ¹  +0.55% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/j="foo",n="1"-10                                                                  181.0 ± ∞ ¹                  182.0 ± ∞ ¹  +0.55% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",j!="foo"-10                                                                 179.0 ± ∞ ¹                  180.0 ± ∞ ¹  +0.56% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/i=~".*"-10                                                                        119.0 ± ∞ ¹                  120.0 ± ∞ ¹       ~ (p=0.114 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/i=~".+"-10                                                                       2.650M ± ∞ ¹                 2.650M ± ∞ ¹  +0.00% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/i=~"^.+$",j=~"X.+"-10                                                            100.0k ± ∞ ¹                 100.0k ± ∞ ¹  +0.00% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/i=~""-10                                                                         2.650M ± ∞ ¹                 2.650M ± ∞ ¹       ~ (p=1.000 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/i!=""-10                                                                         2.650M ± ∞ ¹                 2.650M ± ∞ ¹       ~ (p=1.000 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i=~".*",j="foo"-10                                                          190.0 ± ∞ ¹                  192.0 ± ∞ ¹  +1.05% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="X",i=~"^.+$",j="foo"-10                                                        37.00 ± ∞ ¹                  36.00 ± ∞ ¹  -2.70% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i=~".*",i!="2",j="foo"-10                                                   222.0 ± ∞ ¹                  227.0 ± ∞ ¹  +2.25% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i!=""-10                                                                   2.650M ± ∞ ¹                 2.650M ± ∞ ¹       ~ (p=0.914 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i!="",j="foo"-10                                                           2.650M ± ∞ ¹                 2.650M ± ∞ ¹       ~ (p=1.000 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i!="",j=~"X.+"-10                                                          100.1k ± ∞ ¹                 100.1k ± ∞ ¹  +0.00% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i!="",j=~"XXX|YYY"-10                                                       53.00 ± ∞ ¹                  53.00 ± ∞ ¹       ~ (p=1.000 n=4) ²
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i=~"X|Y|Z",j="foo"-10                                                       73.00 ± ∞ ¹                  74.00 ± ∞ ¹  +1.37% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i=~".+",j="foo"-10                                                         2.650M ± ∞ ¹                 2.650M ± ∞ ¹       ~ (p=0.457 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i=~"1.+",j="foo"-10                                                        294.6k ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i=~".+",i!="2",j="foo"-10                                                  2.650M ± ∞ ¹                 2.650M ± ∞ ¹       ~ (p=0.857 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i=~".+",i!~"2.*",j="foo"-10                                                2.945M ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/n="X",i=~"^.+$",i!~"^.*2.*$",j="foo"-10                                           38.00 ± ∞ ¹                  36.00 ± ∞ ¹  -5.26% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/i=~"0xxx|1xxx|2xxx"-10                                                            184.0 ± ∞ ¹                  185.0 ± ∞ ¹  +0.54% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/i=~"(0|1|2)xxx"-10                                                                184.0 ± ∞ ¹                  185.0 ± ∞ ¹  +0.54% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/i=~"[0-2]xxx"-10                                                                  184.0 ± ∞ ¹                  185.0 ± ∞ ¹  +0.54% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/i!~[0-2]xxx-10                                                                    238.0 ± ∞ ¹                  239.0 ± ∞ ¹  +0.42% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/i=~".*",_i!~[0-2]xxx-10                                                           248.0 ± ∞ ¹                  249.0 ± ∞ ¹  +0.40% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/i=~"<unique_prefix>.+"-10                                                         105.0 ± ∞ ¹                  106.0 ± ∞ ¹  +0.95% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i=~"<unique_prefix>.+"-10                                                   183.0 ± ∞ ¹                  183.0 ± ∞ ¹       ~ (p=1.000 n=4) ²
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i!~"<unique_prefix>.+"-10                                                   216.0 ± ∞ ¹                  217.0 ± ∞ ¹  +0.46% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/p!=""-10                                                                          112.0 ± ∞ ¹                  113.0 ± ∞ ¹  +0.89% (p=0.029 n=4)
geomean                                                                                                                           1.573k                       1.100k        +0.72%               ³
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal
³ benchmark set differs from baseline; geomeans may not be comparable

NB both benchmark comparisons compare main with this PR + estimation improvements

I think we should continue this discussion and work in another PR. But I am not sure if we want to merge that before merging this PR? Marco, what do you think?

@pracucci
Copy link
Collaborator

pracucci commented Apr 6, 2023

I think we should continue this discussion and work in another PR. But I am not sure if we want to merge that before merging this PR? Marco, what do you think?

I agree to merge this PR and continue the discussion separately. The regression doesn't look terrible to me.


I just have one last easy idea 👇

(it was only string, now it's struct{ string, struct { int64, int64 }})

Does it help a bit if we change PostingListOffset like this?

type PostingListOffset struct {
	index.Range
	LabelValue string
}

Signed-off-by: Dimitar Dimitrov <dimitar.dimitrov@grafana.com>
@dimitarvdimitrov dimitarvdimitrov marked this pull request as ready for review April 6, 2023 14:02
@dimitarvdimitrov dimitarvdimitrov requested a review from a team as a code owner April 6, 2023 14:02
@dimitarvdimitrov
Copy link
Contributor Author

I just have one last easy idea 👇

(it was only string, now it's struct{ string, struct { int64, int64 }})

Does it help a bit if we change PostingListOffset like this?

It didn't seem to work (fefb871), I think embedding structs is mostly syntactic sugar.

BucketIndexReader_ExpandedPostings with inlined field (compared to this PR)
goos: darwin
goarch: arm64
pkg: github.com/grafana/mimir/pkg/storegateway
                                                                                 │ after-131acfef06d6a5f12dda87c218afc5bd9d42b5cd.txt │ after-fefb87137ea769fb482e19677fd34031230e901e.txt │
                                                                                 │                       sec/op                       │           sec/op            vs base                │
BucketIndexReader_ExpandedPostings/binary_reader/i=~"<unique_prefix>.+"-10                                               309.2µ ± ∞ ¹                 298.9µ ± ∞ ¹       ~ (p=0.343 n=4)
BucketIndexReader_ExpandedPostings/binary_reader/n="1",i=~"<unique_prefix>.+"-10                                         1.582m ± ∞ ¹                 1.540m ± ∞ ¹  -2.64% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/binary_reader/n="1",i!~"<unique_prefix>.+"-10                                         5.577m ± ∞ ¹                 5.378m ± ∞ ¹       ~ (p=0.343 n=4)
BucketIndexReader_ExpandedPostings/binary_reader/p!=""-10                                                                19.48m ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/i=~"<unique_prefix>.+"-10                                               324.5µ ± ∞ ¹                 325.0µ ± ∞ ¹       ~ (p=1.000 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i=~"<unique_prefix>.+"-10                                         1.540m ± ∞ ¹                 1.558m ± ∞ ¹       ~ (p=0.343 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i!~"<unique_prefix>.+"-10                                         5.384m ± ∞ ¹                 5.683m ± ∞ ¹  +5.55% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/p!=""-10                                                                20.53m ± ∞ ¹
geomean                                                                                                                  2.713m                       1.387m        -0.49%               ²
¹ need >= 6 samples for confidence interval at level 0.95
² benchmark set differs from baseline; geomeans may not be comparable

                                                                                 │ after-131acfef06d6a5f12dda87c218afc5bd9d42b5cd.txt │ after-fefb87137ea769fb482e19677fd34031230e901e.txt │
                                                                                 │                        B/op                        │            B/op             vs base                │
BucketIndexReader_ExpandedPostings/binary_reader/i=~"<unique_prefix>.+"-10                                              1.711Mi ± ∞ ¹                1.711Mi ± ∞ ¹       ~ (p=1.000 n=4)
BucketIndexReader_ExpandedPostings/binary_reader/n="1",i=~"<unique_prefix>.+"-10                                        2.949Mi ± ∞ ¹                2.949Mi ± ∞ ¹       ~ (p=0.057 n=4)
BucketIndexReader_ExpandedPostings/binary_reader/n="1",i!~"<unique_prefix>.+"-10                                        13.84Mi ± ∞ ¹                13.84Mi ± ∞ ¹       ~ (p=0.686 n=4)
BucketIndexReader_ExpandedPostings/binary_reader/p!=""-10                                                               56.05Mi ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/i=~"<unique_prefix>.+"-10                                              1.711Mi ± ∞ ¹                1.711Mi ± ∞ ¹       ~ (p=0.914 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i=~"<unique_prefix>.+"-10                                        2.949Mi ± ∞ ¹                2.949Mi ± ∞ ¹       ~ (p=0.114 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i!~"<unique_prefix>.+"-10                                        13.84Mi ± ∞ ¹                13.84Mi ± ∞ ¹       ~ (p=0.343 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/p!=""-10                                                               56.05Mi ± ∞ ¹
geomean                                                                                                                 7.910Mi                      4.118Mi        -0.00%               ²
¹ need >= 6 samples for confidence interval at level 0.95
² benchmark set differs from baseline; geomeans may not be comparable

                                                                                 │ after-131acfef06d6a5f12dda87c218afc5bd9d42b5cd.txt │ after-fefb87137ea769fb482e19677fd34031230e901e.txt │
                                                                                 │                     allocs/op                      │         allocs/op           vs base                │
BucketIndexReader_ExpandedPostings/binary_reader/i=~"<unique_prefix>.+"-10                                                107.0 ± ∞ ¹                  107.0 ± ∞ ¹       ~ (p=1.000 n=4)
BucketIndexReader_ExpandedPostings/binary_reader/n="1",i=~"<unique_prefix>.+"-10                                          184.0 ± ∞ ¹                  184.0 ± ∞ ¹       ~ (p=1.000 n=4)
BucketIndexReader_ExpandedPostings/binary_reader/n="1",i!~"<unique_prefix>.+"-10                                          218.0 ± ∞ ¹                  218.0 ± ∞ ¹       ~ (p=1.000 n=4) ²
BucketIndexReader_ExpandedPostings/binary_reader/p!=""-10                                                                 113.0 ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/i=~"<unique_prefix>.+"-10                                                106.0 ± ∞ ¹                  106.0 ± ∞ ¹       ~ (p=1.000 n=4) ²
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i=~"<unique_prefix>.+"-10                                          184.0 ± ∞ ¹                  184.0 ± ∞ ¹       ~ (p=1.000 n=4) ²
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i!~"<unique_prefix>.+"-10                                          218.0 ± ∞ ¹                  218.0 ± ∞ ¹       ~ (p=1.000 n=4) ²
BucketIndexReader_ExpandedPostings/stream_reader/p!=""-10                                                                 113.0 ± ∞ ¹
geomean                                                                                                                   148.2                        162.3        +0.00%               ³
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal
³ benchmark set differs from baseline; geomeans may not be comparable
BucketIndexReader_ExpandedPostings with inlined field (compared to main)
goos: darwin
goarch: arm64
pkg: github.com/grafana/mimir/pkg/storegateway
                                                                                         │ before-cab54ae516ae69b9334b9c779c6225f4dd1e3d13.txt │ after-fefb87137ea769fb482e19677fd34031230e901e.txt │
                                                                                         │                       sec/op                        │          sec/op            vs base                 │
BucketIndexReader_ExpandedPostings/stream_reader/n="X"-10                                                                         10.10µ ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/n="X",j="foo"-10                                                                 20.61µ ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/n="X",j!="foo"-10                                                                20.55µ ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/j=~"XXX|YYY"-10                                                                  911.6n ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/j=~"X.+"-10                                                                      904.2n ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/i=~"X|Y|Z"-10                                                                    41.22µ ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/n="1"-10                                                                         4.652m ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/n="1",j="foo"-10                                                                 46.23m ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/j="foo",n="1"-10                                                                 46.10m ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/n="1",j!="foo"-10                                                                35.91m ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/i=~".*"-10                                                                       87.58m ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/i=~".+"-10                                                                        1.315 ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/i=~"^.+$",j=~"X.+"-10                                                            8.415m ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/i=~""-10                                                                          1.414 ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/i!=""-10                                                                          1.315 ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i=~".*",j="foo"-10                                                         50.66m ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/n="X",i=~"^.+$",j="foo"-10                                                       20.86µ ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i=~".*",i!="2",j="foo"-10                                                  59.00m ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i!=""-10                                                                    1.188 ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i!="",j="foo"-10                                                            1.214 ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i!="",j=~"X.+"-10                                                          7.658m ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i!="",j=~"XXX|YYY"-10                                                      13.13µ ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i=~"X|Y|Z",j="foo"-10                                                      50.22µ ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i=~".+",j="foo"-10                                                          1.219 ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i=~"1.+",j="foo"-10                                                        148.7m ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i=~".+",i!="2",j="foo"-10                                                   1.229 ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i=~".+",i!~"2.*",j="foo"-10                                                 1.362 ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/n="X",i=~"^.+$",i!~"^.*2.*$",j="foo"-10                                          21.07µ ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/i=~"0xxx|1xxx|2xxx"-10                                                           128.7µ ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/i=~"(0|1|2)xxx"-10                                                               128.8µ ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/i=~"[0-2]xxx"-10                                                                 129.3µ ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/i!~[0-2]xxx-10                                                                   96.23m ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/i=~".*",_i!~[0-2]xxx-10                                                          102.0m ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/i=~"<unique_prefix>.+"-10                                                        194.3µ ± ∞ ¹                325.0µ ± ∞ ¹  +67.27% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i=~"<unique_prefix>.+"-10                                                  1.497m ± ∞ ¹                1.558m ± ∞ ¹        ~ (p=0.200 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i!~"<unique_prefix>.+"-10                                                  5.236m ± ∞ ¹                5.683m ± ∞ ¹        ~ (p=0.200 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/p!=""-10                                                                         20.15m ± ∞ ¹
BucketIndexReader_ExpandedPostings/binary_reader/i=~"<unique_prefix>.+"-10                                                                                    298.9µ ± ∞ ¹
BucketIndexReader_ExpandedPostings/binary_reader/n="1",i=~"<unique_prefix>.+"-10                                                                              1.540m ± ∞ ¹
BucketIndexReader_ExpandedPostings/binary_reader/n="1",i!~"<unique_prefix>.+"-10                                                                              5.378m ± ∞ ¹
geomean                                                                                                                           4.248m                      1.387m        +23.61%               ²
¹ need >= 6 samples for confidence interval at level 0.95
² benchmark set differs from baseline; geomeans may not be comparable

                                                                                         │ before-cab54ae516ae69b9334b9c779c6225f4dd1e3d13.txt │ after-fefb87137ea769fb482e19677fd34031230e901e.txt │
                                                                                         │                        B/op                         │           B/op             vs base                 │
BucketIndexReader_ExpandedPostings/stream_reader/n="X"-10                                                                        1.332Ki ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/n="X",j="foo"-10                                                                2.004Ki ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/n="X",j!="foo"-10                                                               2.004Ki ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/j=~"XXX|YYY"-10                                                                  1008.0 ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/j=~"X.+"-10                                                                       952.0 ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/i=~"X|Y|Z"-10                                                                   2.050Ki ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/n="1"-10                                                                        12.13Mi ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/n="1",j="foo"-10                                                                18.88Mi ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/j="foo",n="1"-10                                                                18.88Mi ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/n="1",j!="foo"-10                                                               18.88Mi ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/i=~".*"-10                                                                      273.6Mi ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/i=~".+"-10                                                                      422.4Mi ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/i=~"^.+$",j=~"X.+"-10                                                           9.167Mi ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/i=~""-10                                                                        218.6Mi ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/i!=""-10                                                                        422.4Mi ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i=~".*",j="foo"-10                                                        20.42Mi ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/n="X",i=~"^.+$",j="foo"-10                                                      2.191Ki ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i=~".*",i!="2",j="foo"-10                                                 21.65Mi ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i!=""-10                                                                  193.1Mi ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i!="",j="foo"-10                                                          199.9Mi ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i!="",j=~"X.+"-10                                                         9.170Mi ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i!="",j=~"XXX|YYY"-10                                                     2.896Ki ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i=~"X|Y|Z",j="foo"-10                                                     4.319Ki ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i=~".+",j="foo"-10                                                        199.8Mi ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i=~"1.+",j="foo"-10                                                       35.06Mi ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i=~".+",i!="2",j="foo"-10                                                 201.1Mi ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i=~".+",i!~"2.*",j="foo"-10                                               217.7Mi ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/n="X",i=~"^.+$",i!~"^.*2.*$",j="foo"-10                                         2.379Ki ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/i=~"0xxx|1xxx|2xxx"-10                                                          18.11Ki ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/i=~"(0|1|2)xxx"-10                                                              18.03Ki ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/i=~"[0-2]xxx"-10                                                                18.03Ki ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/i!~[0-2]xxx-10                                                                  272.1Mi ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/i=~".*",_i!~[0-2]xxx-10                                                         273.6Mi ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/i=~"<unique_prefix>.+"-10                                                       879.5Ki ± ∞ ¹              1751.8Ki ± ∞ ¹  +99.18% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i=~"<unique_prefix>.+"-10                                                 2.097Mi ± ∞ ¹               2.949Mi ± ∞ ¹  +40.62% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i!~"<unique_prefix>.+"-10                                                 12.99Mi ± ∞ ¹               13.84Mi ± ∞ ¹   +6.56% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/p!=""-10                                                                        56.05Mi ± ∞ ¹
BucketIndexReader_ExpandedPostings/binary_reader/i=~"<unique_prefix>.+"-10                                                                                   1.711Mi ± ∞ ¹
BucketIndexReader_ExpandedPostings/binary_reader/n="1",i=~"<unique_prefix>.+"-10                                                                             2.949Mi ± ∞ ¹
BucketIndexReader_ExpandedPostings/binary_reader/n="1",i!~"<unique_prefix>.+"-10                                                                             13.84Mi ± ∞ ¹
geomean                                                                                                                          1.648Mi                     4.118Mi        +43.98%               ²
¹ need >= 6 samples for confidence interval at level 0.95
² benchmark set differs from baseline; geomeans may not be comparable

                                                                                         │ before-cab54ae516ae69b9334b9c779c6225f4dd1e3d13.txt │ after-fefb87137ea769fb482e19677fd34031230e901e.txt │
                                                                                         │                      allocs/op                      │         allocs/op           vs base                │
BucketIndexReader_ExpandedPostings/stream_reader/n="X"-10                                                                          27.00 ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/n="X",j="foo"-10                                                                  35.00 ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/n="X",j!="foo"-10                                                                 35.00 ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/j=~"XXX|YYY"-10                                                                   22.00 ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/j=~"X.+"-10                                                                       22.00 ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/i=~"X|Y|Z"-10                                                                     37.00 ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/n="1"-10                                                                          147.0 ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/n="1",j="foo"-10                                                                  181.0 ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/j="foo",n="1"-10                                                                  181.0 ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/n="1",j!="foo"-10                                                                 179.0 ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/i=~".*"-10                                                                        119.0 ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/i=~".+"-10                                                                       2.650M ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/i=~"^.+$",j=~"X.+"-10                                                            100.0k ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/i=~""-10                                                                         2.650M ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/i!=""-10                                                                         2.650M ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i=~".*",j="foo"-10                                                          190.0 ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/n="X",i=~"^.+$",j="foo"-10                                                        37.00 ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i=~".*",i!="2",j="foo"-10                                                   222.0 ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i!=""-10                                                                   2.650M ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i!="",j="foo"-10                                                           2.650M ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i!="",j=~"X.+"-10                                                          100.1k ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i!="",j=~"XXX|YYY"-10                                                       53.00 ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i=~"X|Y|Z",j="foo"-10                                                       73.00 ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i=~".+",j="foo"-10                                                         2.650M ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i=~"1.+",j="foo"-10                                                        294.6k ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i=~".+",i!="2",j="foo"-10                                                  2.650M ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i=~".+",i!~"2.*",j="foo"-10                                                2.945M ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/n="X",i=~"^.+$",i!~"^.*2.*$",j="foo"-10                                           38.00 ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/i=~"0xxx|1xxx|2xxx"-10                                                            184.0 ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/i=~"(0|1|2)xxx"-10                                                                184.0 ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/i=~"[0-2]xxx"-10                                                                  184.0 ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/i!~[0-2]xxx-10                                                                    238.0 ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/i=~".*",_i!~[0-2]xxx-10                                                           248.0 ± ∞ ¹
BucketIndexReader_ExpandedPostings/stream_reader/i=~"<unique_prefix>.+"-10                                                         105.0 ± ∞ ¹                  106.0 ± ∞ ¹  +0.95% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i=~"<unique_prefix>.+"-10                                                   183.0 ± ∞ ¹                  184.0 ± ∞ ¹  +0.55% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/n="1",i!~"<unique_prefix>.+"-10                                                   216.0 ± ∞ ¹                  218.0 ± ∞ ¹  +0.93% (p=0.029 n=4)
BucketIndexReader_ExpandedPostings/stream_reader/p!=""-10                                                                          112.0 ± ∞ ¹
BucketIndexReader_ExpandedPostings/binary_reader/i=~"<unique_prefix>.+"-10                                                                                      107.0 ± ∞ ¹
BucketIndexReader_ExpandedPostings/binary_reader/n="1",i=~"<unique_prefix>.+"-10                                                                                184.0 ± ∞ ¹
BucketIndexReader_ExpandedPostings/binary_reader/n="1",i!~"<unique_prefix>.+"-10                                                                                218.0 ± ∞ ¹
geomean                                                                                                                           1.573k                        162.3        +0.81%               ²
¹ need >= 6 samples for confidence interval at level 0.95
² benchmark set differs from baseline; geomeans may not be comparable

@dimitarvdimitrov
Copy link
Contributor Author

I managed to write the test I intended. Is there anything else to address here or can we merge this?

Copy link
Contributor

@colega colega left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, I dislike the renaming of unappliedMatchers to deferredMatchers, but not blocking on that one.

@pracucci
Copy link
Collaborator

I managed to write the test I intended. Is there anything else to address here or can we merge this?

LGTM! Happy to merge this.

@dimitarvdimitrov
Copy link
Contributor Author

I managed to write the test I intended. Is there anything else to address here or can we merge this?

LGTM! Happy to merge this.

Do you have an opinion about the pendingMatchers vs deferredMatchers that oleg brought up?

Signed-off-by: Dimitar Dimitrov <dimitar.dimitrov@grafana.com>
@dimitarvdimitrov
Copy link
Contributor Author

ok, I changed "deferred" with "pending" in the package. I'll merge later today if there are no objections

@dimitarvdimitrov dimitarvdimitrov merged commit d897b18 into main Apr 11, 2023
@dimitarvdimitrov dimitarvdimitrov deleted the dimitar/expanded-postings-unapplied-matchers branch April 11, 2023 14:49
@pracucci
Copy link
Collaborator

I managed to write the test I intended. Is there anything else to address here or can we merge this?

LGTM! Happy to merge this.

Do you have an opinion about the pendingMatchers vs deferredMatchers that oleg brought up?

I do, but I think getting this stuff done is more important than keep talking about naming :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants