…ain) (#11756)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

#11814)

Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>

Signed-off-by: Danny Kopping <danny.kopping@grafana.com>

…11975)

In order to compare the keyspace from the bloom gateway's ring with the fingerprints, they need to use the same keyspace.

The ring, however, uses tokens within a `uint32` keyspace, whereas the fingerprints use a `uint64` keyspace.
Therefore the ranges from the ring tokens need to be mapped into the `uint64` keyspace. This is done bit-shifting the Min value 32 bits to the left and the Max value 32 bits to the left plus adding `(1<<32)-1` to "fill" the remaining values up to the next Min value.

The structs used to combined ring instance information and token/fingerprint ranges are yet to be simplified. However, this is not goal of this PR.

Signed-off-by: Christian Haudum <christian.haudum@gmail.com>
Co-authored-by: Owen Diehl <ow.diehl@gmail.com>

… single binary (#12005)

Signed-off-by: Danny Kopping <danny.kopping@grafana.com>

…sage) and use CollapsingLowestDense store for ddsketch (#11905)

Signed-off-by: Callum Styan <callumstyan@gmail.com>

Co-authored-by: Periklis Tsirakidis <periklis@redhat.com>

Signed-off-by: Christian Haudum <christian.haudum@gmail.com>

Co-authored-by: Periklis Tsirakidis <periklis@redhat.com>

…ases + additional metrics (#12015)

Co-authored-by: Trevor Whitney <trevorjwhitney@gmail.com>

…in bloom gateway (#11987)

This PR limits the amount of data being processed for a single multiplexed iteration to the union of the fingerprint bounds of its requests, instead of looking at blocks from the complete fingerprint range.

Signed-off-by: Christian Haudum <christian.haudum@gmail.com>

…12018)

Use the recently introduced `tokenRangesForInstance()` function to determine the the ranges for each replication set.

For this purpose, the function moves into the `bloomutils` packages.

This PR eliminates the structs `addrsWithBounds` and `instanceWithFingerprints` (which had somewhat similar usage) and unifies them into `rsWithRanges` that holds both the fingerprint ranges and the assigned GroupedChunkRefs.

Signed-off-by: Christian Haudum <christian.haudum@gmail.com>

This PR contains two changes:

1) Extracting the block without an intermediate temp file.
   The commit is cherry-picked from #12021, but that PR got merged into the wrong branch 🙈 

2) Strip `.tar.gz` suffix from block file when extracting it into a directory.

Signed-off-by: Christian Haudum <christian.haudum@gmail.com>

…ply (#11567)

Signed-off-by: Sheikh-Abubaker <sheikhabubaker761@gmail.com>
Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>
Co-authored-by: Karsten Jeschkies <karsten.jeschkies@grafana.com>

Fail startup if directory does not exist or there are not enough permissions. This prevents the bloomstore to fail later in the process once it tries to download and extract blocks.

Signed-off-by: Christian Haudum <christian.haudum@gmail.com>

* Wire up bloom querier metrics to fix nil pointer panic
* Use `bloomutils.TokenRangesForInstance` instead of `subRing.GetTokenRangesForInstance` to avoid `zone not set` error

Signed-off-by: Christian Haudum <christian.haudum@gmail.com>

…PageDecoder (#12050)

The bloomstore service is instantiated once at startup and used as dependency for both the bloom compactor and the bloom gateway.

This will prevent instantiating the store in different ways.


Signed-off-by: Christian Haudum <christian.haudum@gmail.com>

Signed-off-by: Christian Haudum <christian.haudum@gmail.com>

Co-authored-by: Christian Haudum <christian.haudum@gmail.com>

#12058)

Signed-off-by: Danny Kopping <danny.kopping@grafana.com>

…on (#12072)

Signed-off-by: Danny Kopping <danny.kopping@grafana.com>

…s not exist (#12080)

The check `!dirEntry.IsDir()` panicked in case `dirEntry` is `nil`, which is the case when the root directory does not exist.

The PR also adds more logging.

Signed-off-by: Christian Haudum <christian.haudum@gmail.com>

#12079)

Signed-off-by: Christian Haudum <christian.haudum@gmail.com>

Co-authored-by: loki-gh-app[bot] <160051081+loki-gh-app[bot]@users.noreply.github.com>

Co-authored-by: Jack Baldry <jack.baldry@grafana.com>

…ore (#12048)

* Add metric for size of metas/blocks fetched

Getting the distribution of the uncompressed size of metas and blocks,
grouped by their source (cache/filesystem/storage).

* Move blocks/metas fetched metrics into bloomstore

* Add minimal logging to bloom gateway processor

* Observe queue duration of tasks

The histogram metric gives us the insight about how long tasks spend
their time enqueued waiting to be processed.

* Remove queueMetrics field from bloom gateway struct

The metrics are only used by the queue instance, which is part of the
bloom gateway struct as well.

* Move various bloom gateway metrics into separate file

This commit introduces a new metrics struct that contains both the
metrics specific the server/handler and the metrics specific to the
worker.

This has the advantage that metrics can be instantiated and accessed
from one single place.

---

Signed-off-by: Christian Haudum <christian.haudum@gmail.com>

The warnings suggest something going wrong where it's not the case.

Signed-off-by: Christian Haudum <christian.haudum@gmail.com>