Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update from upstream repository #267

Merged
merged 45 commits into from Mar 2, 2024
Merged

Update from upstream repository #267

merged 45 commits into from Mar 2, 2024

Conversation

xperimental
Copy link

@xperimental xperimental commented Mar 1, 2024

renovate bot and others added 30 commits February 19, 2024 16:22
…ain) (#11756)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
#11814)

Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>
Signed-off-by: Danny Kopping <danny.kopping@grafana.com>
…11975)

In order to compare the keyspace from the bloom gateway's ring with the fingerprints, they need to use the same keyspace.

The ring, however, uses tokens within a `uint32` keyspace, whereas the fingerprints use a `uint64` keyspace.
Therefore the ranges from the ring tokens need to be mapped into the `uint64` keyspace. This is done bit-shifting the Min value 32 bits to the left and the Max value 32 bits to the left plus adding `(1<<32)-1` to "fill" the remaining values up to the next Min value.

The structs used to combined ring instance information and token/fingerprint ranges are yet to be simplified. However, this is not goal of this PR.

Signed-off-by: Christian Haudum <christian.haudum@gmail.com>
Co-authored-by: Owen Diehl <ow.diehl@gmail.com>
… single binary (#12005)

Signed-off-by: Danny Kopping <danny.kopping@grafana.com>
…sage) and use CollapsingLowestDense store for ddsketch (#11905)

Signed-off-by: Callum Styan <callumstyan@gmail.com>
Co-authored-by: Periklis Tsirakidis <periklis@redhat.com>
Signed-off-by: Christian Haudum <christian.haudum@gmail.com>
Co-authored-by: Periklis Tsirakidis <periklis@redhat.com>
Co-authored-by: Trevor Whitney <trevorjwhitney@gmail.com>
…in bloom gateway (#11987)

This PR limits the amount of data being processed for a single multiplexed iteration to the union of the fingerprint bounds of its requests, instead of looking at blocks from the complete fingerprint range.

Signed-off-by: Christian Haudum <christian.haudum@gmail.com>
…12018)

Use the recently introduced `tokenRangesForInstance()` function to determine the the ranges for each replication set.

For this purpose, the function moves into the `bloomutils` packages.

This PR eliminates the structs `addrsWithBounds` and `instanceWithFingerprints` (which had somewhat similar usage) and unifies them into `rsWithRanges` that holds both the fingerprint ranges and the assigned GroupedChunkRefs.

Signed-off-by: Christian Haudum <christian.haudum@gmail.com>
This PR contains two changes:

1) Extracting the block without an intermediate temp file.
   The commit is cherry-picked from #12021, but that PR got merged into the wrong branch 🙈 

2) Strip `.tar.gz` suffix from block file when extracting it into a directory.

Signed-off-by: Christian Haudum <christian.haudum@gmail.com>
…ply (#11567)

Signed-off-by: Sheikh-Abubaker <sheikhabubaker761@gmail.com>
Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>
Co-authored-by: Karsten Jeschkies <karsten.jeschkies@grafana.com>
Fail startup if directory does not exist or there are not enough permissions. This prevents the bloomstore to fail later in the process once it tries to download and extract blocks.

Signed-off-by: Christian Haudum <christian.haudum@gmail.com>
* Wire up bloom querier metrics to fix nil pointer panic
* Use `bloomutils.TokenRangesForInstance` instead of `subRing.GetTokenRangesForInstance` to avoid `zone not set` error

Signed-off-by: Christian Haudum <christian.haudum@gmail.com>
The bloomstore service is instantiated once at startup and used as dependency for both the bloom compactor and the bloom gateway.

This will prevent instantiating the store in different ways.


Signed-off-by: Christian Haudum <christian.haudum@gmail.com>
chaudum and others added 14 commits February 26, 2024 11:47
Signed-off-by: Christian Haudum <christian.haudum@gmail.com>
Co-authored-by: Christian Haudum <christian.haudum@gmail.com>
#12058)

Signed-off-by: Danny Kopping <danny.kopping@grafana.com>
…on (#12072)

Signed-off-by: Danny Kopping <danny.kopping@grafana.com>
…s not exist (#12080)

The check `!dirEntry.IsDir()` panicked in case `dirEntry` is `nil`, which is the case when the root directory does not exist.

The PR also adds more logging.

Signed-off-by: Christian Haudum <christian.haudum@gmail.com>
#12079)

Signed-off-by: Christian Haudum <christian.haudum@gmail.com>
Co-authored-by: loki-gh-app[bot] <160051081+loki-gh-app[bot]@users.noreply.github.com>
Co-authored-by: Jack Baldry <jack.baldry@grafana.com>
…ore (#12048)

* Add metric for size of metas/blocks fetched

Getting the distribution of the uncompressed size of metas and blocks,
grouped by their source (cache/filesystem/storage).

* Move blocks/metas fetched metrics into bloomstore

* Add minimal logging to bloom gateway processor

* Observe queue duration of tasks

The histogram metric gives us the insight about how long tasks spend
their time enqueued waiting to be processed.

* Remove queueMetrics field from bloom gateway struct

The metrics are only used by the queue instance, which is part of the
bloom gateway struct as well.

* Move various bloom gateway metrics into separate file

This commit introduces a new metrics struct that contains both the
metrics specific the server/handler and the metrics specific to the
worker.

This has the advantage that metrics can be instantiated and accessed
from one single place.

---

Signed-off-by: Christian Haudum <christian.haudum@gmail.com>
@periklis
Copy link

periklis commented Mar 1, 2024

/approve

@periklis
Copy link

periklis commented Mar 1, 2024

/lgtm

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Mar 1, 2024
Copy link

openshift-ci bot commented Mar 1, 2024

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: periklis, xperimental

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:
  • OWNERS [periklis,xperimental]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Mar 1, 2024
@openshift-ci-robot
Copy link

/retest-required

Remaining retests: 0 against base HEAD 62176f0 and 2 for PR HEAD a8aa3f6 in total

The warnings suggest something going wrong where it's not the case.

Signed-off-by: Christian Haudum <christian.haudum@gmail.com>
@openshift-ci openshift-ci bot removed the lgtm Indicates that a PR is ready to be merged. label Mar 1, 2024
Copy link

openshift-ci bot commented Mar 1, 2024

New changes are detected. LGTM label has been removed.

Copy link

openshift-ci bot commented Mar 1, 2024

@xperimental: all tests passed!

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@periklis periklis merged commit 5935fea into openshift:main Mar 2, 2024
6 of 7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet