Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Store] Metric to capture timestamp of the last loaded block #4710

Merged
merged 1 commit into from Oct 27, 2021

Conversation

spaparaju
Copy link
Contributor

@spaparaju spaparaju commented Sep 27, 2021

Signed-off-by: SriKrishna Paparaju paparaju@gmail.com

Currently Thanos does not have visibility into when a block is loaded . Lack of this visibility can be a challenge either with chosen downsampling options (or any other reason) might cause data loss and there is no way to know the extent of the data loss. This PR adds metric to capture timestamp of the last loaded block.

  • I added CHANGELOG entry for this change.
  • Change is not relevant to the end user.

Changes

Verification

Added unit test logic, tested thru 'TestBucketStore_e2e() of github.com/thanos-io/thanos/pkg/store'

Copy link
Collaborator

@matej-g matej-g left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it looks fine 👍 But maybe could you give us a bit more context why this is required? What is the use case for it etc.?

pkg/store/bucket_e2e_test.go Show resolved Hide resolved
pkg/store/bucket.go Outdated Show resolved Hide resolved
@spaparaju
Copy link
Contributor Author

I think it looks fine 👍 But maybe could you give us a bit more context why this is required? What is the use case for it etc.?

Good point! Updated the PR description.

@bill3tt
Copy link
Contributor

bill3tt commented Oct 5, 2021

In the Thanos UI stores tab, each store already reports the min_time which (I think) comes from Store's InfoResponse RPC. That's the exact piece of information we want to expose here. Is that what your change is exposing here?

In an ideal world, each component that exposes the Store API would expose this information. Users deploy Thanos in all kinds of configuration, it would be useful to have this information available even if they are deploying components other than Store.

@spaparaju
Copy link
Contributor Author

spaparaju commented Oct 7, 2021

I believe Min time and Max time are set here, currently do not capture the timestamp of a block that recently got loaded. This comment confirms this observation.

pkg/store/bucket.go Outdated Show resolved Hide resolved
pkg/store/bucket.go Outdated Show resolved Hide resolved
pkg/store/bucket.go Outdated Show resolved Hide resolved
pkg/store/bucket_e2e_test.go Outdated Show resolved Hide resolved
Signed-off-by: SriKrishna Paparaju <paparaju@gmail.com>
Copy link
Contributor

@yeya24 yeya24 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@yeya24 yeya24 merged commit 24602c4 into thanos-io:main Oct 27, 2021
akanshat pushed a commit to akanshat/thanos that referenced this pull request Oct 30, 2021
Signed-off-by: SriKrishna Paparaju <paparaju@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants