Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

distributor: Wrap errors from pushing to ingesters #3055

Merged
merged 3 commits into from
Oct 3, 2022

Conversation

aknuds1
Copy link
Contributor

@aknuds1 aknuds1 commented Sep 27, 2022

What this PR does

In the distributor component, wrap errors stemming from calling the push gRPC method on ingesters. If an errors stems from a timeout, that gets wrapped with a specific message.

I hope this will be useful in diagnosing write requests timing out in the ingester component (e.g. due to CPU starvation).

Which issue(s) this PR fixes or relates to

Checklist

  • Tests updated
  • [na] Documentation added
  • [na] CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX]

@aknuds1 aknuds1 force-pushed the chore/distributor-wrap-push-error branch from 90958ac to c034f27 Compare September 27, 2022 11:36
@aknuds1 aknuds1 marked this pull request as ready for review September 27, 2022 12:03
@aknuds1 aknuds1 requested a review from a team as a code owner September 27, 2022 12:03
@aknuds1 aknuds1 force-pushed the chore/distributor-wrap-push-error branch from c034f27 to 578e9ae Compare September 27, 2022 13:21
@aknuds1 aknuds1 force-pushed the chore/distributor-wrap-push-error branch from 578e9ae to 42ec2e3 Compare September 28, 2022 07:15
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
@aknuds1 aknuds1 force-pushed the chore/distributor-wrap-push-error branch from 42ec2e3 to bb2dff7 Compare September 28, 2022 13:54
Copy link
Member

@pstibrany pstibrany left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@pstibrany pstibrany merged commit 82cd7b9 into main Oct 3, 2022
@pstibrany pstibrany deleted the chore/distributor-wrap-push-error branch October 3, 2022 07:47
pr00se added a commit that referenced this pull request Oct 13, 2022
Logiraptor pushed a commit that referenced this pull request Oct 13, 2022
pstibrany pushed a commit that referenced this pull request Oct 14, 2022
grafanabot pushed a commit that referenced this pull request Oct 14, 2022
pracucci pushed a commit that referenced this pull request Oct 14, 2022
…3211) (#3217)

This reverts commit 82cd7b9.

(cherry picked from commit e719981)

Co-authored-by: Patryk Prus <p@trykpr.us>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants