Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avoid catastrophic failure when an ES request times out #356

Merged
merged 1 commit into from
Dec 5, 2017

Conversation

blt
Copy link
Collaborator

@blt blt commented Dec 5, 2017

As of 0.20.6 ES it's possible for a bulk request to time out,
which we unwrapped on like goofs. This commit removes that
unwrap and fails the flush in the event of a timeout bubbling up.

This resolves #353 and elastic-rs/elastic#286.

Related to #355.

Signed-off-by: Brian L. Troutwine blt@postmates.com

As of 0.20.6 ES it's possible for a bulk request to time out,
which we unwrapped on like goofs. This commit removes that
unwrap and fails the flush in the event of a timeout bubbling up.

This resolves #353 and elastic-rs/elastic#286.

Related to #355.

Signed-off-by: Brian L. Troutwine <blt@postmates.com>
@blt
Copy link
Collaborator Author

blt commented Dec 5, 2017

@benley @lynchc relevant to your interests. I'll run this in a test-harness for a day plus.

@blt blt requested a review from tsantero December 5, 2017 18:08
Copy link
Contributor

@tsantero tsantero left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the branching here is getting a bit nuts; perhaps a cleaner refactor down the road. otherwise, +1

@blt
Copy link
Collaborator Author

blt commented Dec 5, 2017

@tsantero it's getting rough. I think #355 will help some.

@blt blt merged commit 859ca8a into master Dec 5, 2017
@blt blt deleted the es_timeout_catastrophic_failure branch December 5, 2017 19:58
blt pushed a commit that referenced this pull request Dec 6, 2017
This release adjusts the Elasticsearch sink to better handle timeouts
and clear its internal buffer. Also, we now keep a long-lived sum/count
for Prometheus sink Summaries, even despite the windowing.

  * 859ca8a :: Avoid catastrophic failure when an ES request times out (#356)
  * 071328c :: Perform a linear search in ES buffer (#357)
  * 86f753c :: Introduce Telemetry override for 'count' and 'sample_sum' (#354)

Signed-off-by: Brian L. Troutwine <blt@postmates.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

'elastic' 0.20.6 is unwrap crashing in 0.8.4
2 participants