Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add fluent backoff support for ES #1826

Merged
merged 2 commits into from Apr 10, 2019

Commits on Apr 9, 2019

  1. Retry bulk chunk requests to Elasticsearch 2, 5, and 6

    Implements retries when saving bulk documents to Elasticsearch.
    This is useful for transient connection problems and other failures
    which are temporary in nature.
    
    The current behavior will cause the entire "Save to Elasticsearch" step
    to fail and retry, which creates redundant saves to the cluster. There
    is no existing method to enable a single failed chunk to retry.
    
    This behavior is implemented with two new options, withMaxRetries and
    withRetryPause, which can be passed in the configuration as "maxRetries"
    and "retryPause" respectively.
    
    Catch all exceptions within the chunks.forEach block.
    
    Some of the exceptions thrown are not caught within the Client bulk() method.
    
    Adds new params maxRetries and retryPause to saveAsElasticsearch method
    
    Revert "Adds new params maxRetries and retryPause to saveAsElasticsearch method"
    
    This exceeds the number of max params in scala style. So rely on it coming as cmdLine arg.
    
    This reverts commit 00c299f.
    
    Implements FluentBackoff
    
    Implements @setup for ElasticsearchWriters, restores formatting
    
    Restore indentation formatting on ES Bound constructors
    Josh Whelchel authored and regadas committed Apr 9, 2019
    Configuration menu
    Copy the full SHA
    8c0ebf5 View commit details
    Browse the repository at this point in the history

Commits on Apr 10, 2019

  1. Decouple request retry logic

    regadas committed Apr 10, 2019
    Configuration menu
    Copy the full SHA
    0689f2b View commit details
    Browse the repository at this point in the history