Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Retry bulk chunk requests to Elasticsearch 2, 5, and 6 #1626

Closed
wants to merge 7 commits into from

Commits on Jan 23, 2019

  1. Retry bulk chunk requests to Elasticsearch 2, 5, and 6

    Implements retries when saving bulk documents to Elasticsearch.
    This is useful for transient connection problems and other failures
    which are temporary in nature.
    
    The current behavior will cause the entire "Save to Elasticsearch" step
    to fail and retry, which creates redundant saves to the cluster. There
    is no existing method to enable a single failed chunk to retry.
    
    This behavior is implemented with two new options, withMaxRetries and
    withRetryPause, which can be passed in the configuration as "maxRetries"
    and "retryPause" respectively.
    Josh Whelchel committed Jan 23, 2019
    Configuration menu
    Copy the full SHA
    7146ab9 View commit details
    Browse the repository at this point in the history
  2. Catch all exceptions within the chunks.forEach block.

    Some of the exceptions thrown are not caught within the Client bulk() method.
    Josh Whelchel committed Jan 23, 2019
    Configuration menu
    Copy the full SHA
    9605842 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    00c299f View commit details
    Browse the repository at this point in the history
  4. Revert "Adds new params maxRetries and retryPause to saveAsElasticsea…

    …rch method"
    
    This exceeds the number of max params in scala style. So rely on it coming as cmdLine arg.
    
    This reverts commit 00c299f.
    Josh Whelchel committed Jan 23, 2019
    Configuration menu
    Copy the full SHA
    b4d95d1 View commit details
    Browse the repository at this point in the history

Commits on Feb 19, 2019

  1. Implements FluentBackoff

    Josh Whelchel committed Feb 19, 2019
    Configuration menu
    Copy the full SHA
    2bd5492 View commit details
    Browse the repository at this point in the history

Commits on Feb 21, 2019

  1. Implements @setup for ElasticsearchWriters, restores formatting

    Josh Whelchel committed Feb 21, 2019
    Configuration menu
    Copy the full SHA
    87e7805 View commit details
    Browse the repository at this point in the history
  2. Restore indentation formatting on ES Bound constructors

    Josh Whelchel committed Feb 21, 2019
    Configuration menu
    Copy the full SHA
    d8c1724 View commit details
    Browse the repository at this point in the history