Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Document backoff options #6903

Merged
merged 2 commits into from Jun 13, 2018
Merged

Conversation

dedemorton
Copy link
Contributor

@dedemorton dedemorton commented Apr 19, 2018

Adds backup options for LS and ES outputs (closes #5419)

NOTE: I wasn't sure quite where to add the options in the yaml file.

Do we want to say anything about the backoff factor used here? Probably not since it's not configurable?

@dedemorton dedemorton requested a review from urso April 19, 2018 19:28
@dedemorton dedemorton added the in progress Pull request is currently in progress. label Apr 19, 2018
@dedemorton dedemorton force-pushed the add_backoff_settings branch 2 times, most recently from f5b22af to b16c10e Compare April 19, 2018 21:50
@dedemorton
Copy link
Contributor Author

@urso I hope I got the details right this time. Right now we're ordering the options by how commonly they are used (with related options appearing next to each other). Not sure that's the best approach, but for now, I've added the options to an arbitrary location because I'm not sure how common it is for users to turn these knobs.

@dedemorton dedemorton added backport needs_backport PR is waiting to be backported to other branches. and removed in progress Pull request is currently in progress. backport labels Apr 19, 2018
@dedemorton dedemorton mentioned this pull request Apr 19, 2018
37 tasks
#backoff.init: 1s

# The maximum number of seconds to wait before attempting to connect to
# Elasticsearch after a connection error. The default is 60s.
Copy link

@urso urso May 29, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some more clarification:

backoff.init is applied on an network error. If connection was active, but connection gets lost or some other protocol/network error occurs, beats will disconnect and use backoff.init. After backoff.init, beats try to reconnect. If the reconnect fails, the backoff is increased exponentially up to backoff.max. After successful reconnect, the backoff timer is reset. On next network error, backoff will start with again with backoff.init.

A (initial) connection error is just one potential error among others.

Copy link
Contributor Author

@dedemorton dedemorton Jun 1, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@urso I'm trying to get all these details into the paragraph without making it too long. Does this work?

The number of seconds to wait before trying to reconnect to Elasticsearch after
a network error. After waiting `backoff.init` seconds, {beatname_uc} tries to
reconnect. If the attempt fails, the backoff timer is increased exponentially up
to `backoff.max`. After a successful connection, the backoff timer is reset. The
default is 1s.

I've left the following sentence out because it restates the previous sentence: "On next network error, backoff will start with again with backoff.init."

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dedemorton dedemorton force-pushed the add_backoff_settings branch 3 times, most recently from a7a272a to 4032597 Compare June 5, 2018 18:17
@dedemorton
Copy link
Contributor Author

@urso This PR is ready for a final review.

Copy link

@urso urso left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@urso
Copy link

urso commented Jun 5, 2018

@dedemorton The PR breaks the build. make update did update the fields.yml in winlogbeat. This file doesn't seem to be correct. Can you remove the changes to fields.yml from your PR?

@dedemorton
Copy link
Contributor Author

@urso I had to rebase and then run make update again. I guess that's a good general best practice...that you should rebase a stale branch before running make update to pick up doc changes (even if GitHub isn't asking you to rebase).

@urso urso merged commit 8f9592e into elastic:master Jun 13, 2018
dedemorton added a commit to dedemorton/beats that referenced this pull request Jun 18, 2018
Adds backoff options for LS and ES outputs (closes elastic#5419)
@dedemorton dedemorton removed the needs_backport PR is waiting to be backported to other branches. label Jun 18, 2018
ruflin pushed a commit that referenced this pull request Jun 19, 2018
Adds backoff options for LS and ES outputs (closes #5419)
leweafan pushed a commit to leweafan/beats that referenced this pull request Apr 28, 2023
Adds backoff options for LS and ES outputs (closes elastic#5419)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Document backoff setting
2 participants