Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use highly load preferrable fluent-plugin-elasticsearch configuration #300

Merged

Conversation

cosmo0920
Copy link
Contributor

fluent-plugin-elasticsearch reloads connection after 10000 requests. (Not correspond to events counts because ES plugin uses bulk API.)

This functionality which is originated from elasticsearch-ruby gem is enabled by default.

Sometimes this reloading functionality bothers users to send events with ES plugin.

On k8s platform, users sometimes shall specify the following settings:

reload_connections false
reconnect_on_error true
reload_on_failure true

This is originally reported at
uken/fluent-plugin-elasticsearch#525.

On k8s, Fluentd sometimes handles flood of events.
This is a pitfall to use fluent-plugin-elasticsearch on k8s.
So, this parameter set should be default.

Signed-off-by: Hiroshi Hatake hatake@clear-code.com

fluent-plugin-elasticsearch reloads connection after 10000 requests. (Not correspond to events counts because ES plugin uses bulk API.)

This functionality which is originated from elasticsearch-ruby gem is enabled by default.

Sometimes this reloading functionality bothers users to send events with ES plugin.

On k8s platform, users sometimes shall specify the following settings:

```aconf
reload_connections false
reconnect_on_error true
reload_on_failure true
```

This is originally reported at
uken/fluent-plugin-elasticsearch#525.

On k8s, Fluentd sometimes handles flood of events.
This is a pitfall to use fluent-plugin-elasticsearch on k8s.
So, this parameter set should be default.

Signed-off-by: Hiroshi Hatake <hatake@clear-code.com>
@repeatedly repeatedly merged commit c25c03e into fluent:master Apr 12, 2019
@repeatedly
Copy link
Member

OK. Thanks!

@cosmo0920 cosmo0920 deleted the es-high-load-perferrable-configuration branch April 13, 2019 04:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants