New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[FLINK-7697][connectors] Add metrics for ElasticSearchSink #8568
Conversation
Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community Automated ChecksLast check on commit bdea9df (Thu Sep 23 18:02:36 UTC 2021) Warnings:
Mention the bot in a comment to re-run the automated checks. Review Progress
Please see the Pull Request Review Guide for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commandsThe @flinkbot bot supports the following commands:
|
Will there be any new progess? Currently in 1.9.0, es connector is still a black box. I've tried to turn on logging of |
影响了哪个版本 |
are there any metrics available for the elasticsearch sink? |
Connector has been moved to https://github.com/apache/flink-connector-elasticsearch |
What is the purpose of the change
This pull request add metrics to the Elasticsearch connectors, to make it easier to operate and monitor in a production environment.
Brief change log
The following metrics have been added:
Note: Gauge doesn't feel like the right metric type for measuring bulkRequest latency, as it will only capture the last requestLatency before the gauge gets polled. Flink metric system currently lacks a timer type that can be used to capture event latencies in a richer way. I opened FLINK-12680 to track addition of a timer type.
Another metric-related thing we noticed during internal reviews is that the SimpleCounter is not thread safe, which makes it somewhat risky/expensive to use it in a multithreaded context. I opened FLINK-12681 to track making SimpleCounter thread-safe.
Verifying this change
This change is a trivial rework / code cleanup without any test coverage. Also manually verified the metrics by hooking with our internal time-series system.
Does this pull request potentially affect one of the following parts:
@Public(Evolving)
: noDocumentation