Removed AspectJ based metrics for ZooKeeper #10533
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
merlimat
added
the
type/enhancement
merlimat
requested review from
sijie,
rdhabalia,
jerrypeng,
jiazhai,
eolivelli and
codelipenghui
codelipenghui
approved these changes
May 11, 2021
|
/pulsarbot run-failure-checks |
eolivelli
approved these changes
May 11, 2021
Sounds good. We could go to 3.7.0 after Pulsar 2.8 is cut. |
rdhabalia
approved these changes
May 11, 2021
rdhabalia
reviewed
May 11, 2021
|
@rdhabalia The metrics that ZooKeeper exports are going to be different, in names but also types. It will not be possible to maintain a compatibility layer here. Even with that, I think that it would be much better to just move to the metrics supported by ZooKeeper instead of the current situation. |
codelipenghui
pushed a commit
that referenced
this pull request
Jun 3, 2021
…10803) ### Motivation After upgrade zookeeper version to 3.6.2 in #8590 and removed AspectJ based metrics for ZooKeeper in #10533, the zookeeper's prometheus metric has lost if we start zookeeper by `bin/puldar-daemon start zookeeper`. Due to zookeeper 3.6.0+ has add internal prometheus metric provider, so we can turn on by default in pulsar. ### Modification 1. turn on zookeeper prometheus metric provider by default in `conf/zookeeper.conf` and use 8000 as default port sync with old zookeeper metric port 2. add grafana panel for new zookeeper metrics 3. remove old prometheus metric provider in `ZooKeeperStarter` and `ConfigurationStoreStarter`. 4. update reference-metric.md doc
yangl
pushed a commit
to yangl/pulsar
that referenced
this pull request
Jun 23, 2021
…pache#10803) ### Motivation After upgrade zookeeper version to 3.6.2 in apache#8590 and removed AspectJ based metrics for ZooKeeper in apache#10533, the zookeeper's prometheus metric has lost if we start zookeeper by `bin/puldar-daemon start zookeeper`. Due to zookeeper 3.6.0+ has add internal prometheus metric provider, so we can turn on by default in pulsar. ### Modification 1. turn on zookeeper prometheus metric provider by default in `conf/zookeeper.conf` and use 8000 as default port sync with old zookeeper metric port 2. add grafana panel for new zookeeper metrics 3. remove old prometheus metric provider in `ZooKeeperStarter` and `ConfigurationStoreStarter`. 4. update reference-metric.md doc
pkumar-singh
pushed a commit
to pkumar-singh/pulsar
that referenced
this pull request
Aug 10, 2021
…pache#10803) After upgrade zookeeper version to 3.6.2 in apache#8590 and removed AspectJ based metrics for ZooKeeper in apache#10533, the zookeeper's prometheus metric has lost if we start zookeeper by `bin/puldar-daemon start zookeeper`. Due to zookeeper 3.6.0+ has add internal prometheus metric provider, so we can turn on by default in pulsar. 1. turn on zookeeper prometheus metric provider by default in `conf/zookeeper.conf` and use 8000 as default port sync with old zookeeper metric port 2. add grafana panel for new zookeeper metrics 3. remove old prometheus metric provider in `ZooKeeperStarter` and `ConfigurationStoreStarter`. 4. update reference-metric.md doc
4 tasks
bharanic-dev
pushed a commit
to bharanic-dev/pulsar
that referenced
this pull request
Mar 18, 2022
…pache#10803) ### Motivation After upgrade zookeeper version to 3.6.2 in apache#8590 and removed AspectJ based metrics for ZooKeeper in apache#10533, the zookeeper's prometheus metric has lost if we start zookeeper by `bin/puldar-daemon start zookeeper`. Due to zookeeper 3.6.0+ has add internal prometheus metric provider, so we can turn on by default in pulsar. ### Modification 1. turn on zookeeper prometheus metric provider by default in `conf/zookeeper.conf` and use 8000 as default port sync with old zookeeper metric port 2. add grafana panel for new zookeeper metrics 3. remove old prometheus metric provider in `ZooKeeperStarter` and `ConfigurationStoreStarter`. 4. update reference-metric.md doc
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation
So far, we have been relying on a wacky approach to gather metrics for Zookeeper usage. That was done because ZK used not to have a decent set of exported metrics in server or client side.
We were relying on AspectJ to intercept specific points in ZooKeeper client/server execution and collect the data points and expose them through Prometheus.
There are few problem with this approach:
Since we have already upgraded ZooKeeper to 3.6, it already has support for pluggable metrics provider and Prometheus is one of the options. There's no point anymore to continue doing the AspectJ hack.
In separate PRs I'll add the ZK config and submit ZK Grafana dashboards.