[doc] Add a documentation page for metrics reference #4910
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
*Motivation* Add a documentation page for metrics reference
sijie
added
the
doc
sijie
requested review from
merlimat,
jiazhai,
codelipenghui and
Jennifer88huang-zz
|
@Anonymitaet Could you please help review? Thank you. |
|
@jennifer88huang glad to help |
Anonymitaet
reviewed
Aug 8, 2019
site2/docs/reference-metrics.md
Outdated
| The broker metrics are exposed under "/metrics" at port 8080. You can change the port by updating `webServicePort` to a different port | ||
| in `broker.conf` configuration file. | ||
|
|
||
| All the metrics exposed by broker are labelled with `cluster=${pulsar_cluster}`. The value of `${pulsar_cluster}` is the pulsar cluster |
There was a problem hiding this comment.
Suggested change
| All the metrics exposed by broker are labelled with `cluster=${pulsar_cluster}`. The value of `${pulsar_cluster}` is the pulsar cluster | |
| All the metrics exposed by a broker are labelled with `cluster=${pulsar_cluster}`. The value of `${pulsar_cluster}` is the pulsar cluster |
site2/docs/reference-metrics.md
Outdated
|
|
||
| > Namespace metrics are only exposed when `exposeTopicLevelMetricsInPrometheus` is set to false. | ||
|
|
||
| All the namespace metrics will be labelled with following labels: |
There was a problem hiding this comment.
Suggested change
| All the namespace metrics will be labelled with following labels: | |
| All the namespace metrics will be labelled with the following labels: |
site2/docs/reference-metrics.md
Outdated
|
|
||
| ## Monitoring | ||
|
|
||
| You can [setup a Prometheus instance](https://prometheus.io/) to collect all the metrics exposed at Pulsar components and setup |
There was a problem hiding this comment.
Suggested change
| You can [setup a Prometheus instance](https://prometheus.io/) to collect all the metrics exposed at Pulsar components and setup | |
| You can [set up a Prometheus instance](https://prometheus.io/) to collect all the metrics exposed at Pulsar components and set up |
setup is a noun
set up is a verb phrase
site2/docs/reference-metrics.md
Outdated
| The metrics exposed by Pulsar are in Prometheus format. The types of metrics are: | ||
|
|
||
| - [Counter](https://prometheus.io/docs/concepts/metric_types/#counter): a cumulative metric that represents a single monotonically increasing counter whose value can only increase or be reset to zero on restart. | ||
| - [Gauge](https://prometheus.io/docs/concepts/metric_types/#gauge): A *gauge* is a metric that represents a single numerical value that can arbitrarily go up and down. |
There was a problem hiding this comment.
Suggested change
| - [Gauge](https://prometheus.io/docs/concepts/metric_types/#gauge): A *gauge* is a metric that represents a single numerical value that can arbitrarily go up and down. | |
| - [Gauge](https://prometheus.io/docs/concepts/metric_types/#gauge): a *gauge* is a metric that represents a single numerical value that can arbitrarily go up and down. |
Same cases for the two items below
site2/docs/reference-metrics.md
Outdated
| | Name | Type | Description | | ||
| |---|---|---| | ||
| | bookie_SERVER_STATUS | Gauge | The server status for bookie server. <br><ul><li>1: the bookie is running in writable mode.</li><li>0: the bookie is running in readonly mode.</li></ul> | | ||
| | bookkeeper_server_ADD_ENTRY_count | Counter | The total number of ADD_ENTRY requests received at the bookie. Label `success` used to distinguish successes and failures | |
There was a problem hiding this comment.
A period should be placed behind the last word of the sentence. Please check all cases.
site2/docs/reference-metrics.md
Outdated
|
|
||
| > Subscription metrics are only exposed when `exposeTopicLevelMetricsInPrometheus` is set to true. | ||
|
|
||
| All the subscription metrics will be labelled with following labels: |
There was a problem hiding this comment.
Suggested change
| All the subscription metrics will be labelled with following labels: | |
| All the subscription metrics are labelled with the following labels: |
site2/docs/reference-metrics.md
Outdated
| > Consumer metrics are only exposed when both `exposeTopicLevelMetricsInPrometheus` and `exposeConsumerLevelMetricsInPrometheus` | ||
| > are set to true. | ||
|
|
||
| All the consumer metrics will be labelled with following labels: |
There was a problem hiding this comment.
Suggested change
| All the consumer metrics will be labelled with following labels: | |
| All the consumer metrics are labelled with the following labels: |
site2/docs/reference-metrics.md
Outdated
| The zookeeper metrics are exposed under "/metrics" at port 8000. You can change the port by configuring a system | ||
| property `stats_server_port` to use a different port. | ||
|
|
||
| ### Server Metrics |
There was a problem hiding this comment.
Suggested change
| ### Server Metrics | |
| ### Server metrics |
We use sentence case rather than title case for headings, please check all occurrences.
|
|
||
| All the metrics exposed by broker are labelled with `cluster=${pulsar_cluster}`. The value of `${pulsar_cluster}` is the pulsar cluster | ||
| name you configured in `broker.conf`. | ||
|
|
There was a problem hiding this comment.
Consider adding the following to show a preview/structure of long contents?
Besides, it is easy for users to locate info for #### (heading 4) contents which are not shown on the right TOC on the webpage.
If agree, please check all cases.
Broker has the following kinds of metrics:
* [Namespace metrics](#namespace-metrics)
* [Replication metrics](#replication-metrics)
* [Topic metrics](#topic-metircs)
* [Subscription metrics](#subscription-metrics)
* [Consumer metrics](#consumer-metrics)
site2/docs/reference-metrics.md
Outdated
| | pulsar_consumer_msg_throughput_out | Gauge | The total message dispatch throughput for a consumer (bytes/second) | | ||
| | pulsar_consumer_available_permits | Gauge | The available permits for for a consumer | | ||
|
|
||
| ## Monitoring |
There was a problem hiding this comment.
Suggested change
| ## Monitoring | |
| ## Monitor |
Use a base verb rather than a noun in headings, which facilitates searching.
|
@Anonymitaet I have addressed all your review comments. PTAL |
Anonymitaet
approved these changes
Aug 12, 2019
|
run cpp tests |
| sidebar_label: Pulsar Metrics | ||
| --- | ||
|
|
||
| <style type="text/css"> |
There was a problem hiding this comment.
do we display this to users?
There was a problem hiding this comment.
it is to set the css style for the table. keep it consistent with other reference pages.
There was a problem hiding this comment.
ok, it belongs to metadata, make sure it will not be displayed to users.
site2/docs/reference-metrics.md
Outdated
|
|
||
| ## ZooKeeper | ||
|
|
||
| The zookeeper metrics are exposed under "/metrics" at port 8000. You can change the port by configuring a system |
There was a problem hiding this comment.
zookeeper --> ZooKeeper
If there is similar cases, please check and refine throughout.
There was a problem hiding this comment.
You can use a different port by configuring the stats_server_port system property.
site2/docs/reference-metrics.md
Outdated
|
|
||
| | Name | Type | Description | | ||
| |---|---|---| | ||
| | zookeeper_server_znode_count | Gauge | Number of z-nodes stored. | |
There was a problem hiding this comment.
Suggested change
| | zookeeper_server_znode_count | Gauge | Number of z-nodes stored. | | |
| | zookeeper_server_znode_count | Gauge | The number of z-nodes stored. | |
There was a problem hiding this comment.
You can check and refine all similar cases.
site2/docs/reference-metrics.md
Outdated
|
|
||
| ## BookKeeper | ||
|
|
||
| The bookkeeper metrics are exposed under "/metrics" at port 8000. You can change the port by updating `prometheusStatsHttpPort` |
There was a problem hiding this comment.
bookkeeper --> BookKeeper
Check and refine all similar cases.
site2/docs/reference-metrics.md
Outdated
|
|
||
| | Name | Type | Description | | ||
| |---|---|---| | ||
| | bookie_journal_JOURNAL_SYNC_count | Counter | The total number of journal fsync operations happening at the bookie. Label `success` used to distinguish successes and failures. | |
There was a problem hiding this comment.
Suggested change
| | bookie_journal_JOURNAL_SYNC_count | Counter | The total number of journal fsync operations happening at the bookie. Label `success` used to distinguish successes and failures. | | |
| | bookie_journal_JOURNAL_SYNC_count | Counter | The total number of journal fsync operations happening at the bookie. The `success` label is used to distinguish successes and failures. | |
site2/docs/reference-metrics.md
Outdated
|
|
||
| ### Namespace metrics | ||
|
|
||
| > Namespace metrics are only exposed when `exposeTopicLevelMetricsInPrometheus` is set to false. |
There was a problem hiding this comment.
Suggested change
| > Namespace metrics are only exposed when `exposeTopicLevelMetricsInPrometheus` is set to false. | |
| > Namespace metrics are only exposed when `exposeTopicLevelMetricsInPrometheus` is set to `false`. |
site2/docs/reference-metrics.md
Outdated
|
|
||
| > Topic metrics are only exposed when `exposeTopicLevelMetricsInPrometheus` is set to true. | ||
|
|
||
| All the topic metrics are labelled with following labels: |
There was a problem hiding this comment.
Suggested change
| All the topic metrics are labelled with following labels: | |
| All the topic metrics are labelled with the following labels: |
site2/docs/reference-metrics.md
Outdated
|
|
||
| > Subscription metrics are only exposed when `exposeTopicLevelMetricsInPrometheus` is set to true. | ||
|
|
||
| All the subscription metrics are labelled with following labels: |
There was a problem hiding this comment.
Suggested change
| All the subscription metrics are labelled with following labels: | |
| All the subscription metrics are labelled with the following labels: |
There was a problem hiding this comment.
Check other similar cases.
site2/docs/reference-metrics.md
Outdated
| |---|---|---| | ||
| | pulsar_consumer_msg_rate_redeliver | Gauge | The total message rate for message being redelivered (messages/second). | | ||
| | pulsar_consumer_unacked_massages | Gauge | The total number of unacked messages of a consumer (messages). | | ||
| | pulsar_consumer_blocked_on_unacked_messages | Gauge | Indicate whether a consumer is blocked on unacked messages or not. <br> <ul><li>1 means the consumer is blocked on waiting unacked messages to be acked.</li><li>0 means the consumer is not blocked on waiting unacked messages to be acked.</li></ul> | |
There was a problem hiding this comment.
Suggested change
| | pulsar_consumer_blocked_on_unacked_messages | Gauge | Indicate whether a consumer is blocked on unacked messages or not. <br> <ul><li>1 means the consumer is blocked on waiting unacked messages to be acked.</li><li>0 means the consumer is not blocked on waiting unacked messages to be acked.</li></ul> | | |
| | pulsar_consumer_blocked_on_unacked_messages | Gauge | Indicate whether a consumer is blocked on unacknowledged messages or not. <br> <ul><li>`1` means the consumer is blocked on waiting unacknowledged messages to be acknowledged.</li><li>`0` means the consumer is not blocked on waiting unacknowledged messages to be acknowledged.</li></ul> | |
site2/docs/reference-metrics.md
Outdated
| You can [set up a Prometheus instance](https://prometheus.io/) to collect all the metrics exposed at Pulsar components and set up | ||
| [Grafana](https://grafana.com/) dashboards to display the metrics and monitor your Pulsar cluster. | ||
|
|
||
| The example Grafana dashboards can be found: |
There was a problem hiding this comment.
Suggested change
| The example Grafana dashboards can be found: | |
| The following are some Grafana dashboards examples: |
|
@jennifer88huang I have addressed your review comments. |
site2/docs/reference-metrics.md
Outdated
|
|
||
| | Name | Type | Description | | ||
| |---|---|---| | ||
| | bookie_journal_JOURNAL_SYNC_count | Counter | The total number of journal fsync operations happening at the bookie. The `success` label used to distinguish successes and failures. | |
There was a problem hiding this comment.
Suggested change
| | bookie_journal_JOURNAL_SYNC_count | Counter | The total number of journal fsync operations happening at the bookie. The `success` label used to distinguish successes and failures. | | |
| | bookie_journal_JOURNAL_SYNC_count | Counter | The total number of journal fsync operations happening at the bookie. The `success` label is used to distinguish successes and failures. | |
|
@jennifer88huang addressed your comments. PTAL |
Jennifer88huang-zz
approved these changes
Aug 20, 2019
codelipenghui
approved these changes
Aug 20, 2019
|
run java8 tests |
1 similar comment
|
run java8 tests |
|
Change the Milestone to 2.4.2, because of conflict. |
wolfstudy
pushed a commit
that referenced
this pull request
Nov 20, 2019
*Motivation* Add a documentation page for metrics reference (cherry picked from commit 5631067)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation
Add a documentation page for metrics reference