Update monitor-prometheus-and-grafana-integration.md #21907
+49
−9
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change | ||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| @@ -1,14 +1,20 @@ | ||||||||||||||||||||||||||||||
| title: Integrate TiDB Cloud with Prometheus and Grafana | ||||||||||||||||||||||||||||||
| summary: Learn how to monitor your TiDB cluster with the Prometheus and Grafana integration. | ||||||||||||||||||||||||||||||
| --- | ||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||
| # Integrate TiDB Cloud with Prometheus and Grafana | ||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||
| TiDB Cloud provides a [Prometheus](https://prometheus.io/) API endpoint. If you have a Prometheus service, you can monitor key metrics of TiDB Cloud from the endpoint easily. | ||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||
| This document describes how to configure your Prometheus service to read key metrics from the TiDB Cloud endpoint and how to view the metrics using [Grafana](https://grafana.com/). | ||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||
| ## Prometheus integration version | ||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||
| TiDB Cloud has supported the project-level Prometheus integration (Beta) since March 4, 2022(?please help update the date). Starting from October 21, 2025, TiDB Cloud introduces the cluster-level Prometheus integration (Preview). | ||||||||||||||||||||||||||||||
|
There was a problem hiding this comment.
Suggested change
|
||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||
| - **Cluster-level Prometheus integration (Preview)**: if no legacy project-level Prometheus integration remains undeleted within your organization by October 21, 2025, TiDB Cloud provides the cluster-level Prometheus integration (Preview) for your organization to experience the latest enhancements. This feature is currently available for Dedicated clusters on AWS and GCP, and we are working to bring it to Azure in the near future. | ||||||||||||||||||||||||||||||
|
Check failure on line 15 in tidb-cloud/monitor-prometheus-and-grafana-integration.md
|
||||||||||||||||||||||||||||||
|
There was a problem hiding this comment.
Suggested change
There was a problem hiding this comment. Avoid documenting future features or products, even in innocuous ways. |
||||||||||||||||||||||||||||||
| - **Legacy project-level Prometheus integration (Beta)**: if at least one legacy project-level Prometheus integration remains undeleted within your organization by October 21, 2025, TiDB Cloud retains both existing and new integrations at the project level for your organization to avoid affecting current dashboards. | ||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||
| ## Prerequisites | ||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||
| - To integrate TiDB Cloud with Prometheus, you must have a self-hosted or managed Prometheus service. | ||||||||||||||||||||||||||||||
|
|
@@ -25,17 +31,31 @@ | |||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||
| ### Step 1. Get a scrape_config file for Prometheus | ||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||
| Before configuring your Prometheus service to read metrics of TiDB Cloud, you need to generate a `scrape_config` YAML file in TiDB Cloud first. The `scrape_config` file contains a unique bearer token that allows the Prometheus service to monitor your target database clusters. | ||||||||||||||||||||||||||||||
|
There was a problem hiding this comment.
Suggested change
|
||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||
| Depending on your [Prometheus integration version](#prometheus-integration-version), the steps to get the `scrape_config` file for Prometheus and access the integration page are different. | ||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||
| <SimpleTab> | ||||||||||||||||||||||||||||||
| <div label="Cluster-level Prometheus integration (Preview)"> | ||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||
| 1. In the [TiDB Cloud console](https://tidbcloud.com/), navigate to the [**Clusters**](https://tidbcloud.com/project/clusters) page of your project, and then click the name of your target cluster to go to its overview page. | ||||||||||||||||||||||||||||||
| 2. In the left navigation pane, click **Settings** > **Integrations**. | ||||||||||||||||||||||||||||||
| 3. On the **Integrations** page, click **Integration to Prometheus(Preview)**. | ||||||||||||||||||||||||||||||
| 4. Click **Add File** to generate and show the scrape_config file for the current cluster. | ||||||||||||||||||||||||||||||
|
There was a problem hiding this comment.
Suggested change
|
||||||||||||||||||||||||||||||
| 5. Make a copy of the scrape_config file content for later use. | ||||||||||||||||||||||||||||||
|
There was a problem hiding this comment.
Suggested change
|
||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||
| </div> | ||||||||||||||||||||||||||||||
| <div label="Legacy project-level Prometheus integration (Beta)"> | ||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||
| 1. In the [TiDB Cloud console](https://tidbcloud.com), switch to your target project using the combo box in the upper-left corner. | ||||||||||||||||||||||||||||||
| 2. In the left navigation pane, click **Project Settings** > **Integrations**. | ||||||||||||||||||||||||||||||
| 3. On the **Integrations** page, click **Integration to Prometheus (BETA)**. | ||||||||||||||||||||||||||||||
| 4. Click **Add File** to generate and show the scrape_config file for the current project. | ||||||||||||||||||||||||||||||
| 5. Make a copy of the `scrape_config` file content for later use. | ||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||
| </div> | ||||||||||||||||||||||||||||||
| </SimpleTab> | ||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||
| > **Note:** | ||||||||||||||||||||||||||||||
| > | ||||||||||||||||||||||||||||||
| > For security reasons, TiDB Cloud only shows a newly generated `scrape_config` file once. Ensure that you copy the content before closing the file window. If you forget to do so, you need to delete the `scrape_config` file in TiDB Cloud and generate a new one. To delete a `scrape_config` file, select the file, click **...**, and then click **Delete**. | ||||||||||||||||||||||||||||||
|
Comment on lines
59
to
61
There was a problem hiding this comment.
Suggested change
|
||||||||||||||||||||||||||||||
|
|
@@ -54,7 +74,20 @@ | |||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||
| After your Prometheus service is reading metrics from TiDB Cloud, you can use Grafana GUI dashboards to visualize the metrics as follows: | ||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||
| 1. Depending on your [Prometheus integration version](#prometheus-integration-version), the link to download the Grafana dashboard JSON of TiDB Cloud for Prometheus is different. | ||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||
| <SimpleTab> | ||||||||||||||||||||||||||||||
| <div label="Cluster-level Prometheus integration (Preview)"> | ||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||
| Please download the Grafana dashboard JSON of TiDB Cloud [here](https://github.com/pingcap/docs/blob/master/tidb-cloud/monitor-prometheus-and-grafana-integration-tidb-cloud-dynamic-tracker.json). | ||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||
| </div> | ||||||||||||||||||||||||||||||
| <div label="Legacy project-level Prometheus integration (Beta)"> | ||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||
| Please download the Grafana dashboard JSON of TiDB Cloud [here](https://github.com/pingcap/docs/blob/master/tidb-cloud/monitor-prometheus-and-grafana-integration-grafana-dashboard-UI.json). | ||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||
| </div> | ||||||||||||||||||||||||||||||
| </SimpleTab> | ||||||||||||||||||||||||||||||
|
Comment on lines
+79
to
+90
There was a problem hiding this comment.
Suggested change
|
||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||
| 2. [Import this JSON to your own Grafana GUI](https://grafana.com/docs/grafana/v8.5/dashboards/export-import/#import-dashboard) to visualize the metrics. | ||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||
|
|
@@ -73,7 +106,7 @@ | |||||||||||||||||||||||||||||
| 1. Follow [Step 1](#step-1-get-a-scrape_config-file-for-prometheus) to create a new `scrape_config` file for Prometheus. | ||||||||||||||||||||||||||||||
| 2. Add the content of the new file to your Prometheus configuration file. | ||||||||||||||||||||||||||||||
| 3. Once you have confirmed that your Prometheus service is still able to read from TiDB Cloud, remove the content of the old `scrape_config` file from your Prometheus configuration file. | ||||||||||||||||||||||||||||||
| 4. On the **Integrations** page of your project/cluster, delete the corresponding old `scrape_config` file to block anyone else from using it to read from the TiDB Cloud Prometheus endpoint. | ||||||||||||||||||||||||||||||
|
There was a problem hiding this comment.
Suggested change
|
||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||
| ## Metrics available to Prometheus | ||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||
|
|
@@ -104,6 +137,13 @@ | |||||||||||||||||||||||||||||
| | tidbcloud_resource_manager_resource_unit_read_request_unit | gauge | cluster_name: `<cluster name>`<br/>resource_group: `<group-name>` | The read request units consumed by Resource Manager | | ||||||||||||||||||||||||||||||
| | tidbcloud_resource_manager_resource_unit_write_request_unit | gauge | cluster_name: `<cluster name>`<br/>resource_group: `<group-name>` | The write request units consumed by Resource Manager | | ||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||
| For cluster-level Prometheus integration, the following additional metrics are also available: | ||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||
|
There was a problem hiding this comment.
Suggested change
|
||||||||||||||||||||||||||||||
| | tidbcloud_dm_task_status | gauge | instance: `instance`<br/>task: `task`<br/>cluster_name: `<cluster name>` | Task State of Data Migration:<br/>0: Invalid<br/>1: New<br/>2: Running<br/>3: Paused<br/>4: Stopped<br/>5: Finished<br/>15: Error | | ||||||||||||||||||||||||||||||
| | tidbcloud_dm_syncer_replication_lag_bucket | gauge | instance: `instance`<br/>cluster_name: `<cluster name>` | Replicate lag (bucket) of Data Migration. | | ||||||||||||||||||||||||||||||
| | tidbcloud_dm_syncer_replication_lag_gauge | gauge | instance: `instance`<br/>task: `task`<br/>cluster_name: `<cluster name>` | Replicate lag (gauge) of Data Migration. | | ||||||||||||||||||||||||||||||
| | tidbcloud_dm_relay_read_error_count | gauge | instance: `instance`<br/>cluster_name: `<cluster name>` | Fail to read binlog from master. | | ||||||||||||||||||||||||||||||
|
Comment on lines
+142
to
+145
There was a problem hiding this comment.
Suggested change
|
||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||
| ## FAQ | ||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||
| - Why does the same metric have different values on Grafana and the TiDB Cloud console at the same time? | ||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.