feat: adds runtime_metrics #3127

killme2008 · 2024-01-10T06:54:33Z

I hereby agree to the terms of the GreptimeDB CLA

What's changed and what's your intention?

Adds runtime_metrics virtual table to information_schema, which can retrieve the runtime metrics by SQL, for example:

mysql> select * from runtime_metrics;
+--------------------------------------------------------+-------------------------+--------------------------------------------------------+---------+-----------+
| metric_name                                            | value                   | labels                                                 | node    | node_type |
+--------------------------------------------------------+-------------------------+--------------------------------------------------------+---------+-----------+
| greptime_app_version                                   |                       1 | short_version=0.5.1, version=greptimedb-main-1fc168b   | unknown | unknown   |
| greptime_catalog_catalog_count                         |                       1 |                                                        | unknown | unknown   |
| greptime_catalog_schema_count                          |                       2 |                                                        | unknown | unknown   |
| greptime_datanode_handle_region_request_elapsed_bucket |                      82 | datanode_region_request_type=open, le=0.005            | unknown | unknown   |
| greptime_datanode_handle_region_request_elapsed_bucket |                      87 | datanode_region_request_type=open, le=0.01             | unknown | unknown   |
| greptime_datanode_handle_region_request_elapsed_bucket |                     102 | datanode_region_request_type=open, le=0.025            | unknown | unknown   |
| greptime_datanode_handle_region_request_elapsed_bucket |                     102 | datanode_region_request_type=open, le=0.05             | unknown | unknown   |
| greptime_datanode_handle_region_request_elapsed_bucket |                     102 | datanode_region_request_type=open, le=0.1              | unknown | unknown   |
| greptime_datanode_handle_region_request_elapsed_bucket |                     102 | datanode_region_request_type=open, le=0.25             | unknown | unknown   |
| greptime_datanode_handle_region_request_elapsed_bucket |                     102 | datanode_region_request_type=open, le=0.5              | unknown | unknown   |
| greptime_datanode_handle_region_request_elapsed_bucket |                     102 | datanode_region_request_type=open, le=1                | unknown | unknown   |
| greptime_datanode_handle_region_request_elapsed_bucket |                     102 | datanode_region_request_type=open, le=2.5              | unknown | unknown   |
| greptime_datanode_handle_region_request_elapsed_bucket |                     102 | datanode_region_request_type=open, le=5                | unknown | unknown   |
| greptime_datanode_handle_region_request_elapsed_bucket |                     102 | datanode_region_request_type=open, le=10               | unknown | unknown   |
| greptime_datanode_handle_region_request_elapsed_bucket |                     102 | datanode_region_request_type=open, le=+Inf             | unknown | unknown   |
| greptime_datanode_handle_region_request_elapsed_sum    |              0.39604821 | datanode_region_request_type=open                      | unknown | unknown   |
| greptime_datanode_handle_region_request_elapsed_count  |                     102 | datanode_region_request_type=open                      | unknown | unknown   |
| greptime_frontend_handle_sql_elapsed_bucket            |                       2 | le=0.005                                               | unknown | unknown   |
| greptime_frontend_handle_sql_elapsed_bucket            |                       2 | le=0.01                                                | unknown | unknown   |
| greptime_frontend_handle_sql_elapsed_bucket            |                       2 | le=0.025                                               | unknown | unknown   |
| greptime_frontend_handle_sql_elapsed_bucket            |                       2 | le=0.05                                                | unknown | unknown   |
| greptime_frontend_handle_sql_elapsed_bucket            |                       5 | le=0.1                                                 | unknown | unknown   |
| greptime_frontend_handle_sql_elapsed_bucket            |                       5 | le=0.25                                                | unknown | unknown   |
| greptime_frontend_handle_sql_elapsed_bucket            |                       5 | le=0.5                                                 | unknown | unknown   |
| greptime_frontend_handle_sql_elapsed_bucket            |                       5 | le=1                                                   | unknown | unknown   |
| greptime_frontend_handle_sql_elapsed_bucket            |                       5 | le=2.5                                                 | unknown | unknown   |
| greptime_frontend_handle_sql_elapsed_bucket            |                       5 | le=5                                                   | unknown | unknown   |
| greptime_frontend_handle_sql_elapsed_bucket            |                       5 | le=10                                                  | unknown | unknown   |
| greptime_frontend_handle_sql_elapsed_bucket            |                       5 | le=+Inf                                                | unknown | unknown   |
| greptime_frontend_handle_sql_elapsed_sum               |             0.243122332 |                                                        | unknown | unknown   |
| greptime_frontend_handle_sql_elapsed_count             |                       5 |                                                        | unknown | unknown   |
| greptime_meta_create_catalog_bucket                    |                       1 | le=0.005                                               | unknown | unknown   |
| greptime_meta_create_catalog_bucket                    |                       1 | le=0.01                                                | unknown | unknown   |
| greptime_meta_create_catalog_bucket                    |                       1 | le=0.025                                               | unknown | unknown   |
| greptime_meta_create_catalog_bucket                    |                       1 | le=0.05                                                | unknown | unknown   |
| greptime_meta_create_catalog_bucket                    |                       1 | le=0.1                                                 | unknown | unknown   |
| greptime_meta_create_catalog_bucket                    |                       1 | le=0.25                                                | unknown | unknown   |
| greptime_meta_create_catalog_bucket                    |                       1 | le=0.5                                                 | unknown | unknown   |
| greptime_meta_create_catalog_bucket                    |                       1 | le=1                                                   | unknown | unknown   |
| greptime_meta_create_catalog_bucket                    |                       1 | le=2.5                                                 | unknown | unknown   |
| greptime_meta_create_catalog_bucket                    |                       1 | le=5                                                   | unknown | unknown   |
| greptime_meta_create_catalog_bucket                    |                       1 | le=10                                                  | unknown | unknown   |
| greptime_meta_create_catalog_bucket                    |                       1 | le=+Inf                                                | unknown | unknown   |
| greptime_meta_create_catalog_sum                       |             0.000043333 |                                                        | unknown | unknown   |
| greptime_meta_create_catalog_count                     |                       1 |                                                        | unknown | unknown   |
| greptime_meta_create_schema_bucket                     |                       3 | le=0.005                                               | unknown | unknown   |
| greptime_meta_create_schema_bucket                     |                       3 | le=0.01                                                | unknown | unknown   |
| greptime_meta_create_schema_bucket                     |                       3 | le=0.025                                               | unknown | unknown   |
| greptime_meta_create_schema_bucket                     |                       3 | le=0.05                                                | unknown | unknown   |
| greptime_meta_create_schema_bucket                     |                       3 | le=0.1                                                 | unknown | unknown   |
| greptime_meta_create_schema_bucket                     |                       3 | le=0.25                                                | unknown | unknown   |
| greptime_meta_create_schema_bucket                     |                       3 | le=0.5                                                 | unknown | unknown   |
| greptime_meta_create_schema_bucket                     |                       3 | le=1                                                   | unknown | unknown   |
| greptime_meta_create_schema_bucket                     |                       3 | le=2.5                                                 | unknown | unknown   |
| greptime_meta_create_schema_bucket                     |                       3 | le=5                                                   | unknown | unknown   |
| greptime_meta_create_schema_bucket                     |                       3 | le=10                                                  | unknown | unknown   |
| greptime_meta_create_schema_bucket                     |                       3 | le=+Inf                                                | unknown | unknown   |
| greptime_meta_create_schema_sum                        | 0.000048583000000000006 |                                                        | unknown | unknown   |
| greptime_meta_create_schema_count                      |                       3 |                                                        | unknown | unknown   |
| greptime_mito_handle_request_elapsed_bucket            |                      82 | le=0.005, type=Open                                    | unknown | unknown   |
| greptime_mito_handle_request_elapsed_bucket            |                      88 | le=0.01, type=Open                                     | unknown | unknown   |
| greptime_mito_handle_request_elapsed_bucket            |                     102 | le=0.025, type=Open                                    | unknown | unknown   |
| greptime_mito_handle_request_elapsed_bucket            |                     102 | le=0.05, type=Open                                     | unknown | unknown   |
| greptime_mito_handle_request_elapsed_bucket            |                     102 | le=0.1, type=Open                                      | unknown | unknown   |
| greptime_mito_handle_request_elapsed_bucket            |                     102 | le=0.25, type=Open                                     | unknown | unknown   |
| greptime_mito_handle_request_elapsed_bucket            |                     102 | le=0.5, type=Open                                      | unknown | unknown   |
| greptime_mito_handle_request_elapsed_bucket            |                     102 | le=1, type=Open                                        | unknown | unknown   |
| greptime_mito_handle_request_elapsed_bucket            |                     102 | le=2.5, type=Open                                      | unknown | unknown   |
| greptime_mito_handle_request_elapsed_bucket            |                     102 | le=5, type=Open                                        | unknown | unknown   |
| greptime_mito_handle_request_elapsed_bucket            |                     102 | le=10, type=Open                                       | unknown | unknown   |
| greptime_mito_handle_request_elapsed_bucket            |                     102 | le=+Inf, type=Open                                     | unknown | unknown   |
| greptime_mito_handle_request_elapsed_sum               |     0.39196437599999984 | type=Open                                              | unknown | unknown   |
| greptime_mito_handle_request_elapsed_count             |                     102 | type=Open                                              | unknown | unknown   |
| greptime_mito_region_count                             |                     102 |                                                        | unknown | unknown   |
| greptime_mito_write_buffer_bytes                       |                  102987 |                                                        | unknown | unknown   |
| greptime_query_create_physicalplan_elapsed_bucket      |                       2 | le=0.005                                               | unknown | unknown   |
| greptime_query_create_physicalplan_elapsed_bucket      |                       3 | le=0.01                                                | unknown | unknown   |
| greptime_query_create_physicalplan_elapsed_bucket      |                       3 | le=0.025                                               | unknown | unknown   |
| greptime_query_create_physicalplan_elapsed_bucket      |                       3 | le=0.05                                                | unknown | unknown   |
| greptime_query_create_physicalplan_elapsed_bucket      |                       3 | le=0.1                                                 | unknown | unknown   |
| greptime_query_create_physicalplan_elapsed_bucket      |                       3 | le=0.25                                                | unknown | unknown   |
| greptime_query_create_physicalplan_elapsed_bucket      |                       3 | le=0.5                                                 | unknown | unknown   |
| greptime_query_create_physicalplan_elapsed_bucket      |                       3 | le=1                                                   | unknown | unknown   |
| greptime_query_create_physicalplan_elapsed_bucket      |                       3 | le=2.5                                                 | unknown | unknown   |
| greptime_query_create_physicalplan_elapsed_bucket      |                       3 | le=5                                                   | unknown | unknown   |
| greptime_query_create_physicalplan_elapsed_bucket      |                       3 | le=10                                                  | unknown | unknown   |
| greptime_query_create_physicalplan_elapsed_bucket      |                       3 | le=+Inf                                                | unknown | unknown   |
| greptime_query_create_physicalplan_elapsed_sum         |             0.010118083 |                                                        | unknown | unknown   |
| greptime_query_create_physicalplan_elapsed_count       |                       3 |                                                        | unknown | unknown   |
| greptime_query_execute_plan_elapsed_bucket             |                       3 | le=0.005                                               | unknown | unknown   |
| greptime_query_execute_plan_elapsed_bucket             |                       3 | le=0.01                                                | unknown | unknown   |
| greptime_query_execute_plan_elapsed_bucket             |                       3 | le=0.025                                               | unknown | unknown   |
| greptime_query_execute_plan_elapsed_bucket             |                       3 | le=0.05                                                | unknown | unknown   |
| greptime_query_execute_plan_elapsed_bucket             |                       3 | le=0.1                                                 | unknown | unknown   |
| greptime_query_execute_plan_elapsed_bucket             |                       3 | le=0.25                                                | unknown | unknown   |
| greptime_query_execute_plan_elapsed_bucket             |                       3 | le=0.5                                                 | unknown | unknown   |
| greptime_query_execute_plan_elapsed_bucket             |                       3 | le=1                                                   | unknown | unknown   |
| greptime_query_execute_plan_elapsed_bucket             |                       3 | le=2.5                                                 | unknown | unknown   |
| greptime_query_execute_plan_elapsed_bucket             |                       3 | le=5                                                   | unknown | unknown   |
| greptime_query_execute_plan_elapsed_bucket             |                       3 | le=10                                                  | unknown | unknown   |
...
...

TODO:

Supports node and node_type for cluster version.

Checklist

I have written the necessary rustdoc comments.
I have added the necessary unit tests and integration tests.
This PR does not require documentation updates.

Refer to a related PR or issue link (optional)

#2931

tests/cases/standalone/common/system/information_schema.result

codecov · 2024-01-10T07:14:51Z

Codecov Report

Attention: 20 lines in your changes are missing coverage. Please review.

Comparison is base (1fc168b) 85.52% compared to head (cba6d8a) 85.04%.
Report is 3 commits behind head on main.

❗ Current head cba6d8a differs from pull request most recent head bb72a84. Consider uploading reports for the commit bb72a84 to get more accurate results

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3127      +/-   ##
==========================================
- Coverage   85.52%   85.04%   -0.48%     
==========================================
  Files         818      819       +1     
  Lines      133850   134003     +153     
==========================================
- Hits       114474   113964     -510     
- Misses      19376    20039     +663

src/catalog/src/information_schema/runtime_metrics.rs

waynexia

Others LGTM

sunng87 · 2024-01-10T08:47:19Z

We need to make sure this table is only available in greptime catalog so it doesn't leak cluster information to tenants.

sunng87 · 2024-01-10T08:49:03Z

And you might want to include a timestamp column for this table although it's always current timestamp.

killme2008 · 2024-01-10T08:49:56Z

And you might want to include a timestamp column for this table although it's always current timestamp.

Because it's always current timestamp, so I think it's not necessary.

killme2008 · 2024-01-10T08:50:51Z

We need to make sure this table is only available in greptime catalog so it doesn't leak cluster information to tenants.

Good catch! Let me add a check.

…e catalog

killme2008 · 2024-01-10T09:13:56Z

We need to make sure this table is only available in greptime catalog so it doesn't leak cluster information to tenants.

Good catch! Let me add a check.

Ensured build_info and runtime_metrics were only available in greptime catalog, refer to cba6d8a. @sunng87

sunng87 · 2024-01-10T09:39:58Z

Wait, I still think a timestamp column can be helpful for:

Get the exact time of metrics value
It might be possible to render them on grafana if there is a timestamp column

killme2008 · 2024-01-10T10:06:48Z

Wait, I still think a timestamp column can be helpful for:

Get the exact time of metrics value

It might be possible to render them on grafana if there is a timestamp column

The second one convinced me. I'll add the timestamp column.

killme2008 · 2024-01-10T10:30:53Z

@sunng87 Added the timestamp column.

feat: adds runtime_metrics

367079a

github-actions bot added Doc not needed This PR do not impact docs Size: M labels Jan 10, 2024

fix: comment

0a20bbc

killme2008 marked this pull request as ready for review January 10, 2024 06:59

killme2008 requested review from waynexia, Taylor-lagrange and shuiyisong January 10, 2024 07:02

github-actions bot added Doc update required This change requires document update on https://github.com/GreptimeTeam/docs and removed Doc not needed This PR do not impact docs labels Jan 10, 2024

waynexia reviewed Jan 10, 2024

View reviewed changes

tests/cases/standalone/common/system/information_schema.result Outdated Show resolved Hide resolved

killme2008 mentioned this pull request Jan 10, 2024

information_schema improvements #2931

Open

37 tasks

waynexia added Doc update required This change requires document update on https://github.com/GreptimeTeam/docs and removed Doc update required This change requires document update on https://github.com/GreptimeTeam/docs labels Jan 10, 2024

sunng87 mentioned this pull request Jan 10, 2024

Update docs for feat: adds runtime_metrics GreptimeTeam/docs#777

Open

waynexia reviewed Jan 10, 2024

View reviewed changes

src/catalog/src/information_schema/runtime_metrics.rs Show resolved Hide resolved

waynexia approved these changes Jan 10, 2024

View reviewed changes

feat: refactor metrics table

9a40013

killme2008 requested a review from sunng87 January 10, 2024 08:42

chore: ensure build_info and runtime_metrics only avaiable in greptim…

cba6d8a

…e catalog

github-actions bot added Size: L and removed Size: M labels Jan 10, 2024

sunng87 approved these changes Jan 10, 2024

View reviewed changes

sunng87 added this pull request to the merge queue Jan 10, 2024

sunng87 removed this pull request from the merge queue due to a manual request Jan 10, 2024

feat: adds timestamp column

bb72a84

killme2008 added this pull request to the merge queue Jan 10, 2024

Merged via the queue into GreptimeTeam:main with commit fd3f23e Jan 10, 2024
14 checks passed

killme2008 deleted the feature/infomration-schema-tables-part4 branch January 10, 2024 11:03

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: adds runtime_metrics #3127

feat: adds runtime_metrics #3127

killme2008 commented Jan 10, 2024 •

edited

codecov bot commented Jan 10, 2024 •

edited

waynexia left a comment

sunng87 commented Jan 10, 2024

sunng87 commented Jan 10, 2024

killme2008 commented Jan 10, 2024

killme2008 commented Jan 10, 2024

killme2008 commented Jan 10, 2024

sunng87 commented Jan 10, 2024

killme2008 commented Jan 10, 2024

killme2008 commented Jan 10, 2024

feat: adds runtime_metrics #3127

feat: adds runtime_metrics #3127

Conversation

killme2008 commented Jan 10, 2024 • edited

What's changed and what's your intention?

Checklist

Refer to a related PR or issue link (optional)

codecov bot commented Jan 10, 2024 • edited

Codecov Report

waynexia left a comment

Choose a reason for hiding this comment

sunng87 commented Jan 10, 2024

sunng87 commented Jan 10, 2024

killme2008 commented Jan 10, 2024

killme2008 commented Jan 10, 2024

killme2008 commented Jan 10, 2024

sunng87 commented Jan 10, 2024

killme2008 commented Jan 10, 2024

killme2008 commented Jan 10, 2024

killme2008 commented Jan 10, 2024 •

edited

codecov bot commented Jan 10, 2024 •

edited