Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

infoschema/metric_schema: add node cpu/mem/net load metric tables #14670

Merged
merged 13 commits into from
Feb 11, 2020
Merged

infoschema/metric_schema: add node cpu/mem/net load metric tables #14670

merged 13 commits into from
Feb 11, 2020

Conversation

crazycs520
Copy link
Contributor

What problem does this PR solve?

add node cpu/mem/net load metric tables.

eg:

> select * from `METRIC_SUMMARY` where metric_name like "node%";
+-------------------------------------+----------------------------+-----------------------------+-----------------------------+-----------------------------+-----------------------------+
| METRIC_NAME                         | TIME                       | SUM_VALUE                   | AVG_VALUE                   | MIN_VALUE                   | MAX_VALUE                   |
+-------------------------------------+----------------------------+-----------------------------+-----------------------------+-----------------------------+-----------------------------+
| node_kernel_context_switches        | 2020-02-07 14:57:05.871000 |      655519.545634          |       65551.9545634         |       64829.2               |       66990.0666667         |
| node_kernel_interrupts              | 2020-02-07 14:57:05.871000 |      285098.561633          |       28509.8561633         |       24890.6               |       36146.0921354         |
| node_disk_io_util                   | 2020-02-07 14:57:05.871000 |           0.0061111293834   |           0.000152778234585 |           0.0               |           0.000822222222471 |
| node_cpu_usage                      | 2020-02-07 14:57:05.871000 |         983.351640034       |          12.2918955004      |           0.0               |          95.0610013553      |
| node_virtual_cpus                   | 2020-02-07 14:57:05.871000 |         400.0               |          40.0               |          40.0               |          40.0               |
| node_disk_throughput                | 2020-02-07 14:57:05.871000 |     4568739.61143           |      114218.490286          |           0.0               |      451955.463698          |
| node_total_memory_swap              | 2020-02-07 14:57:05.871000 |           0.0               |           0.0               |           0.0               |           0.0               |
| node_memory_buffers                 | 2020-02-07 14:57:05.871000 |  8409874432.0               |   840987443.2               |   840986624.0               |   840990720.0               |
| node_memory_dirty                   | 2020-02-07 14:57:05.871000 |     1277952.0               |      127795.2               |       65536.0               |      217088.0               |
| node_network_out_packets            | 2020-02-07 14:57:05.871000 |        5068.3477909         |          20.2733911636      |           0.0               |         309.748672252       |
| node_file_descriptor_allocated      | 2020-02-07 14:57:05.871000 |       85440.0               |        8544.0               |        8400.0               |        8800.0               |
| node_load15                         | 2020-02-07 14:57:05.871000 |          16.02              |           1.602             |           1.31              |           2.02              |
| node_disk_available_size            | 2020-02-07 14:57:05.871000 |           1.02136396513e+13 |           1.45909137876e+11 |      305152.0               |           4.37057007616e+11 |
| node_memory_active                  | 2020-02-07 14:57:05.871000 |           5.00219334656e+11 | 50021933465.6               | 49960075264.0               | 50104033280.0               |
| node_segments_retransmitted         | 2020-02-07 14:57:05.871000 |           0.244446419863    |           0.0244446419863   |           0.0               |           0.155559012422    |
| node_uptime                         | 2020-02-07 14:57:05.871000 |   192730650.591             |    19273065.0591            |    19272795.0582            |    19273335.0592            |
| node_disk_write_latency             | 2020-02-07 14:57:05.871000 |           0.00145512490499  |           7.27562452495e-05 |           0.0               |           0.000188208616744 |
| node_load1                          | 2020-02-07 14:57:05.871000 |          24.92              |           2.492             |           1.18              |           5.19              |
| node_memory_writeback_tmp           | 2020-02-07 14:57:05.871000 |           0.0               |           0.0               |           0.0               |           0.0               |
| node_filesystem_space_used          | 2020-02-07 14:57:05.871000 |        3347.76157822        |          47.8251654031      |           0.0               |          99.8465775993      |
| node_memory_available               | 2020-02-07 14:57:05.871000 |           1.09862304563e+12 |           1.09862304563e+11 |           1.097805824e+11   |           1.09930532864e+11 |
| node_kernel_forks                   | 2020-02-07 14:57:05.871000 |          27.377811365       |           2.7377811365      |           2.39994666785     |           3.93324592787     |
| node_network_utilization_out_hourly | 2020-02-07 14:57:05.871000 | 17404865994.3               |  1740486599.43              |  1740248889.04              |  1741354967.99              |
| node_tcp_in_use                     | 2020-02-07 14:57:05.871000 |         586.0               |          58.6               |          58.0               |          60.0               |
| node_network_out_errors             | 2020-02-07 14:57:05.871000 |           0.0               |           0.0               |           0.0               |           0.0               |
| node_process_open_fd_count          | 2020-02-07 14:57:05.871000 |        4451.0               |         148.366666667       |           7.0               |         428.0               |
| node_disk_size                      | 2020-02-07 14:57:05.871000 |           3.6101607168e+13  |           5.15737245257e+11 |   198896640.0               |           1.57323309875e+12 |
| node_network_utilization_in_hourly  | 2020-02-07 14:57:05.871000 | 13309315966.2               |  1330931596.62              |  1330581288.7               |  1332042573.38              |
| node_network_in_errors              | 2020-02-07 14:57:05.871000 |           0.0               |           0.0               |           0.0               |           0.0               |
| node_disk_state                     | 2020-02-07 14:57:05.871000 |           0.0               |           0.0               |           0.0               |           0.0               |
| node_processes_blocked              | 2020-02-07 14:57:05.871000 |           0.0               |           0.0               |           0.0               |           0.0               |
| node_memory_writeback               | 2020-02-07 14:57:05.871000 |           0.0               |           0.0               |           0.0               |           0.0               |
| node_network_in_packets             | 2020-02-07 14:57:05.871000 |        4958.83630643        |          19.8353452257      |           0.0               |         305.526543855       |
| node_disk_read_latency              | 2020-02-07 14:57:05.871000 | <null>                      | <null>                      | <null>                      | <null>                      |
| node_tcp_connections                | 2020-02-07 14:57:05.871000 |        1121.0               |         112.1               |         107.0               |         114.0               |
| node_network_out_drops              | 2020-02-07 14:57:05.871000 |           0.0               |           0.0               |           0.0               |           0.0               |
| node_network_interface_speed        | 2020-02-07 14:57:05.871000 |       50000.0               |         200.0               |           0.0               |        1000.0               |
| node_memory_cached                  | 2020-02-07 14:57:05.871000 |           8.79347232768e+11 | 87934723276.8               | 86624690176.0               | 89831473152.0               |
| node_load5                          | 2020-02-07 14:57:05.871000 |          20.23              |           2.023             |           1.42              |           2.92              |
| node_network_in_drops               | 2020-02-07 14:57:05.871000 |           0.0               |           0.0               |           0.0               |           0.0               |
| node_memory_mapped                  | 2020-02-07 14:57:05.871000 |  4651339776.0               |   465133977.6               |   463855616.0               |   468094976.0               |
| node_total_memory                   | 2020-02-07 14:57:05.871000 |           1.34905430016e+12 |           1.34905430016e+11 |           1.34905430016e+11 |           1.34905430016e+11 |
| node_memory_shared                  | 2020-02-07 14:57:05.871000 | 44279304192.0               |  4427930419.2               |  4426252288.0               |  4434640896.0               |
| node_memory_free                    | 2020-02-07 14:57:05.871000 |           2.36710756352e+11 | 23671075635.2               | 21735051264.0               | 24982966272.0               |
| node_memory_inactive                | 2020-02-07 14:57:05.871000 |           5.6814526464e+11  | 56814526464.0               | 55505920000.0               | 58710208512.0               |
| node_disk_iops                      | 2020-02-07 14:57:05.871000 |         223.77829734        |           5.5944574335      |           0.0               |          21.2449165537      |
| node_network_out_traffic            | 2020-02-07 14:57:05.871000 |     4852941.17995           |       19411.7647198         |           0.0               |      204910.022222          |
| node_processes_running              | 2020-02-07 14:57:05.871000 |         234.0               |          23.4               |           9.0               |          61.0               |
| node_network_in_traffic             | 2020-02-07 14:57:05.871000 |     3715865.43449           |       14863.4617379         |           0.0               |      205071.888889          |
+-------------------------------------+----------------------------+-----------------------------+-----------------------------+-----------------------------+-----------------------------+

What is changed and how it works?

Check List

Tests

  • Unit test
  • Manual test (add detailed scripts or steps below)

@crazycs520
add comment
f91d1f5 
Signed-off-by: crazycs <crazycs520@gmail.com>
@crazycs520
Merge branch 'master' into metric-node
7eb96d6
@crazycs520
Copy link
Contributor Author

/run-unit-test

4 similar comments
@crazycs520
Copy link
Contributor Author

/run-unit-test

@crazycs520
Copy link
Contributor Author

/run-unit-test

@crazycs520
Copy link
Contributor Author

/run-unit-test

@crazycs520
Copy link
Contributor Author

/run-unit-test

@crazycs520 crazycs520 changed the title iinfoschema/metric_schema: add node cpu/mem/net load metric tables infoschema/metric_schema: add node cpu/mem/net load metric tables Feb 10, 2020
@crazycs520
Copy link
Contributor Author

/rebuild

lonng
lonng reviewed Feb 11, 2020
View reviewed changes
Copy link
Contributor

@lonng lonng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some metric table does not contain a comment, is that expected?

@crazycs520
Copy link
Contributor Author

@lonng, Some metric table name is easy to understand, so they don't have a comment.

Deardrops
Deardrops reviewed Feb 11, 2020
View reviewed changes
infoschema/metric_table_def.go Outdated Show resolved Hide resolved
infoschema/metric_table_def.go Outdated Show resolved Hide resolved
infoschema/metric_table_def.go Outdated Show resolved Hide resolved
infoschema/metric_table_def.go Outdated Show resolved Hide resolved
infoschema/metric_table_def.go
"node_network_interface_speed": {
PromQL: `node_network_transmit_queue_length{$LABEL_CONDITIONS}`,
Labels: []string{"device", "instance"},
Comment: "node_network_transmit_queue_length = transmit_queue_length value of /sys/class/net/<iface>.",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are node_network_interface_speed and node_network_transmit_queue_length same thing?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comment was from tidb-ansible.
https://github.com/pingcap/tidb-ansible/blob/master/scripts/node.json#L7477

crazycs520 and others added 5 commits February 11, 2020 13:11
@crazycs520 @Deardrops
Update infoschema/metric_table_def.go
f87124b 
Co-Authored-By: Maxwell <passagesfall@gmail.com>
@crazycs520 @Deardrops
Update infoschema/metric_table_def.go
618c7dc 
Co-Authored-By: Maxwell <passagesfall@gmail.com>
@crazycs520 @Deardrops
Update infoschema/metric_table_def.go
bb6d00c 
Co-Authored-By: Maxwell <passagesfall@gmail.com>
@crazycs520 @Deardrops
Update infoschema/metric_table_def.go
ac32ac1 
Co-Authored-By: Maxwell <passagesfall@gmail.com>
@crazycs520
Merge branch 'master' into metric-node
00285d2
lonng
lonng reviewed Feb 11, 2020
View reviewed changes
Copy link
Contributor

@lonng lonng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Deardrops
Deardrops previously approved these changes Feb 11, 2020
View reviewed changes
Copy link
Contributor

@Deardrops Deardrops left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@crazycs520
Copy link
Contributor Author

/run-all-tests

@crazycs520
fix name
8ec7407 
Signed-off-by: crazycs <crazycs520@gmail.com>
@crazycs520 crazycs520 added status/all tests passed status/LGT2 Indicates that a PR has LGTM 2. labels Feb 11, 2020
Deardrops
Deardrops approved these changes Feb 11, 2020
View reviewed changes
Copy link
Contributor

@Deardrops Deardrops left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@crazycs520 crazycs520 merged commit 931b477 into pingcap:master Feb 11, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@lonng lonng lonng left review comments

@Deardrops Deardrops Deardrops approved these changes

Assignees
No one assigned
Labels
status/LGT2 Indicates that a PR has LGTM 2. type/usability
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@crazycs520 @lonng @Deardrops