feat: add system table system.memory_statistics

[dantengsky]

I hereby agree to the terms of the CLA available at: https://databend.rs/dev/policies/cla/

Summary

two system tables have been added, what are intended to be used by developers

• system.malloc_stats_totals which shows the basic statistics of jemalloc

mysql> desc system.malloc_stats_totals;
+-------+-----------------+------+---------+-------+
| Field | Type            | Null | Default | Extra |
+-------+-----------------+------+---------+-------+
| name  | VARCHAR         | NO   |         |       |
| value | BIGINT UNSIGNED | NO   | 0       |       |
+-------+-----------------+------+---------+-------+
2 rows in set (0.02 sec)
Read 0 rows, 0.00 B in 0.003 sec., 0 rows/sec., 0.00 B/sec.

mysql> select * from system.malloc_stats_totals;
+------------------+-----------+
| name             | value     |
+------------------+-----------+
| stats.active     |  41484288 |
| stats.allocated  |  36138184 |
| stats.retained   | 123314176 |
| stats.mapped     | 195452928 |
| stats.resident   |  64479232 |
| stats.metadata   |  15329880 |
+------------------+-----------+
6 rows in set (0.03 sec)
Read 6 rows, 191.00 B in 0.007 sec., 824.49 rows/sec., 25.63 KiB/sec.

• system.malloc_stats which shows the statistics of jemalloc in details

which basically is the jemalloc's print_stats, with constants, mutex_statistics and per_arena stats disabled.

mysql> desc system.malloc_stats;
+------------+--------+------+---------+-------+
| Field      | Type   | Null | Default | Extra |
+------------+--------+------+---------+-------+
| statistics | OBJECT | NO   | {}      |       |
+------------+--------+------+---------+-------+
1 row in set (0.03 sec)
Read 0 rows, 0.00 B in 0.003 sec., 0 rows/sec., 0.00 B/sec.

grab the snippets that are interested in by using json path syntax:

mysql> select statistics:jemalloc.stats from system.malloc_stats;
+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| statistics:jemalloc.stats                                                                                                                                                                                     |
+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| {"allocated":33210056,"active":40620032,"metadata":17515608,"metadata_thp":0,"resident":67706880,"mapped":196505600,"retained":124882944,"background_thread":{"num_threads":0,"num_runs":0,"run_interval":0}} |
+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.05 sec)
Read 1 rows, 188.18 KiB in 0.026 sec., 39.08 rows/sec., 7.18 MiB/sec.

mysql> select statistics:jemalloc.stats.allocated from system.malloc_stats;
+-------------------------------------+
| statistics:jemalloc.stats.allocated |
+-------------------------------------+
| 36358504                            |
+-------------------------------------+
1 row in set (0.06 sec)
Read 1 rows, 188.18 KiB in 0.025 sec., 39.23 rows/sec., 7.21 MiB/sec.

mysql> select statistics:jemalloc['stats.arenas'].merged.large from system.malloc_stats;
+-----------------------------------------------------------------------------------------------+
| statistics:jemalloc['stats.arenas'].merged.large                                              |
+-----------------------------------------------------------------------------------------------+
| {"allocated":27463680,"nmalloc":856,"ndalloc":730,"nrequests":1158,"nfills":856,"nflushes":0} |
+-----------------------------------------------------------------------------------------------+
1 row in set (0.06 sec)
Read 1 rows, 188.18 KiB in 0.032 sec., 31.21 rows/sec., 5.73 MiB/sec.

Closes #8942

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Ignored Deployment

Name	Status	Preview	Updated
databend	⬜️ Ignored (Inspect)		Nov 24, 2022 at 2:36PM (UTC)

[dantengsky]

converting to draft , adding more stats