Skip to content

2.27.0.0-b305

2.27.0.0-b305: [#27875] YSQL: add YB test GUC yb_test_preload_catalog_tables
@myang2021 myang2021 tagged this 04 Jul 23:22 
Summary:
Currently, if we see DDL statements execution and suspect some sudden PG memory
spike across active connections, it would be nice to know what are the
catalog/rel cache memory footprint. If either the gflag
`--ysql_catalog_preload_additional_tables` or
`--ysql_catalog_preload_additional_table_list` is enabled, we simply need to
start a new connection and then execute some yb functions to find out the memory
consumption:
```
select yb_mem_usage();
select yb_mem_usage_sql_kb();
```

However by default these two gflags are false, in this case a new connection
does not do the same catalog preloading as would happen after a DDL statement.

In this diff, I added a new GUC `yb_test_preload_catalog_tables` such that if
enabled, we always perform a full catalog refresh before executing each
top-level statement.

A new unit test is added.
Jira: DB-17482

Test Plan:
1.
./yb_build.sh release --cxx-test pg_catalog_version-test --gtest_filter PgCatalogVersionTest.TestPreloadCatalogTables

2. Manual test

(1) start a RF-1 local dev cluster
./bin/yb-ctl create --rf 1

(2) run the following ysql session

(2.1) first connection after cluster creation
```
ysqlsh (15.12-YB-2.27.0.0-b0)
Type "help" for help.

yugabyte=# select yb_mem_usage();
           yb_mem_usage
----------------------------------
 Session memory usage = 80644 kbs
(1 row)

yugabyte=# select yb_mem_usage_sql_kb();
 yb_mem_usage_sql_kb
---------------------
               14091
(1 row)

yugabyte=# \q
```

(2.2) second connection after cluster creation.
```
ysqlsh (15.12-YB-2.27.0.0-b0)
Type "help" for help.

yugabyte=# select yb_mem_usage();
           yb_mem_usage
----------------------------------
 Session memory usage = 50268 kbs
(1 row)

yugabyte=# select yb_mem_usage_sql_kb();
 yb_mem_usage_sql_kb
---------------------
                2034
(1 row)

yugabyte=# set yb_test_preload_catalog_tables = true;
SET
yugabyte=# select yb_mem_usage();
           yb_mem_usage
----------------------------------
 Session memory usage = 83024 kbs
(1 row)

yugabyte=# select yb_mem_usage_sql_kb();
 yb_mem_usage_sql_kb
---------------------
               14013
(1 row)

```

We can see that after `set yb_test_preload_catalog_tables = true;`, both functions
returned significantly larger values, reflecting the effect of preloading catalog tables.

Reviewers: kfranz, sanketh, mihnea, #db-approvers

Reviewed By: kfranz, #db-approvers

Subscribers: yql

Differential Revision: https://phorge.dev.yugabyte.com/D45213
Assets 2
Loading