Summary:
This diff starts to use the two new function added in part 1:
* yb_increment_db_catalog_version_with_inval_messages
* yb_increment_all_db_catalog_versions_with_inval_messages
A new gflag `--TEST_yb_enable_invalidation_messages` is introduced to enable the
new code paths that perform incremental catalog cache invalidation. When a DDL
statement is about to increment the catalog version, it also calls
`YbGetSubGroupInvalMessages` which is a adapted from the PG function
`xactGetCommittedInvalidationMessages` to return the array of invalidation
messages that are about to be sent out if the DDL transaction is committed.
Note that we do this right before the DDL transaction is about to commit so the
array of invalidation messages are no different from those that would get
actually sent out if the DDL transaction does commit successfully.
The array of invalidation messages are simply sent as binary byte array and is
inserted into `pg_yb_invalidation_messages` table. A new gflag
`--TEST_yb_invalidation_message_expiration_secs` is used to control the expiration
time of the array of messages. A next invocation of one of these two new
functions is going to delete those old messages that are older than
`--TEST_yb_invalidation_message_expiration_secs`.
A third gflag `--TEST_yb_max_num_invalidation_messages` (default value is 4096)
is used to turn off incremental catalog cache invalidation if the DDL statement
generates too many invalidation messages. We do not want to insert unbounded
number of invalidation messages into `pg_yb_invalidation_messages` as the entire
table would be sent as part of tserver/master heartbeat responses that cost
network bandwidth and also may subject to certain RPC limits.
The three new gflags are added as test gflags, the plan is to change them to
normal gflags once the new feature stabilizes.
To avoid updating too many unit test logs, I added
`yb_is_calling_internal_function_for_ddl` to hide the SQL statements executed by
`yb_increment_db_catalog_version_with_inval_messages` or
`yb_increment_all_db_catalog_versions_with_inval_messages`, and also applied to
the existing function `yb_increment_all_db_catalog_versions_with` which are
meant to be internal functions but because they are implemented as plain SQL
functions they would be treated as such and could cause various test log changes
(e.g., stats numbers).
A new sanity unit test is added that shows invalidation messages are both
portable and stable. A second new unit test checks for multiple ddls embedded
in a BEGIN block, we get expected number of invalidation messages for each
DDL.
Test Plan:
(1) Run these tests with both `--TEST_yb_enable_invalidation_messages=true` and
`--TEST_yb_enable_invalidation_messages=false`
./yb_build.sh --cxx-test pg_catalog_version-test
./yb_build.sh --java-test 'org.yb.pgsql.TestYsqlUpgrade'
./yb_build.sh --java-test org.yb.pgsql.TestPgRegressMisc#testPgRegressMiscIndependent
Note that the test log `yb_catalog_version.out` differs due to catalog version
numbers shift. There is a special case where a DDL statement did not generate
any invalidation messages under the new feature and therefore the catalog
version is no longer incremented. With the new feature off, the current code
thought the DDL has made some writes to the PG catalog and therefore needs to
increment catalog version. In fact the DDL made some writes to a user defined
table (via a trigger) instead of any catalog tables. Had we been able to detect
that we only needed to increment catalog verion when the DDL writes to some
catalog table, we would have avoided the catalog version increment. In the diff
I have left the test output `yb_catalog_version.out` that is produced under
`--TEST_yb_enable_invalidation_messages=false`. The test log needs update when
the gflag is turned on by default in the future. This test log is the only one that
differs in the entire jenkins suite when the `--TEST_yb_enable_invalidation_messages`
is flipped.
(2)
Manual sanity test to verify the invalidation messages are stored in
pg_yb_invalidation_messages and is associated with the new catalog version of DB
13515 (yugabyte).
```
yugabyte=# create table foo(id int);
CREATE TABLE
yugabyte=# select * from pg_yb_catalog_version;
db_oid | current_version | last_breaking_version
--------+-----------------+-----------------------
1 | 1 | 1
4 | 1 | 1
5 | 1 | 1
13515 | 1 | 1
13516 | 1 | 1
(5 rows)
yugabyte=# select * from pg_yb_invalidation_messages;
db_oid | current_version | message_time | messages
--------+-----------------+--------------+----------
(0 rows)
yugabyte=# drop table foo;
DROP TABLE
yugabyte=# select * from pg_yb_catalog_version;
db_oid | current_version | last_breaking_version
--------+-----------------+-----------------------
1 | 1 | 1
4 | 1 | 1
5 | 1 | 1
13515 | 2 | 1
13516 | 1 | 1
(5 rows)
yugabyte=# select * from pg_yb_invalidation_messages;
db_oid | current_version | message_time |
messages
--------+-----------------+--------------+--------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------------------------------------------------
----------------------------------------
13515 | 2 | 1738969425 | \x50270000de00290001270000cb34000040c1eb0a000000004f270000de00290001270000cb3400005ac4b8530
000000050270000de00290001270000cb34000047a2537b000000004f270000de00290001270000cb34000021e2d2ca0000000007270000de00290001270000cb34000
0849f9c130000000006270000de00290001270000cb34000002517d240000000007270000de00290001270000cb340000d519492c0000000006270000de00290001270
000cb340000532bd64f0000000007270000de00290001270000cb3400000ce84cf30000000006270000de00290001270000cb340000f1f7a7e80000000007270000de0
0290001270000cb340000ecbba9650000000006270000de00290001270000cb34000084a01e300000000007270000de00290001270000cb3400003f53cbc6000000000
6270000de00290001270000cb3400001310debc0000000007270000de00290001270000cb340000c76e67a20000000006270000de00290001270000cb340000f3cf9e8
c0000000007270000de00290001270000cb34000017e0201d0000000006270000de00290001270000cb3400004ba32d1e0000000037270000de00290001270000cb340
000465708530000000036270000de00290001270000cb34000021e2d2ca00000000fb000000de0029000681c2d8cb340000300a0000e7550000fb000000de002900068
1c2d8cb340000300a0000e7550000fe000000de00290003a28bffcb3400000040000001000000fb000000de0029000681c2d8cb340000300a0000e7550000fb000000d
e0029000680c2d800000000be040000e7550000
(1 row)
```
Reviewers: kfranz, sanketh, mihnea
Reviewed By: kfranz
Subscribers: ybase, yql
Differential Revision: https://phorge.dev.yugabyte.com/D41799
