Skip to content

2.25.0.0-b13

2.25.0.0-b13: [#23882] YSQL: Improve cache re-invalidation for alter table commands
@fizaaluthra fizaaluthra tagged this 19 Sep 18:42 
Summary:
Background:

When DDL atomicity is enabled, DDL transaction verification may lead to additional schema version bumps
on an altered table.

This can create a scenario where if an ALTER TABLE operation increments the table’s schema version,
and subsequently performs a scan on it, any following DMLs on the same table within the same session
may encounter a schema version mismatch error.
This happens because after the YB alter invalidates the table cache entry, the table scan reloads it.
When DDL transaction verification bumps the schema version of the table again,
the previously reloaded table cache entry becomes invalid, and would need to be reloaded again.

Commit 53477ae0f02a333542ab0e857310d93f24b85f96 introduced a re-invalidation mechanism
to solve this problem.

This diff makes some changes to the re-invalidation mechanism:

  - Instead of using the YB alter table handles to keep track of the affected tables, simply use the table oids. Although the usage of statement handles doesn't seem to cause any issues on YB master, it causes issues on YB PG15 as upstream PG has changed the flow of some alter table commands. Specifically, the YB PG memory context (`YBCPgMemctx`) where the statement handles are allocated may be freed in an earlier catch block than the one that executes `YbATInvalidateTableCacheAfterAlter`.

  - In `YbATInvalidateTableCacheAfterAlter`, we now check if the relation still exists as we have to retrieve its database oid and relfilenode oid. This is necessary because legacy rewrite operations that alter the relation's oid might have dropped the old relation. If the relation has been dropped, there's no need to invalidate cache entries, as any queries referencing the dropped relation will fail anyway.

  - Commit 53477ae0f02a333542ab0e857310d93f24b85f96 added an optimization to skip schema version increments for alter type without rewrite. However, this approach may be flawed because YB currently drops and recreates dependent indexes when altering a column type, even when no rewrite occurs. This issue is tracked under #24007. For now, revert the changes to skip the schema version increment on the base table, so that we correctly track the relation as altered and execute the re-invalidation mechanism. Also remove the now unused variables 'rewriteState' and 'rewrite' from `YBCPrepareAlterTable` and `YBCPrepareAlterTableCmd`.

    - Add a function in ybc_pggate to invalidate the table cache entry for a given database oid and relfilenode oid.
Jira: DB-12786

Test Plan: ./yb_build.sh --cxx-test pgwrapper_pg_ddl_atomicity-test --gtest_filter PgDdlAtomicityTest.TestTableCacheAfterTxnVerification

Reviewers: myang

Reviewed By: myang

Subscribers: yql

Differential Revision: https://phorge.dev.yugabyte.com/D38012
Assets 2
Loading