sql: support tableoid system column on virtual tables by rafiss · Pull Request #165727 · cockroachdb/cockroach

rafiss · 2026-03-13T19:21:00Z

Previously, the tableoid system column was only supported on regular
tables but not on virtual tables (pg_catalog, information_schema, etc.).
This caused pg_dump to fail when connecting to CockroachDB because
pg_dump queries SELECT x.tableoid, ... FROM pg_extension x ... as
one of its first introspection queries.

This commit adds tableoid support for virtual tables by:

Adding the tableoid column to optVirtualTable's column list in
the optimizer catalog, registered as a cat.System column (hidden
by default, like on regular tables).
Handling the tableoid column in constructVirtualScan by rendering
it as a constant value (the table's descriptor ID) after the virtual
table scan, since virtual table generators don't produce system
column values.
Including the tableoid system column in all indexes' numCols
(both primary and secondary) so the optimizer considers every index
as covering for tableoid queries. The Column() method on
optVirtualIndex is updated to return the tableoid column after all
stored columns for secondary indexes.

informs #20296

Release note (sql change): The tableoid system column is now
supported on virtual tables such as those in pg_catalog and
information_schema. This improves compatibility with PostgreSQL
tools like pg_dump that reference tableoid in their introspection
queries.

trunk-io · 2026-03-13T19:21:05Z

😎 Merged successfully - details.

blathers-crl · 2026-03-13T19:21:05Z

It looks like your PR touches production code but doesn't add or edit any test code. Did you consider adding tests to your PR?

_{🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf.}

cockroach-teamcity · 2026-03-13T19:21:17Z

This change is

github-actions · 2026-03-14T16:54:55Z

pkg/sql/opt_catalog.go

 		// Add 1, since the 0th index will the primary that we added above.
+		// Secondary indexes do not include the tableoid system column in
+		// numCols because the Column() method for secondary indexes uses a
+		// different offset scheme and would panic on out-of-bounds access.
 		ot.indexes[idx.Ordinal()] = optVirtualIndex{
 			tab:          ot,
 			idx:          idx,
 			indexOrdinal: idx.Ordinal(),
 			// The virtual indexes don't return the bogus PK key?
-			numCols: ot.ColumnCount(),
+			numCols: numNonSystemCols,


Potential bug: panic via IndexJoin on virtual tables.

Setting numCols: numNonSystemCols (excluding tableoid) on secondary indexes while the primary index includes tableoid creates a covering-check divergence. When a query like SELECT tableoid FROM pg_class WHERE oid = <value> is planned, the optimizer finds the secondary index on oid non-covering (tableoid is missing from IndexColumns()), generates an IndexJoinExpr, and execution reaches ConstructIndexJoin at pkg/sql/opt_exec_factory.go:681 which does table.(*optTable).desc — a hard type assertion that panics because virtual tables are *optVirtualTable, not *optTable.

Multiple pg_catalog tables (pg_class, pg_attribute, pg_constraint, pg_proc, etc.) have secondary indexes, making this user-triggerable.

Suggested fixes (pick one):

(A) Include tableoid in secondary index numCols too (set numCols: ot.ColumnCount()) and update the Column() method of optVirtualIndex to handle tableoid for secondary indexes.

(B) Add a virtual table guard in GenerateConstrainedScans (pkg/sql/opt/xform/select_funcs.go) to skip IndexJoin generation for virtual tables.

(C) Set NoIndexJoin = true on ScanPrivate.Flags when building virtual table scans.

Option B or C is simplest and safest since virtual tables have never supported index joins.

This seems like a valid concern - is it possible to have an index join into the primary index of the virtual table?

Yes, this was a real bug. With numCols: numNonSystemCols on secondary indexes, the optimizer saw tableoid as not covered by secondary indexes and could attempt an index join into the primary index, which fails for virtual tables.

I've fixed this by going with option (A) from the AI review: secondary indexes now include tableoid in their numCols (set to ot.ColumnCount()), and the Column() method on optVirtualIndex is updated to handle the tableoid column after all stored columns for secondary indexes. This way the optimizer sees all indexes as covering for tableoid, so it never considers an index join necessary.

I also added a regression test that queries tableoid from pg_class with a filter on oid which uses a secondary index.

Option (C) to set NoIndexJoin = true on virtual table ScanPrivate.Flags also would fix this. IIUC, virtual tables have never supported index joins and the generator always produces all columns, so this would be safe. But I didn't know if there would be unintended consequences from disabling index joins. That is a one-line change though, so if it's safe to make that change, let me know if you'd prefer that instead.

Option A sounds good to me.

github-actions · 2026-03-14T16:54:56Z

AI Review: Potential Issue(s) Detected

An inline comment has been added to the relevant line identifying a potential panic when tableoid is queried on virtual tables with secondary indexes.

Summary: This PR introduces a column count divergence between primary indexes (includes tableoid) and secondary indexes (excludes tableoid) on virtual tables. This can cause the optimizer to generate an IndexJoin for non-covering secondary index scans (e.g., SELECT tableoid FROM pg_class WHERE oid = <value>), which panics in ConstructIndexJoin at pkg/sql/opt_exec_factory.go:681 due to a hard type assertion table.(*optTable).desc failing on *optVirtualTable.

View full analysis

If helpful: add O-AI-Review-Real-Issue-Found label.
If not helpful: add O-AI-Review-Not-Helpful label.

yuzefovich

Nice! I only have the same question as the AI reviewer 😃

@yuzefovich reviewed 3 files and all commit messages, and made 3 comments.
Reviewable status: complete! 0 of 0 LGTMs obtained (waiting on rafiss and ZhouXing19).

pkg/sql/opt_catalog.go line 2608 at r1 (raw file):

		}

		// Add 1, since the 0th index will the primary that we added above.

nit: Add 1 ... sentence has been stale for a while now, we should remove it.

yuzefovich · 2026-03-14T19:03:09Z

pkg/sql/opt_catalog.go

 		// Add 1, since the 0th index will the primary that we added above.
+		// Secondary indexes do not include the tableoid system column in
+		// numCols because the Column() method for secondary indexes uses a
+		// different offset scheme and would panic on out-of-bounds access.
 		ot.indexes[idx.Ordinal()] = optVirtualIndex{
 			tab:          ot,
 			idx:          idx,
 			indexOrdinal: idx.Ordinal(),
 			// The virtual indexes don't return the bogus PK key?
-			numCols: ot.ColumnCount(),
+			numCols: numNonSystemCols,


This seems like a valid concern - is it possible to have an index join into the primary index of the virtual table?

Previously, the `tableoid` system column was only supported on regular tables but not on virtual tables (pg_catalog, information_schema, etc.). This caused `pg_dump` to fail when connecting to CockroachDB because pg_dump queries `SELECT x.tableoid, ... FROM pg_extension x ...` as one of its first introspection queries. This commit adds `tableoid` support for virtual tables by: 1. Adding the `tableoid` column to `optVirtualTable`'s column list in the optimizer catalog, registered as a `cat.System` column (hidden by default, like on regular tables). 2. Handling the `tableoid` column in `constructVirtualScan` by rendering it as a constant value (the table's descriptor ID) after the virtual table scan, since virtual table generators don't produce system column values. 3. Including the `tableoid` system column in all indexes' `numCols` (both primary and secondary) so the optimizer considers every index as covering for tableoid queries. The `Column()` method on `optVirtualIndex` is updated to return the tableoid column after all stored columns for secondary indexes. Release note (sql change): The `tableoid` system column is now supported on virtual tables such as those in `pg_catalog` and `information_schema`. This improves compatibility with PostgreSQL tools like `pg_dump` that reference `tableoid` in their introspection queries. Co-Authored-By: roachdev-claude <roachdev-claude-bot@cockroachlabs.com>

rafiss

@rafiss resolved 1 discussion.
Reviewable status: complete! 0 of 0 LGTMs obtained (waiting on yuzefovich and ZhouXing19).

yuzefovich

Nice!

@yuzefovich reviewed 3 files and all commit messages, and made 2 comments.
Reviewable status: complete! 1 of 0 LGTMs obtained (waiting on rafiss and ZhouXing19).

yuzefovich · 2026-03-15T18:46:09Z

pkg/sql/opt_catalog.go

 		// Add 1, since the 0th index will the primary that we added above.
+		// Secondary indexes do not include the tableoid system column in
+		// numCols because the Column() method for secondary indexes uses a
+		// different offset scheme and would panic on out-of-bounds access.
 		ot.indexes[idx.Ordinal()] = optVirtualIndex{
 			tab:          ot,
 			idx:          idx,
 			indexOrdinal: idx.Ordinal(),
 			// The virtual indexes don't return the bogus PK key?
-			numCols: ot.ColumnCount(),
+			numCols: numNonSystemCols,


Option A sounds good to me.

When pg_dump queries pg_attrdef with a join like: SELECT a.tableoid, ... FROM unnest('{...}'::oid[]) AS src(tbloid) JOIN pg_catalog.pg_attrdef a ON (src.tbloid = a.adrelid) the virtual table lookup join panics with "index out of range [5] with length 5". This happens because pushRow iterates lookupCols which includes the tableoid system column ordinal, but the looked-up row from the virtual table generator only contains public columns. Fix this by detecting the tableoid ordinal in pushRow and producing its value as a constant from the table descriptor, matching how constructVirtualScan handles tableoid for regular virtual table scans. Informs: cockroachdb#20296 Release note: None Co-Authored-By: roachdev-claude <roachdev-claude-bot@cockroachlabs.com>

rafiss · 2026-03-16T15:06:27Z

TFTR!

/trunk merge

rafiss force-pushed the vtable-tableoid branch from 699feeb to a1dd3e8 Compare March 14, 2026 16:27

rafiss marked this pull request as ready for review March 14, 2026 16:28

rafiss requested a review from a team as a code owner March 14, 2026 16:28

rafiss requested review from ZhouXing19 and yuzefovich and removed request for a team March 14, 2026 16:28

github-actions bot reviewed Mar 14, 2026

View reviewed changes

github-actions bot added the o-AI-Review-Potential-Issue-Detected AI reviewer found potential issue. Never assign manually—auto-applied by GH action only. label Mar 14, 2026

yuzefovich reviewed Mar 14, 2026

View reviewed changes

rafiss added the O-AI-Review-Real-Issue-Found AI reviewer found real issue label Mar 15, 2026

rafiss force-pushed the vtable-tableoid branch from a1dd3e8 to 5937e4b Compare March 15, 2026 06:04

rafiss commented Mar 15, 2026

View reviewed changes

rafiss force-pushed the vtable-tableoid branch from 5937e4b to 3a7b268 Compare March 15, 2026 06:06

yuzefovich approved these changes Mar 15, 2026

View reviewed changes

rafiss force-pushed the vtable-tableoid branch from a7424c6 to 29e8237 Compare March 16, 2026 15:06

trunk-io bot merged commit 00a1bd9 into cockroachdb:master Mar 16, 2026
26 checks passed

celeste-cockroachdb bot added the target-release-26.2.0 label Mar 16, 2026

rafiss deleted the vtable-tableoid branch March 18, 2026 01:34

Conversation

rafiss commented Mar 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

trunk-io bot commented Mar 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

blathers-crl bot commented Mar 13, 2026

Uh oh!

cockroach-teamcity commented Mar 13, 2026

Uh oh!

github-actions bot Mar 14, 2026

Choose a reason for hiding this comment

Uh oh!

yuzefovich Mar 14, 2026

Choose a reason for hiding this comment

Uh oh!

rafiss Mar 15, 2026

Choose a reason for hiding this comment

Uh oh!

yuzefovich Mar 15, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Mar 14, 2026

AI Review: Potential Issue(s) Detected

Uh oh!

yuzefovich left a comment

Choose a reason for hiding this comment

Uh oh!

yuzefovich Mar 14, 2026

Choose a reason for hiding this comment

Uh oh!

rafiss left a comment

Choose a reason for hiding this comment

Uh oh!

yuzefovich left a comment

Choose a reason for hiding this comment

Uh oh!

yuzefovich Mar 15, 2026

Choose a reason for hiding this comment

Uh oh!

rafiss commented Mar 16, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

rafiss commented Mar 13, 2026 •

edited

Loading

trunk-io bot commented Mar 13, 2026 •

edited

Loading