Skip to content

2.25.1.0-b81

2.25.1.0-b81: [#24320] YSQL: Fix "Duplicate table" error caused by OID collision
@myang2021 myang2021 tagged this 23 Dec 23:45 
Summary:
The symptom of this bug looks like
```
./10.9.129.124/yb-master.ip-10-9-129-124.us-west-2.compute.internal.yugabyte.log.INFO.20241201-032724.27006:W1201 03:55:26.178005 53035 catalog_manager.cc:3985] Found table: pc_index_001 [id=00004002000030008000000000004025, state=RUNNING]. Failed creating table with error: Already present (yb/master/catalog_manager.cc:3975): Object with id 00004002000030008000000000004025 ('pc_db_001.pc_index_001') already exists Request:
```

Following is the list of events that caused this error:

1. OID 16421 was allocated in the source database 16384
```
./10.9.129.124/postgresql-2024-12-01_032746.log:I1201 03:31:32.408409 33065 varsup.c:714] allocated new object id 16421 in database 16384
```

2. A backup is done for the source database 16384

3. A restore is done in target database 16386

In the restore SQL script, 16421 was forced to be used for a given PG relation
```
SELECT pg_catalog.binary_upgrade_set_next_index_relfilenode('16421'::pg_catalog.oid);
```

The PG log:
```
./10.9.129.124/yb-tserver.ip-10-9-129-124.us-west-2.compute.internal.yugabyte.log.INFO.20241201-032745.28028:I1201 03:46:07.034967 28342 pg_create_table.cc:99] PgCreateTable: creating a transactional index: pc_db_001.pc_index_001/{ database_oid: 16386 object_oid: 16421 }
```

The corresponding master log: (note that 0x4025 == 16421)

```
I1201 03:46:07.042412 27229 catalog_manager.cc:4290] Successfully created index pc_index_001 [id=00004002000030008000000000004025] in pc_db_001 [id=00004002000030008000000000000000] per request from 10.9.129.124:41483
```

4. Later in the target database 16386, 16421 is allocated during a table rewrite operation (reindex)
```
./10.9.129.124/postgresql-2024-12-01_032746.log:I1201 03:55:26.177358 47008 varsup.c:714] allocated new object id 16421 in database 16386
```

16421 is used to create another index
```
./10.9.129.124/yb-tserver.ip-10-9-129-124.us-west-2.compute.internal.yugabyte.log.INFO.20241201-032745.28028:I1201 03:55:26.177520 53217 pg_create_table.cc:99] PgCreateTable: creating a transactional index: pc_db_001.pc_partial_index_001/{ database_oid: 16386 object_oid: 16421 }
```

5. the corresponding master log:

```
./10.9.129.124/yb-master.ip-10-9-129-124.us-west-2.compute.internal.yugabyte.log.INFO.20241201-032724.27006:W1201 03:55:26.178005 53035 catalog_manager.cc:3985] Found table: pc_index_001 [id=00004002000030008000000000004025, state=RUNNING]. Failed creating table with error: Already present (yb/master/catalog_manager.cc:3975): Object with id 00004002000030008000000000004025 ('pc_db_001.pc_index_001') already exists Request:
```

The table id `00004002000030008000000000004025` is already used by
`pc_db_001.pc_index_001`. We are trying to reuse it for
`pc_db_001.pc_partial_index_001`, which results in the
`Already present` error, which showed up as `Duplicate table` error at PG side.

Analysis: the OID 16421 is allocated by the OID allocator 16386. Normally when
an OID is used as a table OID, PG will check `pg_class` to see whether the
OID already exists and skip it if it does. However if the OID is used as
`relfilenode` then PG does not check pg_class for OID collision, instead it only
calls `DoesRelFileExist` to see if there is an existing file of the same name
for this relation. In PG a relation has an on-disk file. But in YB a relation
does not have an on-disk file (except for a temporary relation). As a result the
call to `DoesRelFileExist` returns false which means there is no collision and
16421 is good for use.

However YB uses 16421 differently from PG, it is also used to compose a table id
so it is used in the same way as a table OID, not as a `relfilenode` in the
PG's sense.

To fix the bug, I adjusted the PG code so that in YB context, it will check for
pg_class for OID collision. In this way we'll find that 16421 already exists in
pg_class and it will be skipped and we will try the next OID 16422 until a good
one is found that does not collide with any existing pg_class OIDs.

A new test is added that would fail before this diff.
Jira: DB-13220

Test Plan: ./yb_build.sh debug --cxx-test pgwrapper_pg_libpq-test   --gtest_filter PgLibPqTest.TableRewriteOidCollision

Reviewers: yguan, mihnea

Reviewed By: yguan

Subscribers: fizaa, dhruva.upadhyaya, yql

Differential Revision: https://phorge.dev.yugabyte.com/D40516
Assets 2
Loading