Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[YSQL] Creating a table can cause a constraint violation on pg_class_oid_index. #1962

Open
aphyr opened this issue Aug 2, 2019 · 2 comments

Comments

@aphyr
Copy link

commented Aug 2, 2019

With version 1.3.1.0, I've hit occasional (~1:10 runs) issues creating YSQL tables during Jepsen tests. For instance, the append test creates a single table like so:

  (setup-cluster! [this test c conn-wrapper]
    (c/execute! c (j/create-table-ddl table
                                      [[:k :int]
                                       [:v :text]
                                       ["PRIMARY KEY" "(k)"]])))

... which is executed in a global lock, so only one client should be acting at a time. Sometimes, but not always, this can throw:

org.postgresql.util.PSQLException: ERROR: duplicate key value violates unique constraint "pg_class_oid_index"

which suggests to me that there might be some sort of double-allocation of OIDs possible during table creation--Postgres mailing lists suggest this might be due to OID overflow or filesystem corruption, but I doubt either of those is at play here. Since we create a fresh cluster every time, I wonder if there might be some sort of table initialization code that's running concurrently with our table creation, and the two can occasionally race? Not sure!

For full logs, see 20190802T132514.000-0400.zip.

To reproduce this, try Jepsen 23dbb5e78725318c35d07dcc6d727ebb3ab7a3e7, and run something like

lein run test --os debian --version 1.3.1.0 --time-limit 5 -w ysql/append --test-count 30

I've only hit this once, after a handful of runs, so I'm not sure how frequent a problem it is yet. I'll post back if I see it again!

@frozenspider frozenspider added this to To do in Jepsen Testing via automation Aug 2, 2019

@frozenspider frozenspider added this to To do in SQL Support via automation Aug 2, 2019

@frozenspider frozenspider changed the title Creating a table can cause a constraint violation on pg_class_oid_index. [YSQL] Creating a table can cause a constraint violation on pg_class_oid_index. Aug 2, 2019

@aphyr

This comment has been minimized.

Copy link
Author

commented Aug 2, 2019

Hit a second case, looks to be roughly 1:20 runs so far. 20190802T144108.000-0400.zip

@ndeodhar

This comment has been minimized.

Copy link
Contributor

commented Aug 2, 2019

This happens because our SQL DDL statements are not transactional. I see this in the logs:

I0802 18:41:39.796936 48903 pg_ddl.cc:141] PgCreateTable: creating a transactional table: postgres.append
I0802 18:41:44.952004 48903 table_creator.cc:286] Created table postgres.append of type PGSQL_TABLE_TYPE
I0802 18:41:49.305517 48913 meta_cache.cc:588] Got resp error 7, code=OK                             
W0802 18:41:49.306098 48913 meta_cache.cc:602] Leader Master has changed, re-trying... 

So postgres system tables were updated but there was a master leader change and so create table never completed successfully. When client retries request on a new leader, we see the duplicate key error.

Related issue about transactional DDLs: #1383

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
6 participants
You can’t perform that action at this time.