Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[YSQL] Make system catalog tables transactional #1383

Open
robertpang opened this issue May 15, 2019 · 2 comments

Comments

@robertpang
Copy link
Contributor

commented May 15, 2019

Currently for implementation reasons system catalog tables are non-transactional. There has been a case where a client reads the tables in an inconsistent state while loading the sys catalog cache, leading to FATAL error. We should make the tables transactional to avoid such issues. This change can also make it easy to rollback sys catalog table changes in case of a failed DDL operation.

@robertpang robertpang added this to To do in SQL Support via automation May 15, 2019

@ndeodhar ndeodhar added this to the v2.0 milestone May 15, 2019

@mbautin

This comment has been minimized.

Copy link
Collaborator

commented Jun 19, 2019

Some initial things I'm noticing trying to implement this:

  • Switching the main sys catalog table to be transactional works, at least on the surface. We still have to specify transaction_participant_context and transaction_coordinator_context in sys_catalog.cc.
  • We need to flip postgres sys catalog tables to be transactional.
  • We need to create the txn status table earlier in the CreateTable handler implementation in catalog_manager.cc so it is ready by the time we start sending DML to postgres sys catalog tables.
  • To create the transaction status table, at least one tserver needs to be up (in the RF=1 case). Might need to wait for tserver to start before the master starts running initdb.
  • Alternatively, we could allow the sys catalog tablet to also act as a transaction status tablet for transactions involving sys catalog tables.

@mbautin mbautin self-assigned this Jun 19, 2019

@mbautin

This comment has been minimized.

Copy link
Collaborator

commented Jun 19, 2019

Also, our buffering operations needed to be disabled because it is currently not supported for transactional tables.

WIP patch: https://gist.githubusercontent.com/mbautin/db313ed7eae0c81a8dc12ff620c00706/raw
Log (fails on trying to create the second table, need to figure out if we're missing a commit): https://gist.githubusercontent.com/mbautin/69a884cee447402836a1a67905ce0967/raw

Also as another idea we should probably running initdb in non-transactional mode for speed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
3 participants
You can’t perform that action at this time.