Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Online DDL - stage 1 #80

Merged
merged 5 commits into from
Oct 17, 2022
Merged

Online DDL - stage 1 #80

merged 5 commits into from
Oct 17, 2022

Conversation

losfair
Copy link
Owner

@losfair losfair commented Oct 16, 2022

Related: #70

This PR implements a restricted form of online DDL. It permits concurrent read-only DML when a schema migration task is in progress.

Currently you can already do non-blocking DDL with regular transactions, but that has a few limitations:

  • Schema migrations in a large database usually take a long time, and are likely to be indefinitely starved by concurrent small transactions in a busy system.
  • A transaction can write at most 50,000 pages of data. This forces the application developer to split a schema migration into multiple smaller transactions, and the application may see inconsistent data in between.

This PR introduces single-writer lock, which when enabled permits only a single writer, the schema migration task, to write to the database. All other tasks see the latest consistent snapshot before the lock.

@losfair
Copy link
Owner Author

losfair commented Oct 16, 2022

It works, but still needs a careful review for correctness.

So much concurrency here.

@losfair losfair marked this pull request as ready for review October 16, 2022 16:49
@losfair losfair merged commit c986ebe into main Oct 17, 2022
@losfair losfair mentioned this pull request Oct 17, 2022
2 tasks
@losfair losfair mentioned this pull request Feb 13, 2023
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant