Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for reserving a primary key #272

Open
Tracked by #251
ecton opened this issue Feb 16, 2023 · 1 comment
Open
Tracked by #251

Add support for reserving a primary key #272

ecton opened this issue Feb 16, 2023 · 1 comment

Comments

@ecton
Copy link
Member

ecton commented Feb 16, 2023

From a conversation on discord with @D1plo1d, a request was made to provide functionality similar to PostgreSQL's NEXTVAL.

We should have a way to get a reserved primary key for a given collection such that the next push to a collection would skip over any reserved keys.

This enables additional transactional flows without the need for interactive transactions.

Implementation-wise, I think the best approach would be to add a copy of the "last-value" key for a given collection to the header of the Nebari tree. Instead of looking at the last key when pushing a document, this last_key would be used instead.

A new function could be added that could increment the last_key and return its value without inserting a new document. I'm uncertain whether this should be a transactional function or atomic with persistence happening the next time the collection is updated. PostgreSQL's documentation has this caveat with their implementation:

If the database cluster crashes before committing a transaction containing a nextval or setval call, the sequence state change might not have made its way to persistent storage, so that it is uncertain whether the sequence will have its original or updated state after the cluster restarts. This is harmless for usage of the sequence within the database, since other effects of uncommitted transactions will not be visible either. However, if you wish to use a sequence value for persistent outside-the-database purposes, make sure that the nextval call has been committed before doing so.

@ecton
Copy link
Member Author

ecton commented Mar 16, 2023

I've added this to the Improve Transactional Performance checklist because this would alter the storage format -- might as well include that part in the format change that is coming with the adoption of the new file format.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant