Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SQLAlchemy: Use server-side now() function for "autoincrement" columns #24

Merged
merged 1 commit into from
Sep 21, 2023

Conversation

amotl
Copy link
Member

@amotl amotl commented Sep 18, 2023

Now uses the server-side now() function to patch column models using the "autoincrement" property, as suggested by @seut at crate/sqlalchemy-cratedb#77. Thanks!

@codecov
Copy link

codecov bot commented Sep 18, 2023

Codecov Report

Patch coverage is 100.00% of modified lines.

Files Changed Coverage
mlflow_cratedb/patch/crate_python.py 100.00%

📢 Thoughts on this report? Let us know!.

@amotl amotl marked this pull request as ready for review September 18, 2023 11:31
@amotl amotl force-pushed the amo/trim-autoincrement-column branch from 964fccb to e251ef8 Compare September 18, 2023 11:51
@amotl amotl requested a review from seut September 18, 2023 12:43
@amotl amotl changed the title SQLAlchemy: Trim implementation for "autoincrement" columns SQLAlchemy: Use server-side now() function for "autoincrement" columns Sep 18, 2023
@amotl amotl force-pushed the amo/trim-autoincrement-column branch from e251ef8 to 628b97c Compare September 18, 2023 15:19
@amotl
Copy link
Member Author

amotl commented Sep 18, 2023

That's a fluke, possibly because of too less granularity in the time domain when only using millisecond resolution. However, this visited the test suite on CI already on a previous occasion, see crate/crate-python#11, so it is not holding me back on merging this.

-- https://github.com/crate-workbench/mlflow-cratedb/actions/runs/6224569395/job/16893059034?pr=24#step:6:175

@hammerhead
Copy link
Member

Does autoincrement in SQLAlchemy guarantee uniqueness? Although unlikely in practice, using NOW() makes it possible that two concurrent INSERT statements assign the same value to such a column?

@amotl
Copy link
Member Author

amotl commented Sep 19, 2023

Does autoincrement in SQLAlchemy guarantee uniqueness?

It will have the same guarantees what the relevant DBMS has to offer. In the case of PostgreSQL, it is implemented by its "sequences" subsystem. I think it will guarantee one the highest levels of uniqueness, because, as a feature typically used for generating primary keys, it is used by all kinds of DBMS all over the world for this purpose.

Although unlikely in practice, using NOW() makes it possible that two concurrent INSERT statements assign the same value to such a column?

You are right, but this is not unlikely at all, and can easily be invoked by corresponding test cases simulating high concurrency, what MLflow actually does, that's why it is occasionally tripping. This is why I also was proposing an alternative client-side approach with microsecond resolution at crate/sqlalchemy-cratedb#77, to lower the chance of collisions. However, in this case, the outcome doesn't show any significant difference, and works in 95% of the test runs.

@amotl amotl force-pushed the amo/trim-autoincrement-column branch from 628b97c to b1c04d6 Compare September 21, 2023 13:08
@amotl
Copy link
Member Author

amotl commented Sep 21, 2023

Another test run on this patch worked well, so it can't be that wrong.

@amotl amotl merged commit bc8036f into main Sep 21, 2023
3 checks passed
@amotl amotl deleted the amo/trim-autoincrement-column branch September 21, 2023 19:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants