-
Notifications
You must be signed in to change notification settings - Fork 2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
avoid DataLayer insert ancestors precheck #15728
Conversation
now we always write and we read if this is a known row it looks to me like this change has performance tradeoffs, whats the average expected case ? |
Yes, this change is intended to boost performance by reducing the number of queries. I can't say that a detailed analysis has been done but evidence so far is that we tend to insert new rows, not re-insert the exact same info, so cutting the query count in half (for this function) when doing that expected normal case is an improvement. The check is just in case we end up inserting the same keys (SQLite checked) with different values in other columns. It is a double check for errors elsewhere, but I wanted to get the optimization in place without working through whether the double check was critical or important or slightly useful or.... We think this does that. |
This pull request has conflicts, please resolve those before we can evaluate the pull request. |
1 similar comment
This pull request has conflicts, please resolve those before we can evaluate the pull request. |
Conflicts have been resolved. A maintainer will review the pull request shortly. |
* Cache root changes in insert_batch loop * Reduce selects for root and generation * remove sql logging, extracted to #15690 * remove optimization extracted to #15691 * remove optimization extracted to #15728 * Delete test_batch_speed.py benchmark provided in another already merged pr * add `Root.to_row()` * shift optimization to `DataStore.get_keys_values_dict()` * rename to `intermediate_root` * less tuple returning * Revert "shift optimization to `DataStore.get_keys_values_dict()`" This reverts commit b12ae08. * add `test_data_store.test_insert_batch_reference_and_side()` --------- Co-authored-by: Kyle Altendorf <sda@fstab.net>
Purpose:
The expectation is that we will not trigger collisions usually so the except block won't be triggered and we'll save an sql query.
extracted from #15613
Current Behavior:
New Behavior:
Testing Notes: