Add batched SQL for insert/update operations #306
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This introduce a new way of doing "migration" of data based on the
operation table. Before, we simply executed the SQL query as the
migration were run. Like for a "new" operation, we did an insert of a
"transactions" row and an update to the "operations" table for each
entry in the synced file. For a file containing 1000 keys, we did a
total of 2000 SQL queries.
After this commit, we only do 2 SQL queries (well 4 since we batch the
SQL params by 500). By aggregating operation like {:update_all,
operation.id, %{synced: true}}, we are able to batch the operations
efficiently.
This results in a much faster sync/add translations/correct/uncorrect
operations.
(I also snuck in a global API token fix)