perf(core): use batch inserts in UoW (postgres & mongodb) #865
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
UoW will now batch all inserts statements if the driver supports returning all of their PKs back (so postgres and mongodb currently). This also changes how `Collection.contains()` was implemented - it did a linear check on all values, comparing by the PK for persisted entities. This took extreme amount of time with huge collections, just to add to a collection 1000 times, it was as slow as 1s for this, without persisting anything. Now the collection items are stored as `Set`, and `contains` method does a simple identity lookup (`set.has(entity)`). Another issue that slowed it down was the array like index support, where we are able to use `coll[0]` to access the first item in the collection. This was implemented as a re-assigning of all items to the collection object, which was extremely slow for large collections. Now when we add to a collection, we only append the last index instead of reassigning everything. Closes #732
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
UoW will now batch all inserts statements if the driver supports returning all of their
PKs back (so postgres and mongodb currently).
This also changes how
Collection.contains()was implemented - it did a linear checkon all values, comparing by the PK for persisted entities. This took extreme amount of
time with huge collections, just to add to a collection 1000 times, it was as slow as
1s for this, without persisting anything.
Now the collection items are stored as
Set, andcontainsmethod does a simple identitylookup (
set.has(entity)). Another issue that slowed it down was the array like index support,where we are able to use
coll[0]to access the first item in the collection. This wasimplemented as a re-assigning of all items to the collection object, which was extremely slow
for large collections. Now when we add to a collection, we only append the last index instead
of reassigning everything.
Closes #732