-
Notifications
You must be signed in to change notification settings - Fork 131
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
chore(client): Performance improvements on initial sync downstream pa…
…th (#1363) I profiled the downstream path for initial syncs, excluding SQLite performance as that will be driver and storage dependent and I plan to focus on that separately, and implemented some changes on the most significant bottlenecks for that pathway. The tests were done using 10k issues and 50k comments on Linearlite (although now benchmarking suite @icehaunter made is available so I will be using that), and performance improvements were checked in isolation with JS perf tests as well (on Chrome/V8 but I avoided any super niche optimizations) In memory performance is improved for this path, excluding the execution of the SQL statements the 10k issues and 50k comments go through `_applySubscriptionData` in under 200ms on my machine. One very significant overhead is the decoding, first of the Satellite messages in the generated library which I don't think we can do much about, but then also to JS variables with our decoders that spend a lot of time and also cause lots of GC pauses. There are ways we could optimise this path but I was reluctant to spend time on that, ultimately a lot of the deserialization is pointless in the sense that it is re-serialized for inserting to permanent storage, but if we want to keep things modular and pluggable with e.g. JS SQLite drivers this might be necessary. Some individual commits contain messages with a brief description of the performance bits, mostly I tried to reuse objects to reduce GC pauses and that seemed to give the most benefit. Measured performance with 10k issues and 50k comments of `_handleSubscriptionData`: on main: `265ms ± 36ms` after improvements: `135ms ± 9ms`
- Loading branch information
Showing
9 changed files
with
270 additions
and
141 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,5 @@ | ||
--- | ||
"electric-sql": patch | ||
--- | ||
|
||
Minor performance improvements in downstream initial sync path |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.