Join GitHub today
GitHub is home to over 50 million developers working together to host and review code, manage projects, and build software together.
Sign upImplement sync record compaction #6250
Closed
Labels
Comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Description
Currently there are on average about 10K sync records per user which can correspond to new bookmarks, deletes or bookmark modifications, including re-ordering. Although the records are encrypted and it is impossible to tell, I believe that there are many more bookmark records than there are actual bookmarks, perhaps by two orders of magnitude. I believe that having this number of records can make it very difficult to add new devices to the sync chain (having to pull down thousands and thousands of records instead of just hundreds). Therefore, I believe we should compact these records where possible to reduce the number to deal with.
Because of how the records are encrypted, we are unable to compact these server side. Therefore, we need to implement sync compaction client side. Talking with @bridiver, we have a proposed algorithm:
When enabling this, we should notice significant drops in number of records. @mrose17 has provided a sync chain that we can evaluate this against when we are ready.