Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Trigger sync compaction for bookmarks every week #3749

Merged
merged 5 commits into from Nov 7, 2019
Merged

Trigger sync compaction for bookmarks every week #3749

merged 5 commits into from Nov 7, 2019

Conversation

@darkdh
Copy link
Member

darkdh commented Oct 18, 2019

fix brave/brave-browser#6552

Submitter Checklist:

Test Plan:

#3749 (comment) can be referenced to generate update records easily

Update

  1. Make sure launch old Brave on device A and device B for fresh profiles
  2. Sync device A and device B
  3. Sync 2 bookmarks
  4. Keep changing bookmarks name or URL of the 2 bookmarks for multiple times on device A
  5. Make sure device B get all the update
  6. Quit Brave on device A and launch new Brave with the same profile
  7. Open chrome://inspect/#extensions and inspect Brave Sync on device A
  8. After seeing compaction deletes ... on console and wait for extra 3 mins
  9. Connect device C to the sync chain
  10. Open chrome://inspect/#extensions and inspect Brave Sync on device C
  11. You should see exactly got 2 decrypted records in BOOKMARKS after 0 on console
  12. And 2 bookmarks are up to date with device A and device B

Delete

  1. Make sure launch old Brave on device A and device B for fresh profiles
  2. Sync device A and device B
  3. Sync 1 bookmark
  4. Keep changing bookmarks name or URL of the bookmark for multiple times on device A
  5. Make sure device B get all the update
  6. Delete the bookmark on device A
  7. Make sure device B also deletes the bookmark
  8. Quit Brave on device A and launch new Brave with the same profile
  9. Open chrome://inspect/#extensions and inspect Brave Sync on device A
  10. After seeing compaction deletes ... on console and wait for extra 3 mins
  11. Connect device C to the sync chain
  12. Open chrome://inspect/#extensions and inspect Brave Sync on device C
  13. You should see exactly got 1 decrypted records in BOOKMARKS after 0 on console
  14. And device C should still have no bookmark

Reviewer Checklist:

  • New files have MPL-2.0 license header.
  • Request a security/privacy review as needed.
  • Adequate test coverage exists to prevent regressions
  • Verify test plan is specified in PR before merging to source

After-merge Checklist:

  • The associated issue milestone is set to the smallest version that the
    changes has landed on.
  • All relevant documentation has been updated.
@darkdh darkdh self-assigned this Oct 18, 2019
@darkdh darkdh force-pushed the sync-compaction branch from bbd6997 to 5061946 Oct 21, 2019
@darkdh darkdh marked this pull request as ready for review Oct 21, 2019
Copy link
Contributor

AlexeyBarabash left a comment

PR looks good

@AlexeyBarabash
Copy link
Contributor

AlexeyBarabash commented Oct 22, 2019

Verified it works with STR of:

I. Making bookmarks with lots of updates

  1. Get clean profiles A and B
  2. Establish the sync chain, wait until there will be two devices
  3. Open page brave://bookmarks on device A
  4. Call page inspector, right click on tree side => Inspect
  5. On console tab paste the code
for (var i = 0; i < 10; ++i) {
chrome.bookmarks.create({'parentId':BOOKMARKS_BAR_ID, 'title': `Bookmark_${i}`,
                         'url': `https://A_${i}.com`});
}

  1. Wait while bookmarks will appear on device B
  2. On device A paste and enter this in brave://bookmarks Inspector console:
function processNode(node) {
    if(node.children) {
        node.children.forEach(function(child) { processNode(child); });
    }
    if(node.title && node.url) { 
	chrome.bookmarks.update(node.id, {'title': node.title+`_A`});
    }
}
chrome.bookmarks.getTree(function(itemTree){
    itemTree.forEach(function(item){
        processNode(item);
    });
});

paste and run it several times
8. Wait for device B will get the changes
9. Close browser device A

II. Initiating compact operation

  1. Open folder of profile A, find Preferences file, in my case this is /home/alexey/.config/BraveSoftware/TEST/A/Default/Preferences
  2. Open Preferences file with text editor and remove part like "last_compact_time":"13216219229248049", , save and close file
  3. Launch device A
  4. Ensure you see compaction deletes messages in console or in inspector console

III. Verifying compact was correct

  1. Get clean profile C
  2. Copy sync code from device A
  3. Connect device C to the sync chain
  4. Ensure device C has the same bookmarks as device A
  5. See in inspector console for Brave Sync extension get-existing-objects gives objects without repeating by objectId
extensions::api::brave_sync::OnSendCompactSyncCategory::Create(
category_name).release());
std::unique_ptr<Event> event(
new Event(extensions::events::FOR_TEST,

This comment has been minimized.

Copy link
@bridiver

bridiver Oct 22, 2019

Collaborator

FOR_TEST?

This comment has been minimized.

Copy link
@darkdh

darkdh Oct 22, 2019

Author Member

it's been this way since we have BraveSyncEventRouter

This comment has been minimized.

Copy link
@darkdh

darkdh Oct 22, 2019

Author Member

it is just a events::HistogramValue and I can't find a suitable value from the existing list. Should we extend the enum? Please note that brave sync will eventually go away and migrated to native implementation

base::Time::Now() - last_compact_time >
base::TimeDelta::FromDays(kCompactPeriodInDays)) {
brave_sync_client_->SendCompactSyncCategory(kBookmarks);
brave_sync_prefs_->SetLastCompactTime(base::Time::Now());

This comment has been minimized.

Copy link
@bridiver

bridiver Oct 22, 2019

Collaborator

shouldn't we wait to do this until the compaction is complete?

This comment has been minimized.

Copy link
@darkdh

darkdh Oct 22, 2019

Author Member

You mean, when the objects are really deleted on s3?
The compaction API is designed to have no callback, if we want callback when the object is really deleted, we would need extra listObject query to check. The deletion is not instant, it takes approximately 1 minute

This comment has been minimized.

Copy link
@darkdh

darkdh Oct 22, 2019

Author Member

Also, the compaction could take a significant long time to pull down all the record if the recordset is huge

This comment has been minimized.

Copy link
@darkdh

darkdh Oct 29, 2019

Author Member

addressed in bb0c6c3
which requires deps update when brave/sync#355 is merged

Copy link
Member

SergeyZhukovsky left a comment

++

@darkdh darkdh force-pushed the sync-compaction branch from 5061946 to e07c36b Oct 22, 2019
@darkdh darkdh changed the title Triger sync compaction for bookmarks every week Trigger sync compaction for bookmarks every week Oct 22, 2019
@darkdh darkdh force-pushed the sync-compaction branch 2 times, most recently from 56d2773 to aef2165 Oct 28, 2019
@darkdh darkdh added the feature/sync label Nov 2, 2019
@darkdh darkdh force-pushed the sync-compaction branch from bb0c6c3 to 0e2d34f Nov 5, 2019
@darkdh darkdh force-pushed the sync-compaction branch 2 times, most recently from bda5e45 to 527100a Nov 6, 2019
@darkdh darkdh force-pushed the sync-compaction branch from 527100a to f1b7039 Nov 6, 2019
darkdh added 4 commits Oct 18, 2019
confusion
@darkdh darkdh force-pushed the sync-compaction branch from f1b7039 to 8d915f6 Nov 6, 2019
@darkdh darkdh merged commit a315f9a into master Nov 7, 2019
2 checks passed
2 checks passed
continuous-integration/jenkins/pr-head This commit looks good
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
@darkdh darkdh deleted the sync-compaction branch Nov 7, 2019
@darkdh darkdh added this to the 0.74.x - Nightly milestone Nov 7, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

4 participants
You can’t perform that action at this time.