Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

deleteFeed to support removing OOO #417

Merged
merged 1 commit into from
Jan 24, 2023
Merged

deleteFeed to support removing OOO #417

merged 1 commit into from
Jan 24, 2023

Conversation

staltz
Copy link
Member

@staltz staltz commented Jan 24, 2023

Context

ssbc/ssb-replication-scheduler#14

Problem

deleteFeed doesn't support deleting messages that were added with OOO, because we have a state check, and addOOO has been recently updated #412 to not update the Base state.

Solution

Drop the state check in deleteFeed.

Add tests.

@staltz staltz requested a review from arj03 January 24, 2023 14:10
@github-actions
Copy link

Benchmark results

Part Duration
Create 5000 new messages 365.25ms
Validate 5000 messages 480.18ms
Native to db format 5000 messages 108.89ms
Db to native format 5000 messages 85.59ms
Add 1000 elements 494.55ms
Add 1000 box1 msgs 1000.61ms
Unbox 1000 box1 msgs first run 215.75ms
Unbox 1000 box1 msgs second run 106.79ms
Add 1000 box1 msgs 961.55ms
Query 1000 msgs first run 51.68ms
Query 1000 msgs second run 12.16ms
Add 1000 box2 msgs 1271.58ms
Unbox 1000 box2 msgs first run 315.94ms
Unbox 1000 box2 msgs second run 146.10ms
Migrate (+db1) 10673.75ms
Migrate (alone) 3564.19ms
Migrate (+db1 +db2) 8202.69ms
Migrate (+db2) 5657.83ms
Migrate continuation (+db2) 832.68ms
Memory usage without indexes 755.13 MB = 38.65 MB + etc
Initial indexing 1544.82ms
Initial indexing maxcpu=86 3927.84ms
Initial indexing compat 733.11ms
Two indexes updating concurrently 2709.03ms
Key one initial 48.69ms
Key two 1.81ms
Key one again 0.99ms
Reboot and key one again 47.59ms
Latest root posts 1700.88ms
Latest posts 8.10ms
Votes one initial 1436.19ms
Votes again 0.54ms
HasRoot 1300.33ms
HasRoot again 0.34ms
Author one posts 1401.04ms
Author two posts 17.68ms
Dedicated author one posts 1399.16ms
Dedicated author one posts again 0.56ms
DeleteFeed 4986.93ms
Maximum memory usage 993.84 MB = 60.36 MB + etc
Indexes folder size 10.01mb

Copy link
Member

@arj03 arj03 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great, the tests looks good.

Nice, -2 lines for this "feature" :D

@staltz staltz merged commit 289e00a into master Jan 24, 2023
@staltz staltz deleted the delete-feed-ooo branch January 24, 2023 14:22
@github-actions
Copy link

Benchmark results

Part Duration
Create 5000 new messages 448.41ms
Validate 5000 messages 521.82ms
Native to db format 5000 messages 145.41ms
Db to native format 5000 messages 97.99ms
Add 1000 elements 380.60ms
Add 1000 box1 msgs 1106.29ms
Unbox 1000 box1 msgs first run 246.35ms
Unbox 1000 box1 msgs second run 124.33ms
Add 1000 box1 msgs 1056.72ms
Query 1000 msgs first run 61.98ms
Query 1000 msgs second run 18.04ms
Add 1000 box2 msgs 1483.95ms
Unbox 1000 box2 msgs first run 387.10ms
Unbox 1000 box2 msgs second run 194.26ms
Migrate (+db1) 13290.18ms
Migrate (alone) 4698.11ms
Migrate (+db1 +db2) 9882.37ms
Migrate (+db2) 7338.61ms
Migrate continuation (+db2) 1265.23ms
Memory usage without indexes 741.71 MB = 36.89 MB + etc
Initial indexing 1878.65ms
Initial indexing maxcpu=86 6404.96ms
Initial indexing compat 982.85ms
Two indexes updating concurrently 3282.58ms
Key one initial 59.67ms
Key two 1.33ms
Key one again 0.93ms
Reboot and key one again 54.08ms
Latest root posts 2004.96ms
Latest posts 10.84ms
Votes one initial 1647.66ms
Votes again 0.62ms
HasRoot 1468.29ms
HasRoot again 0.34ms
Author one posts 1566.30ms
Author two posts 21.09ms
Dedicated author one posts 1601.89ms
Dedicated author one posts again 0.57ms
DeleteFeed 6768.55ms
Maximum memory usage 1009.55 MB = 58.21 MB + etc
Indexes folder size 10.01mb

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants