Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add getIndexingActive() API #385

Merged
merged 1 commit into from
Aug 22, 2022
Merged

add getIndexingActive() API #385

merged 1 commit into from
Aug 22, 2022

Conversation

staltz
Copy link
Member

@staltz staltz commented Aug 22, 2022

We need this for pausing the ssb-friends / ssb-replication-scheduler integration, because indexingActive is triggered right before log.stream to update the indexes, and we need this "right before" behavior, that status doesn't give us.

I haven't thought out this API that well, I think in the future we could find a more developer friendly way of doing this.

@staltz staltz requested a review from arj03 August 22, 2022 11:01
@arj03
Copy link
Member

arj03 commented Aug 22, 2022

Well its nice that this an an obz so you can do something elsewhere depending on the status (changes) on this. Should we add these to the README (forgot to mention that with reset), or are they considered "internal" for now?

@staltz
Copy link
Member Author

staltz commented Aug 22, 2022

(forgot to mention that with reset)

Oh, I added README docs for reset, but not for getIndexingActive. I kind of not so proud about this API, I want to see if it's going to be actually handy for ssb-replication-scheduler.

Copy link
Member

@arj03 arj03 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@staltz staltz merged commit 3911f6b into master Aug 22, 2022
@staltz staltz deleted the indexing-active branch August 22, 2022 11:08
@github-actions
Copy link

Benchmark results

Part Duration
Create 5000 new messages 553.70ms
Validate 5000 messages 547.25ms
Native to db format 5000 messages 150.71ms
Db to native format 5000 messages 108.31ms
Add 1000 elements 391.63ms
Add 1000 box1 msgs 1184.70ms
Unbox 1000 box1 msgs first run 207.10ms
Unbox 1000 box1 msgs second run 137.41ms
Add 1000 box1 msgs 1142.61ms
Query 1000 msgs first run 46.50ms
Query 1000 msgs second run 17.60ms
Add 1000 box2 msgs 1811.07ms
Unbox 1000 box2 msgs first run 289.76ms
Unbox 1000 box2 msgs second run 223.07ms
Migrate (+db1) 15597.14ms
Migrate (alone) 5703.52ms
Migrate (+db1 +db2) 11732.83ms
Migrate (+db2) 8506.79ms
Migrate continuation (+db2) 1423.82ms
Memory usage without indexes 751.04 MB = 38.18 MB + etc
Initial indexing 945.91ms
Initial indexing maxcpu=86 5148.36ms
Initial indexing compat 938.19ms
Two indexes updating concurrently 1220.40ms
Key one initial 58.56ms
Key two 1.05ms
Key one again 1.23ms
Reboot and key one again 47.51ms
Latest root posts 1010.90ms
Latest posts 9.33ms
Votes one initial 693.63ms
Votes again 0.59ms
HasRoot 480.91ms
HasRoot again 0.48ms
Author one posts 694.68ms
Author two posts 30.31ms
Dedicated author one posts 667.70ms
Dedicated author one posts again 0.79ms
DeleteFeed 3663.45ms
Maximum memory usage 948.33 MB = 62.84 MB + etc
Indexes folder size 10.01mb

@github-actions
Copy link

Benchmark results

Part Duration
Create 5000 new messages 367.25ms
Validate 5000 messages 452.54ms
Native to db format 5000 messages 114.38ms
Db to native format 5000 messages 88.17ms
Add 1000 elements 364.92ms
Add 1000 box1 msgs 928.11ms
Unbox 1000 box1 msgs first run 167.41ms
Unbox 1000 box1 msgs second run 112.29ms
Add 1000 box1 msgs 900.09ms
Query 1000 msgs first run 37.32ms
Query 1000 msgs second run 25.03ms
Add 1000 box2 msgs 1283.62ms
Unbox 1000 box2 msgs first run 209.26ms
Unbox 1000 box2 msgs second run 141.74ms
Migrate (+db1) 12209.75ms
Migrate (alone) 3900.09ms
Migrate (+db1 +db2) 8296.72ms
Migrate (+db2) 6014.54ms
Migrate continuation (+db2) 908.89ms
Memory usage without indexes 751.55 MB = 39.44 MB + etc
Initial indexing 748.04ms
Initial indexing maxcpu=86 3970.25ms
Initial indexing compat 754.72ms
Two indexes updating concurrently 1018.82ms
Key one initial 47.42ms
Key two 0.73ms
Key one again 1.41ms
Reboot and key one again 40.06ms
Latest root posts 793.63ms
Latest posts 9.57ms
Votes one initial 547.25ms
Votes again 1.31ms
HasRoot 379.73ms
HasRoot again 0.31ms
Author one posts 479.03ms
Author two posts 23.97ms
Dedicated author one posts 491.00ms
Dedicated author one posts again 0.73ms
DeleteFeed 2702.94ms
Maximum memory usage 1011.47 MB = 61.32 MB + etc
Indexes folder size 10.01mb

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants