Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

miner/oracle: optimize eligibility validation #4883

Closed
dshulyak opened this issue Aug 21, 2023 · 0 comments
Closed

miner/oracle: optimize eligibility validation #4883

dshulyak opened this issue Aug 21, 2023 · 0 comments
Assignees
@dshulyak
Labels
area/tortoise

Comments

@dshulyak
Copy link
Contributor

dshulyak commented Aug 21, 2023 •
edited

the code below should be executed only for reference ballot. the sync is incredibly slow with that loop

image

also for current epoch we should get this data from tortoise rather then lru. lru is bad here, because if set to 10_000 and in fact there are 11_000 in active set then this cache will be threshed. tortoise on the other hand needs to store this data regardless, and it stores it in a compact form.
@dshulyak dshulyak mentioned this issue Aug 21, 2023
Closed
bors bot pushed a commit that referenced this issue Aug 21, 2023
@dshulyak
datastore: parametrize and increase cache size (#4882)
3f8aeb2 
tried to sync node and noticed that it spends a lot of time looping over activeset in eligibility validation. i will open an issue to optimize that code path, but for now we need to increase number of activations stored in cache.

related: #4883
@dshulyak dshulyak self-assigned this Aug 28, 2023
@dshulyak dshulyak mentioned this issue Aug 30, 2023
Closed
bors bot pushed a commit that referenced this issue Aug 31, 2023
@dshulyak
miner/oracle: optimize eligibility validation (#4923)
1305533 
closes: #4883
related: #4841

active set is validated only for reference ballot in current epoch, for all other ballots eligibilities validated against declared slots in the reference ballot. beside making validation more efficient, it will allow to prune active set data in future changes. reference ballots is still loading data from cached datastore, i plan to remove eventually but not in this change.

additionally i rewrote all tests for eligibility validator to get to 100% coverage and make them less dependent on the implementation.
@bors bors bot closed this as completed in 2c917b9 Aug 31, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@dshulyak dshulyak

Labels
area/tortoise
Projects
Dev team kanban
Archived in project
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[Merged by Bors] - optimize ballot eligibility validation dshulyak/go-spacemesh
1 participant
@dshulyak