Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

config: move crash and stateproof DB defaults to hot dir #5817

Merged
merged 8 commits into from Nov 16, 2023

Conversation

cce
Copy link
Contributor

@cce cce commented Nov 1, 2023

Summary

This moves the crash and stateproof DBs to default to using the hot data directory. Like the ledger DB, they are frequently written while processing consensus and block updates, and are also fairly small in size, so they would benefit from being located in the hot tier.

Resolves #5831

Test Plan

Updated TestEnsureAndResolveGenesisDirs_hierarchy and TestConfiguredDataDirs.
Added TestEnsureAndResolveGenesisDirs_migrate and TestEnsureAndResolveGenesisDirs_migrateFail.

@winder
Copy link
Contributor

winder commented Nov 2, 2023

Why?

@cce cce added the Enhancement label Nov 3, 2023
Copy link

codecov bot commented Nov 3, 2023

Codecov Report

Attention: 2 lines in your changes are missing coverage. Please review.

Comparison is base (8c54c22) 55.63% compared to head (1d8c990) 55.61%.
Report is 14 commits behind head on master.

Files Patch % Lines
config/localTemplate.go 90.47% 1 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #5817      +/-   ##
==========================================
- Coverage   55.63%   55.61%   -0.02%     
==========================================
  Files         475      475              
  Lines       66842    66860      +18     
==========================================
- Hits        37187    37184       -3     
- Misses      27141    27156      +15     
- Partials     2514     2520       +6     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Contributor

@algorandskiy algorandskiy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree on the crash db move but do we really write stateproof db that often?

@cce
Copy link
Contributor Author

cce commented Nov 6, 2023

The stateproof DB is

  • updated every time you observe a stateproof signature message — these start coming in 128 rounds into the interval, each participant issues them staggered over the remaining 128 rounds, and you have to update the DB for each observation.
  • You also need to query the state proof DB to sign the state proof transaction.
  • The voters tracker in ledger calls through OnPrepareVoterCommit to stateproof.Worker.OnPrepareVoterCommit to do gets & writes to the state proof DB as part of the tracker prepareCommit, so you can think of state proof DB as an extension to the tracker DB, since it is read/written/flushed at regular intervals along with tracker DB..

I guess it depends on how cold you think cold will be..

Copy link
Contributor

@algorandskiy algorandskiy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM but use existing file name constants instead of literals

config/localTemplate.go Outdated Show resolved Hide resolved
config/localTemplate.go Outdated Show resolved Hide resolved
algorandskiy
algorandskiy previously approved these changes Nov 16, 2023
Copy link
Contributor

@winder winder left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes look good. Left some suggestions that I think should be adopted, but don't need to block the PR.

@algorandskiy algorandskiy merged commit 948f08b into algorand:master Nov 16, 2023
18 checks passed
@cce cce deleted the crashdbhotdir branch November 16, 2023 15:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

put agreement DB in hot storage tier
3 participants