Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cache checksums for contiguous blocks of pages #350

Open
benbjohnson opened this issue Jun 19, 2023 · 0 comments
Open

Cache checksums for contiguous blocks of pages #350

benbjohnson opened this issue Jun 19, 2023 · 0 comments
Labels
performance Improvements to performance

Comments

@benbjohnson
Copy link
Collaborator

LiteFS maintains an in-memory map of checksums for each page in the database and then aggregates them on commit for each transaction. This is decently fast but can become a bottleneck for larger databases (e.g. 5GB+) with small transactions. To improve this, a higher-level checksum cache can be used to preaggregate blocks of, say, 64 pages at a time. A block in this higher-level cache will be invalidated when a write occurs within its range.

@benbjohnson benbjohnson added the performance Improvements to performance label Jun 19, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
performance Improvements to performance
Projects
None yet
Development

No branches or pull requests

1 participant