Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Loading saving cache improvements #1072

Merged
merged 10 commits into from
Oct 7, 2023
Merged

Loading saving cache improvements #1072

merged 10 commits into from
Oct 7, 2023

Conversation

qarmin
Copy link
Owner

@qarmin qarmin commented Oct 5, 2023

Fixes #1064

  • Remove unneeded code duplication in all modules
  • more debug info when using RUST_LOG=debug that can help find what works bad/slow
  • multithreading removing non existent files
  • due using BTreeMap instead HashMap performance may be a little slower, but other changes should give bigger performance gain than that

Since this commit, old cache will not be usable and will probably throw segmentation fault(maybe in future bincode 2 will better handle cache version mismatch)

On i7 4770 and cheap ssd 1GB drive - 114988 results - 17.1 MB size

  • loading cache - 300ms
  • loading cache with removing outdated files - 370ms
  • saving cache - 800ms

@qarmin qarmin merged commit e976d40 into master Oct 7, 2023
20 checks passed
@qarmin qarmin deleted the loading_saving branch October 7, 2023 16:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

cache reuse for small subset of files
1 participant