Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: use less memory when export #3307

Merged
merged 5 commits into from
Aug 2, 2023
Merged

Conversation

lemmih
Copy link
Contributor

@lemmih lemmih commented Aug 2, 2023

Summary of changes

Changes introduced in this pull request:

  • Map directly from Hash to position when compressing .forest.car.zst files.
  • Show a progress indicator when exporting snapshots without a forest-node.

The memory required to export a mainnet snapshot goes from 14 GiB to 8 GiB.

Reference issue to close (if applicable)

Closes

Other information and links

Change checklist

  • I have performed a self-review of my own code,
  • I have made corresponding changes to the documentation,
  • I have added tests that prove my fix is effective or that my feature works (if possible),
  • I have made sure the CHANGELOG is up-to-date. All user-facing changes should be reflected in this document.

@lemmih lemmih marked this pull request as ready for review August 2, 2023 07:47
@lemmih lemmih requested a review from a team as a code owner August 2, 2023 07:47
@lemmih lemmih requested review from hanabi1224, jdjaustin and ruseinov and removed request for a team August 2, 2023 07:47
@lemmih lemmih enabled auto-merge August 2, 2023 11:50
@lemmih lemmih added this pull request to the merge queue Aug 2, 2023
Merged via the queue into main with commit 91bf582 Aug 2, 2023
23 checks passed
@lemmih lemmih deleted the lemmih/use-less-memory-when-export branch August 2, 2023 12:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants