Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Incremental Compaction Strategy (ICS) #3

Open
marvin-j97 opened this issue Dec 15, 2023 · 0 comments
Open

Incremental Compaction Strategy (ICS) #3

marvin-j97 opened this issue Dec 15, 2023 · 0 comments
Labels
enhancement New feature or request help wanted Extra attention is needed

Comments

@marvin-j97
Copy link
Contributor

marvin-j97 commented Dec 15, 2023

Use target_size to chunk segments into runs of segments.
image
Comparing:

  • Tiered: merges N segments into a bigger segment into the next level
  • Levelled: keeps same-sized segments disjunct in each level, just increasing the segment cap per level
  • Incremental: mix between both. Instead of creating one bigger segment, its split into same sized segments, which creates a sorted run. There may be N sorted runs per level, which can then be merged into the next level in a staggered fashion, solving the double size problem:
    image
@marvin-j97 marvin-j97 added enhancement New feature or request help wanted Extra attention is needed labels Dec 15, 2023
@marvin-j97 marvin-j97 transferred this issue from fjall-rs/fjall Jan 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

1 participant