Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[bloom-compactor] Move meta.json creation at the end of compaction cycle #11234

Merged
merged 5 commits into from
Nov 15, 2023

Conversation

poyzannur
Copy link
Contributor

What this PR does / why we need it:
This is a follow up from #11115
Instead of a creating a meta file per bloom creation, create a meta file per compaction cycle.

Which issue(s) this PR fixes:
Fixes #

Special notes for your reviewer:

@poyzannur poyzannur requested a review from a team as a code owner November 15, 2023 14:08
Copy link
Contributor

Trivy scan found the following vulnerabilities:

@poyzannur poyzannur changed the title Move meta.json creation at the end of compaction cycle [bloom-compactor]Move meta.json creation at the end of compaction cycle Nov 15, 2023
@poyzannur poyzannur changed the title [bloom-compactor]Move meta.json creation at the end of compaction cycle [bloom-compactor] Move meta.json creation at the end of compaction cycle Nov 15, 2023
}

storedBlockRefs := make([]bloomshipper.BlockRef, len(storedBlocks))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TODO: use pool for block refs and for tombstone refs. We should pass the storedBlocks buffer to CompactNewChunks to avoid allocations.

@poyzannur poyzannur merged commit 3f0f8fa into main Nov 15, 2023
7 checks passed
@poyzannur poyzannur deleted the poyzannur/move-meta-creation branch November 15, 2023 16:57
jeschkies pushed a commit that referenced this pull request Nov 21, 2023
…cle (#11234)

**What this PR does / why we need it**:
This is a follow up from #11115
Instead of a creating a meta file per bloom creation, create a meta file
per compaction cycle.

**Which issue(s) this PR fixes**:
Fixes #<issue number>

**Special notes for your reviewer**:
rhnasc pushed a commit to inloco/loki that referenced this pull request Apr 12, 2024
…cle (grafana#11234)

**What this PR does / why we need it**:
This is a follow up from grafana#11115
Instead of a creating a meta file per bloom creation, create a meta file
per compaction cycle.

**Which issue(s) this PR fixes**:
Fixes #<issue number>

**Special notes for your reviewer**:
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants