compact: Make sure sources are removed after compaction.#258
Merged
Conversation
fabxc
reviewed
Mar 27, 2018
| // Eventually the block we just uploaded should get synced into the group again (including sync-delay). | ||
| for _, b := range plan { | ||
| idStr := filepath.Base(b) | ||
| id, err := ulid.Parse(idStr) |
Collaborator
There was a problem hiding this comment.
No need to create idStr, just write ulid.Parse(filepath.Base(b))
|
|
||
| // Spawn a new context so we always delete a block in full on shutdown. | ||
| delCtx, cancel := context.WithTimeout(context.Background(), 5*time.Minute) | ||
| level.Info(cg.logger).Log("msg", "deleting compacted block", "block", id, "compacted", compID) |
Collaborator
There was a problem hiding this comment.
I think the keys here will be somewhat confusing in practice. old_block and replacement_block might be clearer.
| } | ||
|
|
||
| // Spawn a new context so we always delete a block in full on shutdown. | ||
| delCtx, cancel := context.WithTimeout(context.Background(), 5*time.Minute) |
Collaborator
There was a problem hiding this comment.
👍 – though I notice we potentially abort upload of the compacted block but don't clean up remainers if we did that.
Member
Author
There was a problem hiding this comment.
what remainders? Do you mean doing that
?
for _, b := range plan {
id, err := ulid.Parse(filepath.Base(b))
if err != nil {
return compID, errors.Wrapf(err, "plan dir %s", b)
}
if err := os.RemoveAll(b); err != nil {
return compID, errors.Wrapf(err, "remove old block dir %s", id)
}
If we don't upload compacted blocks all if good I think - no need to delete anything. We will do another compaction
Collaborator
There was a problem hiding this comment.
I mean if we upload the result block only partially because the context got canceled. But objstore.UploadDir already does this I remembered.
bwplotka
force-pushed
the
compact-fix-v2
branch
from
5103382 to
eede36e
Compare
Collaborator
|
👍 |
Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
bwplotka
force-pushed
the
compact-fix-v2
branch
from
eede36e to
272c975
Compare
openshift-cherrypick-robot
pushed a commit
to openshift-cherrypick-robot/thanos
that referenced
this pull request
Dec 16, 2025
* Konflux: reintroduce custom changes - multi-arch build on push - hermetic builds - correctly pre-fetching golang deps - build source image Signed-off-by: Jacob Baungard Hansen <jacobbaungard@redhat.com> * Konflux: also prefetch konflux dependencies Signed-off-by: Jacob Baungard Hansen <jacobbaungard@redhat.com> --------- Signed-off-by: Jacob Baungard Hansen <jacobbaungard@redhat.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #246
Signed-off-by: Bartek Plotka bwplotka@gmail.com