Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

S3 multipart enchancement #7350

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

S3 multipart enchancement #7350

wants to merge 1 commit into from

Conversation

fatpat
Copy link

@fatpat fatpat commented Feb 10, 2024

What this PR does

Allow to disable multipart uploads in storage provider and in tools and to configure part size in tools.

note: a PR has been submitted to thanos-io/objstore (thanos-io/objstore#100) to add DisableMultipart option. I've backport the patch manually in vendor/ while waiting for this PR to be merged.

Checklist

  • Tests updated.
  • Documentation added.
  • CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX].
  • about-versioning.md updated with experimental features.

@fatpat fatpat requested review from grafanabot and a team as code owners February 10, 2024 08:32
@CLAassistant
Copy link

CLAassistant commented Feb 10, 2024

CLA assistant check
All committers have signed the CLA.

@fatpat fatpat force-pushed the s3PartSize branch 2 times, most recently from 9eae28c to cef749e Compare February 10, 2024 08:36
@56quarters 56quarters marked this pull request as draft February 21, 2024 17:12
@56quarters
Copy link
Contributor

Thanks for the PR! I've converted it to a draft until the upstream Thanos objstore PR is merged.

Copy link
Contributor

@andyasp andyasp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for taking the time to contribute. It would be helpful if you could explain the motivation for adding this option.

pkg/util/objtools/s3.go Outdated Show resolved Hide resolved
CHANGELOG.md Outdated Show resolved Hide resolved
pkg/storage/bucket/s3/config.go Outdated Show resolved Hide resolved
@duricanikolic
Copy link
Contributor

The CHANGELOG has just been cut to prepare for the next Mimir release. Please rebase main and eventually move the CHANGELOG entry added / updated in this PR to the top of the CHANGELOG document. Thanks!

@fatpat fatpat marked this pull request as ready for review May 13, 2024 12:18
@fatpat fatpat requested a review from jdbaldry as a code owner May 13, 2024 12:18
@fatpat
Copy link
Author

fatpat commented May 13, 2024

Thanks for the PR! I've converted it to a draft until the upstream Thanos objstore PR is merged.

upstream PR has been merged and updated in mimir modules.

I put the PR as ready to review

@56quarters 56quarters self-assigned this May 13, 2024
Copy link
Contributor

@56quarters 56quarters left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like you need to run make reference-help to update the CLI flag everywhere. Could you also add this experimental option to about-versioning.md under the ingester section?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants