Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ensure non-trailing parts are of equal size #50

Merged
merged 1 commit into from Jul 5, 2023

Conversation

JohnNiang
Copy link
Member

What type of PR is this?

/kind improvement

What this PR does / why we need it:

Reshape the DataBuffers into parts of the same size (5MB) except for the last part.

Which issue(s) this PR fixes:

Fixes #49

Does this PR introduce a user-facing change?

保证分片上传时片段大小一致

@f2c-ci-robot f2c-ci-robot bot added kind/improvement Categorizes issue or PR as related to a improvement. release-note Denotes a PR that will be considered when it comes time to generate release notes. labels Jul 5, 2023
@f2c-ci-robot f2c-ci-robot bot requested review from longjuan and wan92hen July 5, 2023 04:30
@longjuan
Copy link
Member

longjuan commented Jul 5, 2023

MULTIPART_MIN_PART_SIZE改成 8M ,兼容Ucloud?

@JohnNiang
Copy link
Member Author

MULTIPART_MIN_PART_SIZE改成 8M ,兼容Ucloud?

建议单独提交 PR 更改这个 part size。当前 PR 需要解决的问题是保证分片大小一致。

Copy link
Member

@ruibaby ruibaby left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/approve

实际测试 20m 左右的文件可以正常上传。

@f2c-ci-robot f2c-ci-robot bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jul 5, 2023
@longjuan
Copy link
Member

longjuan commented Jul 5, 2023

/lgtm
600M文件成功上传,无 OOM

@f2c-ci-robot f2c-ci-robot bot added the lgtm Indicates that a PR is ready to be merged. label Jul 5, 2023
@f2c-ci-robot
Copy link

f2c-ci-robot bot commented Jul 5, 2023

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: guqing, ruibaby

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@f2c-ci-robot f2c-ci-robot bot merged commit 00537c1 into halo-dev:main Jul 5, 2023
1 check passed
@JohnNiang JohnNiang deleted the feat/same-size-part branch July 5, 2023 05:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. kind/improvement Categorizes issue or PR as related to a improvement. lgtm Indicates that a PR is ready to be merged. release-note Denotes a PR that will be considered when it comes time to generate release notes.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Failed to complete multipart upload sometimes when storage vendors require the same size parts
4 participants