Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Complete multipart uploads faster on encrypted backends #15375

Merged
merged 2 commits into from
Jul 21, 2022

Conversation

klauspost
Copy link
Contributor

Description

Only fetch the parts we need and abort as soon as one is missing.

Only fetch the number of parts requested by "ListObjectParts".

How to test this PR?

Already test covered

Types of changes

  • Optimization (provides speedup with no functional changes)

Only fetch the parts we need and abort as soon as one is missing.

Only fetch the number of parts requested by "ListObjectParts".
@harshavardhana
Copy link
Member

Some errors because of this @klauspost

=== RUN   TestListObjectPartsDiskNotFound
Formatting 1st pool, 1 set(s), 16 drives per set.
WARNING: Host local has more than 4 drives of set. A host failure will result in data becoming unavailable.
    object-api-multipart_test.go:1364: Test 9: Erasure: Expected to fail with <ERROR> "Invalid upload id abc", but passed instead
    object-api-multipart_test.go:1389: Test 11: Erasure: Expected NextPartNumberMarker to be "3", but instead found it to be "0"
    object-api-multipart_test.go:1401: Test 11: Erasure: Expected IsTruncated to be "true", but found it to "false"
    object-api-multipart_test.go:1405: Test 12: Erasure: Expected the result to contain info of 1 Parts, but found 0 instead
--- FAIL: TestListObjectPartsDiskNotFound (0.56s)
=== RUN   TestListObjectParts
Formatting 1st pool, 1 set(s), 1 drives per set.
WARNING: Host local has more than 0 drives of set. A host failure will result in data becoming unavailable.
Formatting 1st pool, 2 set(s), 16 drives per set.
WARNING: Host local has more than 4 drives of set. A host failure will result in data becoming unavailable.
WARNING: Host local has more than 4 drives of set. A host failure will result in data becoming unavailable.
    object-api-multipart_test.go:1601: Test 9: Erasure: Expected to fail with <ERROR> "Invalid upload id abc", but passed instead
    object-api-multipart_test.go:1635: Test 11: Erasure: Expected IsTruncated to be "true", but found it to "false"
    object-api-multipart_test.go:1645: Test 12: Erasure: Expected the result to contain info of 1 Parts, but found 0 instead
--- FAIL: TestListObjectParts (1.00s)
=== RUN   TestObjectCompleteMultipartUpload
=== RUN   TestObjectCompleteMultipartUpload/default
Formatting 1st pool, 1 set(s), 1 drives per set.```

@minio-trusted
Copy link
Contributor

Mint Automation

Test Result
mint-large-bucket.sh ✔️
mint-fs.sh ✔️
mint-gateway-s3.sh ✔️
mint-erasure.sh ✔️
mint-dist-erasure.sh ✔️
mint-compress-encrypt-dist-erasure.sh ✔️
mint-pools.sh ✔️
Deleting image on docker hub
Deleting image locally

@harshavardhana harshavardhana merged commit 69bf39f into minio:master Jul 21, 2022
@klauspost klauspost deleted the faster-multipart-complete branch July 22, 2022 07:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants