New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[12] Update aws sdk + s3 improvements #6737

Merged
merged 16 commits into from Oct 9, 2017

Conversation

Projects
None yet
5 participants
@icewind1991
Member

icewind1991 commented Oct 3, 2017

Backport of #4410 and #6602

icewind1991 added some commits Apr 20, 2017

remove non composer aws sdk
Signed-off-by: Robin Appelman <robin@icewind.nl>
update aws sdk and move it to 3rdparty
Signed-off-by: Robin Appelman <robin@icewind.nl>
reuse object read/write/delete logic in s3 implementations
Signed-off-by: Robin Appelman <robin@icewind.nl>
more efficient write only fopen
Signed-off-by: Robin Appelman <robin@icewind.nl>
more efficient permissions for s3 external storage
Signed-off-by: Robin Appelman <robin@icewind.nl>
more efficient is_dir
Signed-off-by: Robin Appelman <robin@icewind.nl>
add stat cache for s3 external storage
Signed-off-by: Robin Appelman <robin@icewind.nl>
disable part files for s3 external storage
Signed-off-by: Robin Appelman <robin@icewind.nl>
fix path style
Signed-off-by: Robin Appelman <robin@icewind.nl>
merge vendor js
Signed-off-by: Robin Appelman <robin@icewind.nl>
fix undefined constants
Signed-off-by: Robin Appelman <robin@icewind.nl>
update autoloader
Signed-off-by: Robin Appelman <robin@icewind.nl>
show diff in autoload checker
Signed-off-by: Robin Appelman <robin@icewind.nl>
@mention-bot

This comment has been minimized.

Show comment
Hide comment
@mention-bot

mention-bot Oct 3, 2017

@icewind1991, thanks for your PR! By analyzing the history of the files in this pull request, we identified @Xenopathic to be a potential reviewer.

mention-bot commented Oct 3, 2017

@icewind1991, thanks for your PR! By analyzing the history of the files in this pull request, we identified @Xenopathic to be a potential reviewer.

icewind1991 added some commits Sep 18, 2017

multipart upload for s3 object storage
Signed-off-by: Robin Appelman <robin@icewind.nl>
only do multipart upload for large files
Signed-off-by: Robin Appelman <robin@icewind.nl>
Add tests for multipart upload
Signed-off-by: Robin Appelman <robin@icewind.nl>
@codecov

This comment has been minimized.

Show comment
Hide comment
@codecov

codecov bot Oct 9, 2017

Codecov Report

Merging #6737 into stable12 will decrease coverage by 0.01%.
The diff coverage is n/a.

@@              Coverage Diff               @@
##             stable12    #6737      +/-   ##
==============================================
- Coverage       53.78%   53.76%   -0.02%     
+ Complexity      22559    22554       -5     
==============================================
  Files            1383     1384       +1     
  Lines           86586    86546      -40     
  Branches         1328     1328              
==============================================
- Hits            46566    46531      -35     
+ Misses          40020    40015       -5
Impacted Files Coverage Δ Complexity Δ
lib/private/Files/ObjectStore/S3.php 50% <0%> (-23.69%) 2% <0%> (-3%)
lib/private/RichObjectStrings/Validator.php 78.26% <0%> (-3.89%) 13% <0%> (ø)
lib/private/Group/Group.php 87.23% <0%> (-1.66%) 35% <0%> (-17%)
lib/private/legacy/template/functions.php 7.14% <0%> (ø) 0% <0%> (ø) ⬇️
...te/Authentication/LoginCredentials/Credentials.php 100% <0%> (ø) 2% <0%> (-2%) ⬇️
...s/files_external/lib/Lib/Storage/StreamWrapper.php 0% <0%> (ø) 14% <0%> (-5%) ⬇️
apps/files_external/lib/Lib/Storage/AmazonS3.php 0% <0%> (ø) 110% <0%> (+18%) ⬆️
lib/private/Files/ObjectStore/S3ObjectTrait.php 85.71% <0%> (ø) 3% <0%> (?)
lib/private/Security/IdentityProof/Signer.php 80.64% <0%> (+0.64%) 7% <0%> (ø) ⬇️
...ib/private/Files/ObjectStore/S3ConnectionTrait.php 57.77% <0%> (+3.61%) 23% <0%> (+1%) ⬆️

codecov bot commented Oct 9, 2017

Codecov Report

Merging #6737 into stable12 will decrease coverage by 0.01%.
The diff coverage is n/a.

@@              Coverage Diff               @@
##             stable12    #6737      +/-   ##
==============================================
- Coverage       53.78%   53.76%   -0.02%     
+ Complexity      22559    22554       -5     
==============================================
  Files            1383     1384       +1     
  Lines           86586    86546      -40     
  Branches         1328     1328              
==============================================
- Hits            46566    46531      -35     
+ Misses          40020    40015       -5
Impacted Files Coverage Δ Complexity Δ
lib/private/Files/ObjectStore/S3.php 50% <0%> (-23.69%) 2% <0%> (-3%)
lib/private/RichObjectStrings/Validator.php 78.26% <0%> (-3.89%) 13% <0%> (ø)
lib/private/Group/Group.php 87.23% <0%> (-1.66%) 35% <0%> (-17%)
lib/private/legacy/template/functions.php 7.14% <0%> (ø) 0% <0%> (ø) ⬇️
...te/Authentication/LoginCredentials/Credentials.php 100% <0%> (ø) 2% <0%> (-2%) ⬇️
...s/files_external/lib/Lib/Storage/StreamWrapper.php 0% <0%> (ø) 14% <0%> (-5%) ⬇️
apps/files_external/lib/Lib/Storage/AmazonS3.php 0% <0%> (ø) 110% <0%> (+18%) ⬆️
lib/private/Files/ObjectStore/S3ObjectTrait.php 85.71% <0%> (ø) 3% <0%> (?)
lib/private/Security/IdentityProof/Signer.php 80.64% <0%> (+0.64%) 7% <0%> (ø) ⬇️
...ib/private/Files/ObjectStore/S3ConnectionTrait.php 57.77% <0%> (+3.61%) 23% <0%> (+1%) ⬆️
@icewind1991

This comment has been minimized.

Show comment
Hide comment
@icewind1991
Member

icewind1991 commented Oct 9, 2017

@rullzer @schiessle please review

@rullzer

This comment has been minimized.

Show comment
Hide comment
@rullzer

rullzer Oct 9, 2017

Member

Why?

This only fixes something that never worked before right (>5gb uploads)? So it is a new feature. Why backport? Also I'm really not that comfortable merging such huge changes into a maintenance release.

Member

rullzer commented Oct 9, 2017

Why?

This only fixes something that never worked before right (>5gb uploads)? So it is a new feature. Why backport? Also I'm really not that comfortable merging such huge changes into a maintenance release.

@karlitschek

This comment has been minimized.

Show comment
Hide comment
@karlitschek

karlitschek Oct 9, 2017

Member

This needs to be backported because of an external requirement.

Member

karlitschek commented Oct 9, 2017

This needs to be backported because of an external requirement.

@icewind1991 icewind1991 merged commit df2fa2f into stable12 Oct 9, 2017

1 check failed

continuous-integration/drone/pr the build failed
Details

@icewind1991 icewind1991 deleted the aws-sdk-12 branch Oct 9, 2017

@MorrisJobke MorrisJobke referenced this pull request Nov 20, 2017

Merged

12.0.4 RC #7225

1 of 1 task complete
@MorrisJobke

This comment has been minimized.

Show comment
Hide comment
@MorrisJobke

MorrisJobke Dec 19, 2017

Member

As this does not have a single review and it caused serious damage on a minor upgrade from 12.0.3 to 12.0.4 (see #7569) I will open a revert for this PR.

Member

MorrisJobke commented Dec 19, 2017

As this does not have a single review and it caused serious damage on a minor upgrade from 12.0.3 to 12.0.4 (see #7569) I will open a revert for this PR.

@MorrisJobke

This comment has been minimized.

Show comment
Hide comment
@MorrisJobke

MorrisJobke Dec 19, 2017

Member

As this does not have a single review and it caused serious damage on a minor upgrade from 12.0.3 to 12.0.4 (see #7569) I will open a revert for this PR.

@icewind1991 Could you take care of this as it seems to be non-trivial, because other backports like #7186 and #7079 went in as well.

Member

MorrisJobke commented Dec 19, 2017

As this does not have a single review and it caused serious damage on a minor upgrade from 12.0.3 to 12.0.4 (see #7569) I will open a revert for this PR.

@icewind1991 Could you take care of this as it seems to be non-trivial, because other backports like #7186 and #7079 went in as well.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment