Improve chunk upload AssemblyStream performance #26062

Merged
merged 1 commit into from Sep 8, 2016

Projects

None yet

4 participants

@PVince81
Collaborator
PVince81 commented Sep 8, 2016

Description

The assembly stream's steam_read method is usually called for every 8192 bytes by PHP.
In the current code, it would call fopen() on the chunk's node in every call, which means for every 8k bytes, which is very unefficient.

This fix caches the stream until the end is reached before resolving the next node.

Related Issue

Partial fix for #25493

Motivation and Context

Chunk assembly is too slow in the new endpoint!

How Has This Been Tested?

Check out this other PR #25494 which contains the fix as well.
Upload a big file of around 60+ MB in the web UI in a subfolder.
Check the network console for the final "MOVE" of ".file".

Before: 20440ms
After: 600ms

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

  • My code follows the code style of this project.
  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.
  • I have read the CONTRIBUTING document.
  • I have added tests to cover my changes.
  • All new and existing tests passed.

Backports:

  • stable9.1 ?

Please review @DeepDiver1975 @owncloud/filesystem

@PVince81 PVince81 Improve chunk upload AssemblyStream performance
3ddc652
@PVince81 PVince81 added this to the 9.2 milestone Sep 8, 2016
@mention-bot

@PVince81, thanks for your PR! By analyzing the annotation information on this pull request, we identified @DeepDiver1975, @LukasReschke and @nickvergessen to be potential reviewers

@guruz guruz referenced this pull request in owncloud/client Sep 8, 2016
Merged

Chunking ng #5102

@guruz
Contributor
guruz commented Sep 8, 2016

👍

@DeepDiver1975
Member

👍

@DeepDiver1975 DeepDiver1975 merged commit fd37042 into master Sep 8, 2016

4 of 5 checks passed

VersionEye Some dependencies have no license.
Details
Scrutinizer 2 new issues, 1 updated code elements
Details
continuous-integration/jenkins/pr This commit looks good
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
licence/cla Contributor License Agreement is signed.
Details
@DeepDiver1975 DeepDiver1975 deleted the dav-fixassemblystreamperf branch Sep 8, 2016
@PVince81 PVince81 added a commit that referenced this pull request Sep 9, 2016
@PVince81 PVince81 Improve chunk upload AssemblyStream performance (#26062) 4038798
@PVince81
Collaborator
PVince81 commented Sep 9, 2016

stable9.1: #26072

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment