[Bug]: Upload using web interface has continuous long pauses without doing anything

[JohnFarl]

⚠️ This issue respects the following points: ⚠️

• This is a bug, not a question or a configuration/webserver/proxy issue.
• This issue is not already reported on Github (I've searched it).
• Nextcloud Server is up to date. See Maintenance and Release Schedule for supported versions.
• Nextcloud Server is running on 64bit capable CPU, PHP and OS.
• I agree to follow Nextcloud's Code of Conduct.

Bug description

During the upload of big files using web interface there are continuous insanely long pauses (pauses last several minutes not just few seconds) that make large files upload insanely slow.

Searching among forum some people suggested in similar cases that this is due chunks processing time, but I have carefully checked running processes and resources usage on the server to try to understand where is the sever bottleneck that cause these long pauses during upload but with my surprise CPU and disk usage are nearly idle status, so these delays cannot be justified with "server it is busy processing chunks" because this is not the case and server resources are blatantly unused while Nextcloud upload "sleep" in these pauses.

The problem isn't client side because if I try exactly the same upload with a different cloud provider e.g. OneDrive or Google Drive , it goes smooth without any of these weird pauses.

I use Cloudflare and my plan has a limit of 100MB max for each upload request but max chunk size is 100000000 byte so this shouldn't be a problem.

The cyclicality of the pauses suggests this is related with the chunks (the pauses happens at every 95.4MB uploaded with my chunk size set to 100000000 byte), but in any case, it is a bug because there isn't any resource limit or overuse that could justify these pauses.

Nextcloud version 25

Steps to reproduce

1. Open Nextcloud web interface in a browser
2. Upload large file with drag and drop

Expected behavior

Smooth upload without significant pauses.

Installation method

Community Manual installation with Archive

Operating system

Debian/Ubuntu

PHP engine version

PHP 7.4

Web server

Apache (supported)

Database engine version

MariaDB

Is this bug present after an update or on a fresh install?

Fresh Nextcloud Server install

Are you using the Nextcloud Server Encryption module?

Encryption is Disabled

What user-backends are you using?

• Default user-backend (database)
• LDAP/ Active Directory
• SSO - SAML
• Other

Configuration report

No response

List of activated Apps

- activity: 2.17.0
  - bruteforcesettings: 2.5.0
  - calendar: 4.0.1
  - circles: 25.0.0
  - cloud_federation_api: 1.8.0
  - comments: 1.15.0
  - contacts: 5.0.1
  - contactsinteraction: 1.6.0
  - dashboard: 7.5.0
  - dav: 1.24.0
  - federatedfilesharing: 1.15.0
  - federation: 1.15.0
  - files: 1.20.1
  - files_external: 1.17.0
  - files_pdfviewer: 2.6.0
  - files_rightclick: 1.4.0
  - files_sharing: 1.17.0
  - files_trashbin: 1.15.0
  - files_versions: 1.18.0
  - firstrunwizard: 2.14.0
  - logreader: 2.10.0
  - lookup_server_connector: 1.13.0
  - nextcloud_announcements: 1.14.0
  - notifications: 2.13.1
  - oauth2: 1.13.0
  - password_policy: 1.15.0
  - photos: 2.0.0
  - privacy: 1.9.0
  - provisioning_api: 1.15.0
  - recommendations: 1.4.0
  - related_resources: 1.0.1
  - richdocuments: 7.0.0
  - serverinfo: 1.15.0
  - settings: 1.7.0
  - sharebymail: 1.15.0
  - spreed: 15.0.0
  - support: 1.8.0
  - survey_client: 1.13.0
  - systemtags: 1.15.0
  - text: 3.6.0
  - theming: 2.0.0
  - twofactor_backupcodes: 1.14.0
  - updatenotification: 1.15.0
  - user_status: 1.5.0
  - viewer: 1.9.0
  - weather_status: 1.5.0
  - workflowengine: 2.7.0

Nextcloud Signing status

No response

Nextcloud Logs

No response

Additional info

No response

[PVince81]

Theory: you see the progress bar advance while the buffers are flushed through the network, even when the data hasn't fully reached the PHP process. Then once the PHP process starts working, this seems to cause a delay.

Note: you cannot compare the upload behavior with other solutions as the deployments and frameworks (non-PHP) used there are different

[PVince81]

I have tested this locally with a 400 MB file and I don't see those pauses between chunks, so it's definitely not a new bug.
The only known pause is at the very end during final assembly (big files only), which is a known issue.

Now if you're uploading multiple files that are bigger than 10 MB, it will use chunking and you'll see "pauses in the middle of the progress bar" that correspond to the chunk assembly of each of the files.

Note: you can also increase the chunk size for the web UI here: https://docs.nextcloud.com/server/latest/admin_manual/configuration_files/big_file_upload_configuration.html#adjust-chunk-size-on-nextcloud-side

If your provider allows up to 100 MB you could set that value there so that you don't get those chunk assembly pauses for files < 100 MB.

[PVince81]

ref: some research about making chunk assembly asynchronous: #28395

[szaimen]

Can you check if that fixes your problem?
nextcloud/documentation#9578 (comment)

[nextcloud-command]

This issue has been automatically marked as stale because it has not had recent activity and seems to be missing some essential information. It will be closed if no further activity occurs. Thank you for your contributions.