Uploading big files works via web-browser and WebDAV, but not via federation, regardless of http2 module usage #19529

RussianNeuroMancer · 2020-02-18T14:02:48Z

Steps to reproduce

Install Nextcloud 18.0.1 on two/three servers, configure every server for big files upload, share folder from one server to another.
Verify that big files uploading via web-browser and WebDAV actually works on every server. Verify that big file transfer between servers works via scp (or anything else) without issues.
Try to transfer big file from one server to another via shared folder.

Expected behaviour

File transfer is completed without errors, like it was with Nextcloud 16 (Nextcloud 17 wasn't tested).

Actual behaviour

Every attempt to transfer anything bigger than a couple hundreds megabytes end with following error on receiver side:
"Exception":"Sabre\\DAV\\Exception\\BadRequest","Message":"Expected filesize of X bytes but read (from Nextcloud client) and wrote (to Nextcloud storage) Y bytes
Disabling http2 does not help - I disabled http2 module on sender and receiver, restarted Apache 2, and tested file transfer multiple times, it still result in "Expected filesize" error.

At the same time uploading multiple gigabytes via WebDAV and web-browser works without issues, so I assume web-server is configured correctly for big files uploading - it worked flawlessly with Nextcloud 16 after all. Transfer big files via scp between servers works too.

Server configuration

Operating system: Ubuntu 20.04

Web server: Apache 2.4.41

Database: PostgreSQL 12.1

PHP version: PHP 7.3.11

Nextcloud version: 18.0.1.3

Updated from an older Nextcloud/ownCloud or fresh install: updated

Where did you install Nextcloud from: tar

Signing status:

Signing status on every tested server

No errors have been found.

List of activated apps:

App list

Enabled:
  - accessibility: 1.4.0
  - activity: 2.11.0
  - admin_audit: 1.8.0
  - bruteforcesettings: 1.5.0
  - calendar: 2.0.1
  - cloud_federation_api: 1.1.0
  - comments: 1.8.0
  - contacts: 3.1.8
  - dav: 1.14.0
  - federatedfilesharing: 1.8.0
  - federation: 1.8.0
  - files: 1.13.1
  - files_fulltextsearch: 1.4.1
  - files_pdfviewer: 1.7.0
  - files_rightclick: 0.15.2
  - files_sharing: 1.10.1
  - files_trashbin: 1.8.0
  - files_versions: 1.11.0
  - files_videoplayer: 1.7.0
  - firstrunwizard: 2.7.0
  - fulltextsearch: 1.4.1
  - fulltextsearch_elasticsearch: 1.5.0
  - logreader: 2.3.0
  - lookup_server_connector: 1.6.0
  - nextcloud_announcements: 1.7.0
  - notifications: 2.6.0
  - oauth2: 1.6.0
  - password_policy: 1.8.0
  - photos: 1.0.0
  - privacy: 1.2.0
  - provisioning_api: 1.8.0
  - recommendations: 0.6.0
  - serverinfo: 1.8.0
  - settings: 1.0.0
  - sharebymail: 1.8.0
  - support: 1.1.0
  - survey_client: 1.6.0
  - systemtags: 1.8.0
  - text: 2.0.0
  - theming: 1.9.0
  - twofactor_backupcodes: 1.7.0
  - updatenotification: 1.8.0
  - viewer: 1.2.0
  - workflowengine: 2.0.0
Disabled:
  - encryption
  - files_external
  - user_ldap

Nextcloud configuration:

Config report

{
    "system": {
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "***REMOVED SENSITIVE VALUE***",
            "***REMOVED SENSITIVE VALUE***"
        ],
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "overwrite.cli.url": "***REMOVED SENSITIVE VALUE***",
        "dbtype": "pgsql",
        "version": "18.0.1.3",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbtableprefix": "oc_",
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "logtimezone": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "filelocking.enabled": "true",
        "memcache.locking": "\\OC\\Memcache\\Redis",
        "memcache.local": "\\OC\\Memcache\\Redis",
        "redis": {
            "host": "***REMOVED SENSITIVE VALUE***",
            "port": 0,
            "timeout": 0
        },
        "asset-pipeline.enabled": false,
        "enabledPreviewProviders": [
            "OC\\Preview\\PNG",
            "OC\\Preview\\JPEG",
            "OC\\Preview\\GIF",
            "OC\\Preview\\BMP",
            "OC\\Preview\\XBitmap",
            "OC\\Preview\\MP3",
            "OC\\Preview\\MarkDown",
            "OC\\Preview\\MSOffice2003",
            "OC\\Preview\\MSOffice2007",
            "OC\\Preview\\MSOfficeDoc",
            "OC\\Preview\\OpenDocumen",
            "OC\\Preview\\PDF"
        ],
        "appstore.experimental.enabled": false,
        "theme": "",
        "loglevel": 2,
        "maintenance": false,
        "mail_smtpmode": "smtp",
        "mail_smtpsecure": "ssl",
        "mail_from_address": "***REMOVED SENSITIVE VALUE***",
        "mail_domain": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpauthtype": "PLAIN",
        "mail_smtpauth": 1,
        "mail_smtphost": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpport": "465",
        "mail_smtpname": "***REMOVED SENSITIVE VALUE***",
        "mail_smtppassword": "***REMOVED SENSITIVE VALUE***",
        "updater.release.channel": "stable",
        "app_install_overwrite": [
            "calendar"
        ]
    }
}

Are you using external storage, if yes which one: no

Are you using encryption: no

Are you using an external user-backend, if yes which one: no

Client configuration

Browser: another Nextcloud 18.0.1.3 instance

Operating system: Ubuntu 20.04

Logs

Web server error log

no errors in web-server log

Nextcloud log (data/nextcloud.log)

Nextcloud log

[webdav] Fatal: Sabre\DAV\Exception\BadRequest: Expected filesize of 11114631900 bytes but read (from Nextcloud client) and wrote (to Nextcloud storage) 257163264 bytes. Could either be a network problem on the sending side or a problem writing to the storage on the server side. at <<closure>>

0. /var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php line 1130
   OCA\DAV\Connector\Sabre\File->put("*** sensitive parameter replaced ***")
1. /var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/CorePlugin.php line 513
   Sabre\DAV\Server->updateFile("*** sensitive parameters replaced ***")
2. <<closure>>
   Sabre\DAV\CorePlugin->httpPut(Sabre\HTTP\Reque ... "}, Sabre\HTTP\Response {})
3. /var/www/nextcloud/3rdparty/sabre/event/lib/EventEmitterTrait.php line 105
   call_user_func_array([Sabre\DAV\CorePlugin {},"httpPut"], [Sabre\HTTP\Requ ... }])
4. /var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php line 479
   Sabre\Event\EventEmitter->emit("method:PUT", [Sabre\HTTP\Requ ... }])
5. /var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php line 254
   Sabre\DAV\Server->invokeMethod(Sabre\HTTP\Reque ... "}, Sabre\HTTP\Response {})
6. /var/www/nextcloud/apps/dav/appinfo/v1/publicwebdav.php line 109
   Sabre\DAV\Server->exec()
7. /var/www/nextcloud/public.php line 81
   require_once("/var/www/nextcl ... p")

PUT /nextcloud/public.php/webdav/*** sensitive parameters replaced ***
from *** sensitive parameters replaced *** at 2020-02-18T20:53:12+08:00

Browser log

no applicable

The text was updated successfully, but these errors were encountered:

RussianNeuroMancer · 2020-03-24T07:26:10Z

Issue is still reproducible with 18.0.3 on every tested server.

RussianNeuroMancer · 2021-01-21T06:15:28Z

Issue is still reproducible with 20.0.5.

szaimen · 2021-06-22T12:52:48Z

Hi there, are you on NC21.0.2 able to upload those files via public links that you cannot upload via a federated share?

RussianNeuroMancer · 2021-07-01T13:16:15Z

Upgraded to 22.0 RC1 on both tested hosts. Issue is still reproducible. Error message is:

[webdav] Fatal: cURL error 28: Operation timed out after 30000 milliseconds with 214787688 out of 1357353583 bytes received (see https://curl.haxx.se/libcurl/c/libcurl-errors.html) for

ghost · 2021-07-31T13:23:05Z

This issue has been automatically marked as stale because it has not had recent activity and seems to be missing some essential information. It will be closed if no further activity occurs. Thank you for your contributions.

RussianNeuroMancer · 2021-07-31T13:30:31Z

@szaimen what other info is needed? As stated in the issue name "Uploading big files works via web-browser", so I not sure why exactly you asking to check public links if there was no issues with this for me in the first place.

szaimen · 2021-08-08T12:59:14Z

Since federated shares are link shares too, it make sense to ask if link shares work since this rules out a configuration issue

MrRinkana · 2022-08-24T19:45:29Z

If I understand things correctly, this issue, #26071 and #30282 might be duplicates, or at least related to the same thing.

If that is true, I believe #33087 (which is a proposed solution to #30282) might fix all of these.

szaimen · 2023-01-09T15:59:41Z

Hi, please update to 24.0.8 or better 25.0.2 and report back if it fixes the issue. Thank you!

RussianNeuroMancer added 0. Needs triage Pending check for reproducibility or if it fits our roadmap bug labels Feb 18, 2020

kesselb added 18-feedback feature: federation labels Feb 18, 2020

nickvergessen removed the 18-feedback label Jul 10, 2020

szaimen added the needs info label Jun 22, 2021

ghost added the stale Ticket or PR with no recent activity label Jul 31, 2021

ghost removed the stale Ticket or PR with no recent activity label Jul 31, 2021

szaimen added 1. to develop Accepted and waiting to be taken care of and removed needs info 0. Needs triage Pending check for reproducibility or if it fits our roadmap labels Aug 8, 2021

szaimen added needs info 0. Needs triage Pending check for reproducibility or if it fits our roadmap and removed 1. to develop Accepted and waiting to be taken care of labels Jan 9, 2023

szaimen closed this as completed Mar 6, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uploading big files works via web-browser and WebDAV, but not via federation, regardless of http2 module usage #19529

Uploading big files works via web-browser and WebDAV, but not via federation, regardless of http2 module usage #19529

RussianNeuroMancer commented Feb 18, 2020

RussianNeuroMancer commented Mar 24, 2020

RussianNeuroMancer commented Jan 21, 2021

szaimen commented Jun 22, 2021

RussianNeuroMancer commented Jul 1, 2021

ghost commented Jul 31, 2021

RussianNeuroMancer commented Jul 31, 2021

szaimen commented Aug 8, 2021

MrRinkana commented Aug 24, 2022

szaimen commented Jan 9, 2023

Uploading big files works via web-browser and WebDAV, but not via federation, regardless of http2 module usage #19529

Uploading big files works via web-browser and WebDAV, but not via federation, regardless of http2 module usage #19529

Comments

RussianNeuroMancer commented Feb 18, 2020

Steps to reproduce

Expected behaviour

Actual behaviour

Server configuration

Client configuration

Logs

Web server error log

Nextcloud log (data/nextcloud.log)

Browser log

RussianNeuroMancer commented Mar 24, 2020

RussianNeuroMancer commented Jan 21, 2021

szaimen commented Jun 22, 2021

RussianNeuroMancer commented Jul 1, 2021

ghost commented Jul 31, 2021

RussianNeuroMancer commented Jul 31, 2021

szaimen commented Aug 8, 2021

MrRinkana commented Aug 24, 2022

szaimen commented Jan 9, 2023