Error: Sabre\DAV\Exception\BadRequest: Expected filesize of... #21382
Comments
nicrame
added
0. Needs triage
|
Getting the same right now: Chrome on Huawei P20 Pro Android 10 fails with below errors. Together with Setup is on Nextcloud 19.0 on Docker with mysql/redis/nginx/php-fpm behind a Traefik 2. I'll debug this a bit further and add informations as I find them. |
|
I also notice the same problem on only one instance of a shared server (several instances deployed).
Error is similar to described above: |
|
Same here:
Error: PHP DB System Client system: MS Windows 10 |
|
@nicrame @KoffeinKaio @jaalzateolaya @Esprit-Libre Try to place the source code and data repository on different disks, most possibly it was caused by IO limitation, even in some cases the website will not respond. Another solution is to turn on OPcache, also help |
|
On my side, OPCache is on. IO disk limitation... I will check: any idea to have relevant metrics? ( But strange thing is that problem occurs mainly (not 100% but nearly) with one instance, and not with the others on the same server. So I'm not confident that this is IO disk limitation. |
Aside of WebDAV syncing, I have another similar issue, php-fpm got stuck when I watch videos...But finally all were solved by placing source code and data into different disks. In terms of disk utilisation, first you can check by But if all look good, good luck to you haha, because I have no other ideas. |
From the beginning i got different HDD for OS, and different for Nextcloud with data directory. Also i got enabled OPCache and APCu. I've got only 4 users, and most of the time Nextcloud do nothing. Sometimes this error occurs, when some file is uploaded, but even then there is only that 1 file transfer :S |
|
I experience the same problem with NC 18.0.4 on mod-php 7.3 and Client 2.6.4 with SMB/CIFS Share External Storage. Until now i do not know where the problem/disconnect while upload occurs. A Timeout/Blocker of PHP/Apache2 or something behind towards the External Storage... All the uploaders are using Notebooks with LTE connection, the broken uploads are often only 500kb. It happens here very often and is going to be a real problem productively. Example Error Message:
|
|
I'm having the same issue with the webclient. Also reported this issue on nextcloud foruns, but so far, no help, https://help.nextcloud.com/t/timeouts-while-uploading-multiple-files/90717 |
|
"me too" ... more information/data : SAMPLE:
Version info:
config Other Observations / Tests
EDIT/UPDATE: on further observation, actually, the Apache2 proxy logs DO HAVE these: this seems to roughly correlate with the errors seen in Nextcloud backend; (they are not 1:1 or perfect correlation, BUT, the timestamps align) ❗ perhaps in my case (and for others) if you too are using Apache2 reverse proxy for Nextcloud, ... if the proxy timeout itself is killing the connection / OR / nextcloud is killing/resetting the connection ... this might be the cause? (TBD on solution) TESTING:
|
|
We are moving our production to seafile after weeks of troubleshooting with no result, due to this unresolveable bug, 30% moved already. I'll no longer help here with logs and configs -.- Edit: fyi, now with seafile we are avoiding webdav+sabredav completely, the sync from linux seafile server to windows share is done by their original client prepared as a windows service. |
|
Is this Issue still valid in NC21.0.2? If not, please close this issue. Thanks! :) |
Yes! I installed manually and also via nextcloudvm script and they both have this issue! |
szaimen
added
1. to develop
|
FYI : I have the issue on my server :
Distribution: Debian Buster Operating system: Linux 4.19.0-17-amd64 #1 SMP Debian 4.19.194-2 (2021-06-21) x86_64 Webserver: Apache (apache2handler) Database: mysql 10.3.29 (Mariadb) PHP version: 7.3.29-1~deb10u1 Modules loaded: Core, date, libxml, openssl, pcre, zlib, filter, hash, Reflection, SPL, sodium, session, standard, apache2handler, mysqlnd, PDO, xml, apcu, bcmath, bz2, calendar, ctype, curl, dom, mbstring, fileinfo, ftp, gd, gettext, gmp, iconv, igbinary, imagick, intl, json, exif, msgpack, mysqli, pdo_mysql, apc, posix, readline, redis, shmop, SimpleXML, sockets, sysvmsg, sysvsem, sysvshm, tokenizer, wddx, xmlreader, xmlwriter, xsl, zip, Phar, memcached, Zend OPcache Nextcloud version: 21.0.3 - 21.0.3.1
Where did you install Nextcloud from: Initialy = Tar + followup install process. Signing statusArray ( ) List of activated appsEnabled:
Configuration (config/config.php){ External storages: yes External storage configurationNo mounts configured Encryption: no User-backends: OC\User\Database Browser: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:90.0) Gecko/20100101 Firefox/90.0 |
|
Does disabling HTTP/2 solve the problem (as proposed in #16138)? |
The module http2 was not enabled on my configuration. |
|
I also have disabled http2 on my Nginx. |
|
Was there a fix to this? I also don't have http2 enabled and have this error Edit: NC22.1.1 stable |
|
I had the same issue, and I found the problem. In my case it was an unexpected place where the tmp files were written, which ran out of space. PHP writes uploads to the tmp folder. I run docker, which means that by default it would get written to the /tmp directory within that container. The containers filesystem is present on my / mount, which is a 10GB disk just enough to run Docker with Nextcloud. In the start I tried to upload larger files (1GB+), but this would work - although now this drive is filling up a bit more due to automated updates with watchtower and logs. This would mean that the available space was just a couple of 100MB's, resulting in this error. I fixed it by mapping the /tmp directory from within the container to my "data" mount (where all data is stored as wel and where I have enough storage available). The only downside is in this case the tmp data is first written to the HDD's and then again written to the same HDD's. This could slow it down a bit, as we need to read AND write to the same HDD's. It may be useful just to expand the root mount, as this is SSD based. If you don't have docker and have tmpfs on /tmp, it could also be that you're running out of RAM. |
|
Same issue here, Nextcloud 21.0.7 Modifying chunks size ? sudo -u www-data php occ config:app:set files max_chunk_size --value 1048000 |
szaimen
added
needs info
0. Needs triage
|
Hi there, @szaimen Simon, |
|
I think most likely is also the Nextcloud VM not following all things mentioned in https://docs.nextcloud.com/server/latest/admin_manual/configuration_files/big_file_upload_configuration.html |
|
Simon, so far modified everything that we recommended and suggested. |
|
So you modified the config manually and adjusted each of the values like and it still does not work? Are you using a reverse proxy maybe? You will need to adjust the values in there as well. Also please make sure that the timeouts are high enough. |
|
Simon, CPU 8 Core // 32GB RAM |
|
These look suspicious to me but I don't have the time to help you finding out the best values for your specific case. |
|
Simon this is rather suboptimal. thx for youre time and help here. |
Best is to contact https://nextcloud.com/de/contact/ then as they will have the time to help you and find out the best values for your specific case. |
|
Apart from that there is this: https://github.com/nextcloud/all-in-one which should handle things correctly out of the box. And https://help.nextcloud.com for help. |
|
Hope this help a little, I was looking how to fix this issue for 6 hours. my server is a 96GB RAM with 24 core CPU. my working pc is a windows 11 connected to a 100 MBps ethernet cable connected to a Cisco switch port. I tested hard disk and change many php and nginx settings but nothing works, So I change the atached port of my pc to a 1000 Mbps ports and then it works perfectly fine.. So I guess it's related to weird incompatibility with nextcloud client and ethernet card or even windows drivers? |
This points indeed towards a bad ethernet card or windows drivers indeed. |
|
I also had this issue with a fresh deployed Nextcloud 25.0.3 and even small files.
Hope this will help some of you. |
|
Thanks @chrisv5179 for the update! |
|
As this seems to be a setup issue I would like to ask you to raise your question in the forums: https://help.nextcloud.com |
|
Hi Same problem here but crazy thing. It's ok with CHROME but not with FIREFOX !
Debian 11 |
|
We have the same problem with the Nextcloud 25.0.4.1 Docker image. |
|
Feel free to create a new issue here: https://github.com/nextcloud/docker/issues |
|
I also encountered this error. It takes me hours to find out the problem. In my case, it is a problem for the client. I use Nextcloud for Mac. However, the default power management settings for MacBook will put the system to sleep when the display is off (even when powered on and the lid is open). When the Mac is sleeping, the network will disconnect periodically. If I prevent my mac from sleeping, this error never occurs again. |
|
Same problem here with Nexctloud 27.0.0 All-in-One and a cifs share as nextcloud data dir |
|
Quiz question. Do you use Nod32 ? ( or smart security by ESET ) |
|
none of them ;) why? as mentioned by @szaimen further down I probably found a solution for me. just wanted to go with a few more tests before making it final. but looks still promising. |
|
because I discovered that at home, the problem comes from NOD32 and the analysis of SSL traffic. if I disable NOD32 to upload, no worries |
|
Apparently this was solved with https://help.nextcloud.com/t/sabre-dav-exception-datadir-on-external-storage/166127/10 |
|
Problem still present. First I had the 0 byte error with the new apache default setting, then random numbers like this issue. NC 26.5. on a Debian 11 VM running nextcloud in HTTP on Apache2, behind a nginx reverse proxy. I've noticed a very distinct behaviour if the file is uploaded in a public share folder vs if the file is uploaded in the same folder but through its owner folder tree. In the first case on Firefox it might create an infinite loop where it always try to reupload said file. On chrome it will give a proper error message if on the public share folder, and if uploaded through a user directory I will get the "is locked" error, but then after a refresh the file is now present and well. Interestingly enough Chrome upload is WAY slower (talking like 3x time slower). I've checked everything from all the nginx config to all the php setting possible, from the apache settings too, there's not a single thread on the matter that I haven't found and i'm kinda hopeless tbh. I'm just starting to think NC is not designed for big files. |
|
problem is back for me :'( |
and others
Uh oh!
There was an error while loading. Please reload this page.
I've got folder with about 31GB (116 633 files and 29 052 folders). I selected it to sync with folder inside Nextcloud app.
On maaaany files there is Sabre\DAV\Exception\BadRequest error. Files with different size, and different types.
There are also other problems i have that i think are not very problematic so i will give them on the end of this report.
Proxy in client is disabled. Connection between PC and Nextcloud box is 1Gb ethernet on both sides. The PC (win10 with desktop client) got 64GB RAM with i7 9gen CPU, the Nextcloud (CentOS 8) box is 8GB RAM, with J4105 CPU, and two HDD's in softraid (raid 1).
Tried so far without success: Disable SELinux, PHP downgrade to 7.3 (REMI's repo), MariaDB downgrade from 10.4.13 (MariaDB repo) to older (from CentOS official repo).
Change Linux distro into CentOS 7, with PHP7.3, nginx 1.16.1, older database and still no success!
Removing redis from config is some kind of solution but it slowdowns everything a lot.
My specific PHP/MariaDB configs that may have something to do.
/etc/php.ini:
memory_limit = 1024M
max_execution_time = 60
max_input_time = 120
post_max_size = 16384M
upload_max_filesize = 16384M
default_socket_timeout = 120
mysqli.cache_size = 2000
/etc/php-fpm.d/www.conf
pm.start_servers = 10
pm.max_children = 100
/etc/my.cnf.d/server.cnf
[server]
skip-name-resolve
innodb_flush_log_at_trx_commit = 2
innodb_log_buffer_size = 32M
innodb_max_dirty_pages_pct = 90
query_cache_type = 1
query_cache_limit = 2M
query_cache_min_res_unit = 2k
query_cache_size = 64M
tmp_table_size= 64M
max_heap_table_size= 64M
slow-query-log = 1
slow-query-log-file = /var/log/mysql/slow.log
long_query_time = 1
[mysqld]
innodb_buffer_pool_size=1G
innodb_io_capacity=4000
Steps to reproduce
Expected behaviour
Find no errors while uploading files.
Actual behaviour
Getting Sabre\DAV\Exception\BadRequest errors.
Sabre\DAV\Exception\BadRequest: Expected filesize of 35358091 bytes but read (from Nextcloud client) and wrote (to Nextcloud storage) 14827520 bytes. Could either be a network problem on the sending side or a problem writing to the storage on the server side. /var/www/nextcloud/apps/dav/lib/Connector/Sabre/Directory.php - line 154:OCA\DAV\Connector\Sabre\File->put() /var/www/nextcloud/apps/dav/lib/Upload/UploadFolder.php - line 46:OCA\DAV\Connector\Sabre\Directory->createFile() /var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php - line 1104:OCA\DAV\Upload\UploadFolder->createFile()/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/CorePlugin.php - line 527:Sabre\DAV\Server->createFile() /var/www/nextcloud/3rdparty/sabre/event/lib/WildcardEmitterTrait.php - line 89:Sabre\DAV\CorePlugin->httpPut() /var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php - line 474:Sabre\DAV\Server->emit() /var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php - line 251:Sabre\DAV\Server->invokeMethod() /var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php - line 319:Sabre\DAV\Server->start() /var/www/nextcloud/apps/dav/lib/Server.php - line 320:Sabre\DAV\Server->exec() /var/www/nextcloud/apps/dav/appinfo/v2/remote.php - line 35:OCA\DAV\Server->exec() /var/www/nextcloud/remote.php - line 167:require_once("/var/www/ne ... p")Screenshots of Nextcloud:
Screenshots of Client:
Server configuration
Operating system:
cat /etc/redhat-release
CentOS Linux release 8.1.1911 (Core)
uname -a
Linux nextcloud 4.18.0-147.8.1.el8_1.x86_64 #1 SMP Thu Apr 9 13:49:54 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Web server:'
nginx -V
nginx version: nginx/1.14.1
built by gcc 8.2.1 20180905 (Red Hat 8.2.1-3) (GCC)
built with OpenSSL 1.1.1 FIPS 11 Sep 2018 (running with OpenSSL 1.1.1c FIPS 28 May 2019)
TLS SNI support enabled
Database:
mariadb -V
mariadb Ver 15.1 Distrib 10.4.13-MariaDB, for Linux (x86_64) using readline 5.1
PHP version:
PHP-FPM connected with nginx thru socket
Version: 7.4.7
Memory Limit: 1 GB
Max Execution Time: 3600
Upload max size: 16 GB
Nextcloud version: (see Nextcloud admin page)
19.0.0.12
Updated from an older Nextcloud/ownCloud or fresh install:
Fresh install
Where did you install Nextcloud from:
nextcloud.com
Signing status:
No errors have been found.
List of activated apps:
sudo -u nginx php occ app:list
Enabled:
Disabled:
Nextcloud configuration:
sudo -u nginx php occ config:list system
{
"system": {
"blacklisted_files": [],
"instanceid": "REMOVED SENSITIVE VALUE",
"passwordsalt": "REMOVED SENSITIVE VALUE",
"secret": "REMOVED SENSITIVE VALUE",
"trusted_domains": [
"localhost",
"nextcloud",
"127.0.0.1",
"192.168.50.2",
],
"datadirectory": "REMOVED SENSITIVE VALUE",
"dbtype": "mysql",
"version": "19.0.0.12",
"overwrite.cli.url": "http://nextcloud",
"dbname": "REMOVED SENSITIVE VALUE",
"dbhost": "REMOVED SENSITIVE VALUE",
"dbport": "",
"dbtableprefix": "1c_",
"mysql.utf8mb4": true,
"dbuser": "REMOVED SENSITIVE VALUE",
"dbpassword": "REMOVED SENSITIVE VALUE",
"installed": true,
"default_language": "pl",
"default_locale": "pl",
"memcache.local": "\OC\Memcache\Redis",
"filelocking.enabled": true,
"memcache.locking": "\OC\Memcache\Redis",
"memcache.distributed": "\OC\Memcache\Redis",
"redis": {
"host": "REMOVED SENSITIVE VALUE",
"port": 6379,
"dbindex": 0,
"timeout": 600
},
"skeletondirectory": "core/empty-default",
"maintenance": false
}
}
Are you using external storage, if yes which one: local/smb/sftp/...
There is smb plugin with attached storage, but it is not used for sync.
Are you using encryption:
no
Are you using an external user-backend, if yes which one: LDAP/ActiveDirectory/Webdav/...
no
Client configuration
Browser:
Using Nextcloud desktop client.
Operating system:
Windows 10 Pro x64
Logs
Server respond "423 Locked" on "PUT http://192.168.50.2/remote.php/dav/files/USERNAME/dire...
Web server error log
nginx error.log is empty.
Nextcloud log (data/nextcloud.log)
{"reqId":"aG0WKf6Jk3WBKZzcfG5l","level":3,"time":"2020-06-12T10:10:42+00:00","remoteAddr":"192.168.50.5","user":"USERNAME","app":"no app in context","method":"PUT","url":"/remote.php/dav/uploads/USERNAME/2948563684/00000000","message":{"Exce
ption":"Sabre\DAV\Exception\BadRequest","Message":"Expected filesize of 35358091 bytes but read (from Nextcloud client) and wrote (to Nextcloud storage) 14827520 bytes. Could either be a network problem on the sending side or a proble
m writing to the storage on the server side.","Code":0,"Trace":[{"file":"/var/www/nextcloud/apps/dav/lib/Connector/Sabre/Directory.php","line":154,"function":"put","class":"OCA\DAV\Connector\Sabre\File","type":"->"},{"file":"/var/www
/nextcloud/apps/dav/lib/Upload/UploadFolder.php","line":46,"function":"createFile","class":"OCA\DAV\Connector\Sabre\Directory","type":"->"},{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":1104,"function":"cr
eateFile","class":"OCA\DAV\Upload\UploadFolder","type":"->"},{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/CorePlugin.php","line":527,"function":"createFile","class":"Sabre\DAV\Server","type":"->"},{"file":"/var/www/nextclo
ud/3rdparty/sabre/event/lib/WildcardEmitterTrait.php","line":89,"function":"httpPut","class":"Sabre\DAV\CorePlugin","type":"->"},{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":474,"function":"emit","class":"S
abre\DAV\Server","type":"->"},{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":251,"function":"invokeMethod","class":"Sabre\DAV\Server","type":"->"},{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Serv
er.php","line":319,"function":"start","class":"Sabre\DAV\Server","type":"->"},{"file":"/var/www/nextcloud/apps/dav/lib/Server.php","line":320,"function":"exec","class":"Sabre\DAV\Server","type":"->"},{"file":"/var/www/nextcloud/apps/
dav/appinfo/v2/remote.php","line":35,"function":"exec","class":"OCA\DAV\Server","type":"->"},{"file":"/var/www/nextcloud/remote.php","line":167,"args":["/var/www/nextcloud/apps/dav/appinfo/v2/remote.php"],"function":"require_once"}],"F
ile":"/var/www/nextcloud/apps/dav/lib/Connector/Sabre/File.php","Line":238,"CustomMessage":"--"},"userAgent":"Mozilla/5.0 (Windows) mirall/2.6.4stable-Win64 (build 20200303) (Nextcloud)","version":"19.0.0.12"}
{"reqId":"aG0WKf6Jk3WBKZzcfG5l","level":4,"time":"2020-06-12T10:10:42+00:00","remoteAddr":"192.168.50.5","user":"USERNAME","app":"webdav","method":"PUT","url":"/remote.php/dav/uploads/USERNAME/2948563684/00000000","message":{"Exception":"Sab
re\DAV\Exception\BadRequest","Message":"Expected filesize of 35358091 bytes but read (from Nextcloud client) and wrote (to Nextcloud storage) 14827520 bytes. Could either be a network problem on the sending side or a problem writing t
o the storage on the server side.","Code":0,"Trace":[{"file":"/var/www/nextcloud/apps/dav/lib/Connector/Sabre/Directory.php","line":154,"function":"put","class":"OCA\DAV\Connector\Sabre\File","type":"->"},{"file":"/var/www/nextcloud/
apps/dav/lib/Upload/UploadFolder.php","line":46,"function":"createFile","class":"OCA\DAV\Connector\Sabre\Directory","type":"->"},{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":1104,"function":"createFile","
class":"OCA\DAV\Upload\UploadFolder","type":"->"},{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/CorePlugin.php","line":527,"function":"createFile","class":"Sabre\DAV\Server","type":"->"},{"file":"/var/www/nextcloud/3rdparty
/sabre/event/lib/WildcardEmitterTrait.php","line":89,"function":"httpPut","class":"Sabre\DAV\CorePlugin","type":"->"},{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":474,"function":"emit","class":"Sabre\DAV\
Server","type":"->"},{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":251,"function":"invokeMethod","class":"Sabre\DAV\Server","type":"->"},{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","li
ne":319,"function":"start","class":"Sabre\DAV\Server","type":"->"},{"file":"/var/www/nextcloud/apps/dav/lib/Server.php","line":320,"function":"exec","class":"Sabre\DAV\Server","type":"->"},{"file":"/var/www/nextcloud/apps/dav/appinfo
/v2/remote.php","line":35,"function":"exec","class":"OCA\DAV\Server","type":"->"},{"file":"/var/www/nextcloud/remote.php","line":167,"args":["/var/www/nextcloud/apps/dav/appinfo/v2/remote.php"],"function":"require_once"}],"File":"/var/
www/nextcloud/apps/dav/lib/Connector/Sabre/File.php","Line":238,"CustomMessage":"--"},"userAgent":"Mozilla/5.0 (Windows) mirall/2.6.4stable-Win64 (build 20200303) (Nextcloud)","version":"19.0.0.12"}
Other Problems. Both are with some GUI elements. The bell on the top right with notifications have always red dot informing thet there are some notifications, but when i click on it there are no (then dot disappears). After refresh, or click on any link inside nextcloud red dot is appearing there.
Second problem is, when i start Nextcloud for first time, or click on my avatar -> About (About Nextcloud window). The first page is empty blue like on the screenshot.
Update1:
Solution for last problem with first run wizard is here: https://help.nextcloud.com/t/first-run-wizard-video-does-not-play-http-302/75281
Update2:
There is also a lot of errors:
session_start(): A session had already been started - ignoring at /var/www/nextcloud/lib/private/Session/Internal.php#209
But i find this is known problem for N19 and many people already reported that.
Update3:
it looks it may be problem with Redis cache. I'm making more tests,
Update4:
I changed 'memcache.local' => '\OC\Memcache\Redis', into 'memcache.local' => '\OC\Memcache\APCu'.
Deleting two lines in config.php fixed the problem:
'filelocking.enabled' => true,
'memcache.locking' => '\OC\Memcache\Redis',
I'll try to learn what is the problem, but i think for now i will leave it disabled so at least, it will work correctly.
Update5:
Fresh CentOS 7 install. Same problem.
The text was updated successfully, but these errors were encountered: