Upload fails on files > 10 MB #198

voroyam · 2018-03-21T13:45:59Z

If a guest user uploads a file that is bigger than 10 MB the upload fails without a helpful error message in the web UI or the ownCloud log

Steps to reproduce:

Create a share with a guest user
Log in as guest user and upload a file in the shared folder
Files < 10 MB get uploaded
Files > 10 MB don't get uploaded

Tested with OC 10.0.3, 10.0.4, 10.0.7

Tested on docker Ubuntu with apache2 PHP 7

enabled apps: guest app.

This is the only thing in the owncloud.log, but it seems not related to the problem, because it is also displayed if the upload is successful.

App::"filesystem",
"method":"MKCOL","url":"\/owncloud\/remote.php\/dav\/uploads\/voroyam%40gmail.com\/web-file-upload-566fa2790c670e81fb3048164457ba1b-1521636862684",
"message":"Storage wrapper 'oc_readonly' was not registered via the 'OC_Filesystem - preSetup' hook which could cause potential problems."}
App::"filesystem",
"method":"DELETE","url":"\/owncloud\/remote.php\/dav\/uploads\/voroyam%40gmail.com\/web-file-upload-566fa2790c670e81fb3048164457ba1b-1521636862684",
"message":"Storage wrapper 'oc_readonly' was not registered via the 'OC_Filesystem - preSetup' hook which could cause potential problems."}
App::"filesystem",
"method":"GET","url":"\/owncloud\/index.php\/apps\/files\/ajax\/getstoragestats.php?dir=%2Fforguests",
"message":"Storage wrapper 'oc_readonly' was not registered via the 'OC_Filesystem - preSetup' hook which could cause potential problems."}

here is the network tab taken during a failed upload

@tomneedham @butonic @PVince81

The text was updated successfully, but these errors were encountered:

cortho · 2018-03-22T09:28:30Z

I guess when guests do not have a home some temporary files cannot be created.

So This is not a guests app issue but a core/webdav issue. Suggestion: close here and reopen in core.

See my logs below.

js errors

client.js?v=d5106cc…:246 MKCOL http://localhost/oc10/remote.php/dav/uploads/guest%40localhost/web-file-upload-5875dff62b65ed35356d174af1689b71-1521709720470 403 (Forbidden)

file-upload.js?v=d5106cc…:377 Uncaught (in promise) TypeError: Cannot read property 'status' of undefined(…)

client.js?v=d5106cc…:246 MKCOL http://localhost/oc10/remote.php/dav/uploads/guest%40localhost/web-file-upload-5875dff62b65ed35356d174af1689b71-1521709720470 403 (Forbidden)

owncloud.log

{
	"reqId":"ZTWkoRj70wg5bxQpROaV",
	"level":0,
	"time":"2018-03-22T09:09:22+00:00",
	"remoteAddr":"127.0.0.1",
	"user":"guest@localhost",
	"app":"webdav",
	"method":"MKCOL",
	"url":"/oc10/remote.php/dav/uploads/guest%40localhost/web-file-upload-5875dff62b65ed35356d174af1689b71-1521709762686",
	"message":"Exception: {
		"Message":"HTTP/1.1 403 Forbidden","Exception":"Sabre\DAV\Exception\Forbidden",
		"Code":0,
		"Trace":
			"#0 /var/www/oc10/apps/dav/lib/Upload/UploadHome.php(48): OCA\DAV\Connector\Sabre\Directory->createDirectory('web-file-upload...')
			#1 /var/www/oc10/lib/composer/sabre/dav/lib/DAV/Server.php(1209): OCA\DAV\Upload\UploadHome->createDirectory('web-file-upload...')
			#2 /var/www/oc10/lib/composer/sabre/dav/lib/DAV/CorePlugin.php(590): Sabre\DAV\Server->createCollection('uploads/guest@loca...', Object(Sabre\DAV\MkCol))
			#3 [internal function]: Sabre\DAV\CorePlugin->httpMkcol(Object(Sabre\HTTP\Request), Object(Sabre\HTTP\Response))
			#4 /var/www/oc10/lib/composer/sabre/event/lib/EventEmitterTrait.php(105): call_user_func_array(Array, Array)
			#5 /var/www/oc10/lib/composer/sabre/dav/lib/DAV/Server.php(479): Sabre\Event\EventEmitter->emit('method:MKCOL', Array)
			#6 /var/www/oc10/lib/composer/sabre/dav/lib/DAV/Server.php(254): Sabre\DAV\Server->invokeMethod(Object(Sabre\HTTP\Request), Object(Sabre\HTTP\Response))
			#7 /var/www/oc10/apps/dav/lib/Server.php(263): Sabre\DAV\Server->exec()
			#8 /var/www/oc10/apps/dav/appinfo/v2/remote.php(31): OCA\DAV\Server->exec()
			#9 /var/www/oc10/remote.php(165): require_once('/var/www/oc10/a...')
			#10 {main}","File":"/var/www/oc10/apps/dav/lib/Connector/Sabre/Directory.php",
		"Line":206,
		"User":"guest@localhost"
	}"
}

{
	"reqId":"2culOUHuyrxzl6JveT8k",
	"level":0,
	"time":"2018-03-22T09:09:22+00:00",
	"remoteAddr":"127.0.0.1",
	"user":"guest@localhost",
	"app":"webdav",
	"method":"DELETE",
	"url":"/oc10/remote.php/dav/uploads/guest%40localhost/web-file-upload-5875dff62b65ed35356d174af1689b71-1521709762686",
	"message":"Exception: {
		"Message":"HTTP/1.1 404 File with name //web-file-upload-5875dff62b65ed35356d174af1689b71-1521709762686 could not be located",
		"Exception":"Sabre\DAV\Exception\NotFound",
		"Code":0,
		"Trace":
			"#0 /var/www/oc10/apps/dav/lib/Upload/UploadHome.php(52): OCA\DAV\Connector\Sabre\Directory->getChild('web-file-upload...')
			#1 /var/www/oc10/lib/composer/sabre/dav/lib/DAV/Tree.php(76): OCA\DAV\Upload\UploadHome->getChild('web-file-upload...')
			#2 /var/www/oc10/apps/dav/lib/Tree.php(48): Sabre\DAV\Tree->getNodeForPath('uploads/guest@loca...')
			#3 /var/www/oc10/lib/composer/sabre/dav/lib/CalDAV/Schedule/Plugin.php(412): OCA\DAV\Tree->getNodeForPath('uploads/guest@loca...')
			#4 [internal function]: Sabre\CalDAV\Schedule\Plugin->beforeUnbind('uploads/guest@loca...')
			#5 /var/www/oc10/lib/composer/sabre/event/lib/EventEmitterTrait.php(105): call_user_func_array(Array, Array)
			#6 /var/www/oc10/lib/composer/sabre/dav/lib/DAV/CorePlugin.php(286): Sabre\Event\EventEmitter->emit('beforeUnbind', Array)
			#7 [internal function]: Sabre\DAV\CorePlugin->httpDelete(Object(Sabre\HTTP\Request), Object(Sabre\HTTP\Response))
			#8 /var/www/oc10/lib/composer/sabre/event/lib/EventEmitterTrait.php(105): call_user_func_array(Array, Array)
			#9 /var/www/oc10/lib/composer/sabre/dav/lib/DAV/Server.php(479): Sabre\Event\EventEmitter->emit('method:DELETE', Array)
			#10 /var/www/oc10/lib/composer/sabre/dav/lib/DAV/Server.php(254): Sabre\DAV\Server->invokeMethod(Object(Sabre\HTTP\Request), Object(Sabre\HTTP\Response))
			#11 /var/www/oc10/apps/dav/lib/Server.php(263): Sabre\DAV\Server->exec()
			#12 /var/www/oc10/apps/dav/appinfo/v2/remote.php(31): OCA\DAV\Server->exec()
			#13 /var/www/oc10/remote.php(165): require_once('/var/www/oc10/a...')
			#14 {main}","File":"/var/www/oc10/apps/dav/lib/Connector/Sabre/Directory.php",
		"Line":257,
		"User":"guest@localhost"
	}"
}

voroyam · 2018-03-22T11:32:40Z

Workaround:

create a /tmp/uploads
Give the folder to your webserver-user chown -R www-data:www-data /tmp/uploads
add this to config.php 'dav.chunk_base_dir' => '/tmp/uploads',
upload files as guest user > 10 MB

cdamken · 2018-03-22T12:02:33Z

@cortho We can use for the guest app that the guest-users use only the temp folder instead of a home folder like step 3 or the defined PHP temp folder.

IMO this is a requirement for this app and not from the main core behavior.

@PVince81 @felixboehm What's your opinion?

I'm just trying to avoid change the core to fix any app. Keep the core simple 😄

PVince81 · 2018-03-26T08:05:41Z

That workaround shouldn't be necessary.

The bug might be that core's read-only wrapper is too strong and also applies to the "cache" and "uploads" folder where it shouldn't.

@IljaN please investigate

cdamken · 2018-04-09T10:04:00Z

@IljaN Any news?

IljaN · 2018-05-08T10:27:36Z

Still debugging, All directories can be created now but final chunk assembly MOVE fails with a 404 on .file

cortho · 2018-05-25T11:45:56Z

I'm quite convinced this is a core issue, because guests do not have an upload directory in their homes (as opposed to full ownCloud users), maybe you'll find something there.

PVince81 · 2018-05-25T11:55:10Z

Yes, likely a core issue to be identified with the readonly storage wrapper.

KuenzelIT · 2018-05-30T10:40:50Z

Does anyone have a clue why this happens at 10MB and not with a smaller/bigger file size?

phil-davis · 2018-05-30T11:22:38Z

Does anyone have a clue why this happens at 10MB and not with a smaller/bigger file size?

less than 10MB the file is uploaded in 1 go, no intermediate pieces.
greater than 10MB the file is uploaded in "chunks" and the chunks assembled at the end - the guest needs some intermediate storage somewhere for the chunks.

PVince81 · 2018-05-30T12:23:58Z

@IljaN please link to your PR when done

PVince81 · 2018-06-04T11:06:33Z

The fix was merged but there's some challenges with the tests:

acceptance tests: Test for chunking upload #212

PVince81 · 2018-06-04T12:27:40Z

and forward port needs fixing for code style:

master port: [master] fix chunked upload for guest app core#31618

PVince81 · 2018-06-06T16:55:32Z

all merged, closing

patrickjahns · 2018-06-12T17:03:57Z

@PVince81
reopen related to objectstorage or separate ticket?

PVince81 · 2018-06-12T20:09:12Z

@patrickjahns let's use a new ticket, is anyone of you working on this or do you need assignment from the server team ?

PVince81 · 2018-06-12T20:28:45Z

new ticket here: #219

voroyam added bug p3-medium sev3-medium labels Mar 21, 2018

voroyam self-assigned this Mar 21, 2018

PVince81 assigned IljaN and unassigned voroyam Mar 26, 2018

PVince81 added p2-high and removed p3-medium labels Mar 26, 2018

PVince81 added this to the development milestone Mar 26, 2018

IljaN mentioned this issue May 30, 2018

[stable10] Exclude uploads directory from read-only cache mask owncloud/core#31596

Merged

9 tasks

IljaN added a commit that referenced this issue May 30, 2018

test for chunking upload #198

09afc17

IljaN mentioned this issue May 30, 2018

Test for chunking upload #212

Merged

IljaN added a commit that referenced this issue Jun 1, 2018

test for chunking upload #198

5361d39

PVince81 assigned PVince81 and unassigned IljaN Jun 4, 2018

PVince81 closed this as completed Jun 6, 2018

davitol mentioned this issue Jun 12, 2018

1.0.0 Testplan owncloud/files_primary_s3#61

Closed

38 tasks

PVince81 mentioned this issue Jun 12, 2018

Chunked upload does not work with primary object stores #219

Open

danielmarschall mentioned this issue Oct 27, 2018

"403 Forbidden" to "MOVE" syncing large files owncloud/core#33119

Closed

fsutil mentioned this issue May 15, 2019

Upload fails on files larger than 10 MB for LDAP users owncloud/core#35239

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Upload fails on files > 10 MB #198

Upload fails on files > 10 MB #198

voroyam commented Mar 21, 2018

cortho commented Mar 22, 2018

voroyam commented Mar 22, 2018

cdamken commented Mar 22, 2018

PVince81 commented Mar 26, 2018

cdamken commented Apr 9, 2018

IljaN commented May 8, 2018

cortho commented May 25, 2018

PVince81 commented May 25, 2018

KuenzelIT commented May 30, 2018

phil-davis commented May 30, 2018 •

edited

Loading

PVince81 commented May 30, 2018

PVince81 commented Jun 4, 2018 •

edited

Loading

PVince81 commented Jun 4, 2018 •

edited

Loading

PVince81 commented Jun 6, 2018

patrickjahns commented Jun 12, 2018

PVince81 commented Jun 12, 2018

PVince81 commented Jun 12, 2018

Upload fails on files > 10 MB #198

Upload fails on files > 10 MB #198

Comments

voroyam commented Mar 21, 2018

Steps to reproduce:

cortho commented Mar 22, 2018

voroyam commented Mar 22, 2018

cdamken commented Mar 22, 2018

PVince81 commented Mar 26, 2018

cdamken commented Apr 9, 2018

IljaN commented May 8, 2018

cortho commented May 25, 2018

PVince81 commented May 25, 2018

KuenzelIT commented May 30, 2018

phil-davis commented May 30, 2018 • edited Loading

PVince81 commented May 30, 2018

PVince81 commented Jun 4, 2018 • edited Loading

PVince81 commented Jun 4, 2018 • edited Loading

PVince81 commented Jun 6, 2018

patrickjahns commented Jun 12, 2018

PVince81 commented Jun 12, 2018

PVince81 commented Jun 12, 2018

phil-davis commented May 30, 2018 •

edited

Loading

PVince81 commented Jun 4, 2018 •

edited

Loading

PVince81 commented Jun 4, 2018 •

edited

Loading