Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

upload >100k not finished #1006

Closed
acarlomagno opened this Issue · 19 comments

9 participants

@acarlomagno

Hi,
I have updated to 4.5.5 but I have always the upload file error

on Chrome when I drag the file from desktop to browser ... the upload process starts ...
if the file size is < 100k ... the upload finish correctly

if the file size is >100k ... the upload progress starts and stopped ... but the upload process, not stop.... the circle icon not stop.

in attachment the screen-shot of upload .... the file is pimove2012.key

regards
AC

@DeepDiver1975

Can you please have a look at the javascript console?
Anything in the web server error log?

@lasselasse

I have the same problem with large web uploads (> 100s of megabytes): progress bar finishes and disappears, but the spinner doesn't stop and the file never appears when browsing the folder.

I suspect this is an effect of the user session expiring during the upload because the upload takes longer than the session timeout, i.e. depending on the individual upload speed this has an impact on files of different sizes.

Circumstantial evidence hinting at a session expiration: After the upload progress bar has disappeared and waiting for a while for the spinner to stop (which it didn't), I wanted to look at that folder in another browser tab which redirected me to the login page, indicating that the session had expired. After logging in, the file I tried to upload wasn't present in the folder.

I'll try to do another upload with a javascript console open.

@lasselasse

To test this issue, I uploaded a file of 835 MB using Firefox 17.0.1, which took almost 3.5 hours with my connection. I used the webdeveloper console to track the progress (see below for its output). The OwnCloud version is 4.5.5 (4.90.6).

In the end, after the upload is done, an error is logged ("SyntaxError: JSON.parse: unexpected character"), which might indicate a problem with the response body of the upload request. I didn't log the reponse body, unfortunately, so I don't know what it was exactly.

In the webdeveloper console I could see that about every 55 minutes an ajax request to refresh the request token ("lifebeat") was issued.

The problem seems to be that while the first of these requests does send the same request token as was sent by the initial file upload, the subsequent refresh requests had the request header value for the request token as "undefined". I've seen some work on the request token code in git (e.g. issue #159), maybe there's still an issue with that?

Also, after the upload has failed, opening a new tab or reloading the page shows me the login page, indicating that my php session has expired.

I can see what I suspect to be the session cookie having been sent with all requests which should keep the php session alive, given that it has a lifetime of 60 minutes. So while the php session probably stayed alive, the request token did not? But why the login page hinting at an expiration of the php session as well? As for the JSON parsing error, I suspect that the upload request reponse body contained an error or maybe even HTML for the login page, which couldn't be parsed.

I hope this helps.

Console output with some additional information (indented):

[12:46:03.761] POST https://de.owncube.com/?app=files&getfile=ajax%2Fupload.php [HTTP/1.1 200 OK 18910ms]
    Request header: requesttoken:c20f3454f98ac0113b40
    Cookie:     507d17dbcc303:pqlv9fk2s3pcrnftp2dtufgfl1

[13:41:35.758] refreshing request token (lifebeat)  core.js:318

[13:41:35.877] POST https://foo.bar/core/ajax/requesttoken.php [HTTP/1.1 200 OK 842ms]
    Request header: requesttoken:c20f3454f98ac0113b40
    Cookie:     507d17dbcc303:pqlv9fk2s3pcrnftp2dtufgfl1

[14:37:23.858] POST https://foo.bar/core/ajax/requesttoken.php [HTTP/1.1 200 OK 822ms]
    Request header: requesttoken:undefined
    Cookie:     507d17dbcc303:pqlv9fk2s3pcrnftp2dtufgfl1

[14:37:23.780] refreshing request token (lifebeat)  core.js:318
  (It seems the webdeveloper console only shows this message once although it was logged twice,
   indicated by a little number "2" in a read circle---irritating but probably well meant.)

[15:33:11.847] POST https://foo.bar/core/ajax/requesttoken.php [HTTP/1.1 200 OK 1529ms]
    Request header: requesttoken:undefined
    Cookie:     507d17dbcc303:pqlv9fk2s3pcrnftp2dtufgfl1

[16:07:05.989] SyntaxError: JSON.parse: unexpected character @ https://de.owncube.com/remote.php/core.js:11
@DeepDiver1975

Sounds very reasonable. Thanks a lot for the detailed analysis.
Looks like we need to rethink the session expiry.

@jancborchardt
Collaborator

No idea about it specifically, but the »lifebeat« issue described by @lassi seems to be a or the main one.

Regarding session expiry we shouldn’t be as hardcore to force log people out. Yes, it’s security measures but not when it’s just annoying. But I suppose if the lifebeat already tries to counter that.

@lasselasse

I've got more information: It looks like the "lifebeat" doesn't work at all because the session expires before it is issued.

I retested the upload, this time with response body logging enabled, and already response of the first lifebeat request indicates an authentication error:

{"data":{"message":"Authentication error"},"status":"error"}

Here's the request log:

[16:15:14.051] POST https://foo.bar/?app=files&getfile=ajax%2Fupload.php
[17:10:41.999] refreshing request token (lifebeat)      core.js:318
[17:10:42.137] POST https://foo.bar/core/ajax/requesttoken.php [HTTP/1.1 200 OK 797ms]

The session had already timed out at this point, opening a new tab on that owncloud instance send me to the login page! It seems that the session timeout is shorter than the lifebeat interval.

@LukasReschke
Collaborator

I'll backport #178 later this week, this should solve the problem.

@LukasReschke LukasReschke was assigned
@karlitschek
Owner

Awesome @LukasReschke. That's great :-)

@tanghus
Collaborator

@LukasReschke I'm not sure the requesttoken handling is the problem. I have the same symptom in master which is described in #527

@LukasReschke
Collaborator

@tanghus This is a different issue, I'll try to look at this also this week. :-)

But first: :christmas_tree: ;-)

@tanghus
Collaborator

Yep, enjoy :)

christmas coder

@mastern

Same problem with file >40Mo and no server side solution is possible even trying to modify MaxRequestLen FCGID.

Here is the error in chrome as in Firefox after passing 30~40Mo :
"Failed to load resource -> http://ownclouddomain.com/?app=files&getfile=ajax%2Fupload.php"

@lasselasse

The problem persists with version 4.5.6, I just checked.
The only way to upload big files right now is to use a WebDAV client.

Is anyone working on this? It would be great if this is fixed for the ownCloud 5 release.

@DeepDiver1975

#1125 has not yet been merged into stable45.
@lasselasse maybe you have the chance to test this pull request Thx

@gnozys

SOLUTION FOUND HERE : #1270
It solved mine, I think it's not a bug.

@lasselasse

@DeepDiver1975 I don't run the server, so I cannot try it out.

@gnozys I have no control of the server-side configuration, but I don't think this is the actual issue here since I can see that the requesttoken expires.

@mastern

@gnozys A great great thanks. You're right. I can upload a big files now!

@lasselasse you'll never be able to run it if you don't have your hand on the server config. I have the same problem of you with requesttoken. The LimitRequestBody 2nd line of @gnozys' code in server config here #1270 solved it.

@lasselasse

I asked my provider to check their configuration taking into account the solution described in #1270.

Turns out they have an upload request timeout of 15 minutes (is this the ownCloud default?), so that's probably the reasons for the problems I'm having with large uploads, because they simply take too long. I'm askign them to increase the timeout to be able to verify this.

@karlitschek karlitschek closed this
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.