Large asset uploads timing out

[jscaptura]

Since the switch to GUNICORN, I've had trouble uploading video assets of 400mb+ size. To get around this, I had increased the timeout to give the uploads time to finish. I've since reverted to using the distributed code without any local modifications.

Now, I have a new batch of video assets to upload, and while the progress bar gets to the end, after a while it goes back to the beginning and the "The operation failed. Please reload the page and try again." error pops up. Is there another time limit somewhere that is preventing the processing of large files?

[Lil-briX]

I got exectly the same problem.
in var/log/nginx i found an error log.

2016/06/14 11:57:43 [error] 602#0: *1761 client intended to send too large body: 9886664 bytes, client: 10.0.54.94, server: , request: "POST /api/assets HTTP/1.1", host: "test-raspberry-pi", referrer: "https://test-raspberry-pi/"
2016/06/14 11:57:55 [error] 601#0: *1764 client intended to send too large body: 2340451 bytes, client: 10.0.54.94, server: , request: "POST /api/assets HTTP/1.1", host: "test-raspberry-pi", referrer: "https://test-raspberry-pi/"
2016/06/14 12:11:16 [error] 602#0: *1880 client intended to send too large body: 581950138 bytes, client: 10.0.66.109, server: , request: "POST /api/assets HTTP/1.1", host: "test-raspberry-pi", referrer: "https://test-raspberry-pi/"
2016/06/14 12:22:08 [error] 600#0: *1881 client intended to send too large body: 581950169 bytes, client: 10.0.66.109, server: , request: "POST /api/assets HTTP/1.1", host: "test-raspberry-pi", referrer: "https://test-raspberry-pi/"
2016/06/14 12:24:45 [notice] 14571#0: signal process started
2016/06/14 12:29:48 [error] 14575#0: *1914 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 10.0.66.109, server: , request: "POST /api/assets HTTP/1.1", upstream: "http://127.0.0.1:8080/api/assets", host: "test-raspberry-pi", referrer: "https://test-raspberry-pi/"
2016/06/14 13:07:00 [notice] 19277#0: using inherited sockets from "6;7;"
2016/06/14 13:32:07 [error] 601#0: *43 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 10.0.66.109, server: , request: "POST /api/assets HTTP/1.1", upstream: "http://127.0.0.1:8080/api/assets", host: "test-raspberry-pi", referrer: "https://test-raspberry-pi/"
2016/06/14 14:27:40 [error] 601#0: *103 upstream prematurely closed connection while reading response header from upstream, client: 10.0.66.109, server: , request: "POST /api/assets HTTP/1.1", upstream: "http://127.0.0.1:8080/api/assets", host: "test-raspberry-pi", referrer: "https://test-raspberry-pi/"
2016/06/15 07:56:07 [error] 602#0: *114 upstream prematurely closed connection while reading response header from upstream, client: 10.0.66.109, server: , request: "POST /api/assets HTTP/1.1", upstream: "http://127.0.0.1:8080/api/assets", host: "test-raspberry-pi", referrer: "https://test-raspberry-pi/"
2016/06/15 08:28:13 [error] 600#0: *130 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 10.0.66.109, server: , request: "POST /api/assets HTTP/1.1", upstream: "http://127.0.0.1:8080/api/assets", host: "test-raspberry-pi", referrer: "https://test-raspberry-pi/"
error.log (END)

maybe it has something to do with that? "client intended to send too large body" ?
Moreover it would be usefull to change the Errormessages because they do not help anyone.

Best,
Lil briX

[Lil-briX]

Moreover there is a file in ~/screenly_assets wich is probally my video:

-rw-r--r--  1 pi pi 77168640 Jun 14 14:27 ad9b310db18b495c86bfc53b40481340
pi@raspberrypi:~/screenly_assets $ file ad9b310db18b495c86bfc53b40481340
ad9b310db18b495c86bfc53b40481340: ISO Media, MP4 Base Media v1 [IS0 14496-12:2003]

But for this file there issn't an entry in the db! (~/.screenly/screenly.db)

pi@raspberrypi:~/.screenly $ sqlite3 screenly.db 
SQLite version 3.8.7.1 2014-10-29 13:59:56
Enter ".help" for usage hints.
sqlite> select * from assets;
47efc7864c0b43589517fd58b50ee1ab|02 Groupwise cortical T1_1920.png|/home/pi/screenly_assets/47efc7864c0b43589517fd58b50ee1ab||2016-06-01 15:20:00|2020-06-08 15:20:00|10|image|1|0|1
210df5da0abc492f8f8197bdb8eb9ba5|01 Quantitative T1_1920.png|/home/pi/screenly_assets/210df5da0abc492f8f8197bdb8eb9ba5||2016-06-01 15:26:00|2020-06-08 15:26:00|10|image|1|0|0
542ed1b65cc0405db3d411d2747cab09|04 cortical depth modeling_1920.png|/home/pi/screenly_assets/542ed1b65cc0405db3d411d2747cab09||2016-06-01 15:26:00|2020-06-08 15:26:00|10|image|1|0|3
3642f470996744fd818836c435feb299|05 MP2RAGE_1920.png|/home/pi/screenly_assets/3642f470996744fd818836c435feb299||2016-06-01 15:26:00|2020-06-08 15:26:00|10|image|1|0|4
e9f57ea144264c7dbdf0ccedbcfcf767|06 TSE_1920.png|/home/pi/screenly_assets/e9f57ea144264c7dbdf0ccedbcfcf767||2016-06-01 15:27:00|2020-06-08 15:27:00|10|image|1|0|5
32b2b3ddf1d749b4a7a38df74fd6780c|10 exvivo DWI_1920.png|/home/pi/screenly_assets/32b2b3ddf1d749b4a7a38df74fd6780c||2016-06-01 15:28:00|2020-06-08 15:28:00|10|image|1|0|9
62c3b89428d8497887dbdb70b5fdf15d|11 invivo QSM_1920.png|/home/pi/screenly_assets/62c3b89428d8497887dbdb70b5fdf15d||2016-06-01 15:29:00|2020-06-08 15:29:00|10|image|1|0|10
715154ab6b1e4060b77f2a78b464cb5b|14 genari2_1920.png|/home/pi/screenly_assets/715154ab6b1e4060b77f2a78b464cb5b||2016-06-01 15:29:00|2020-06-08 15:29:00|10|image|1|0|13
ba6e417c391c4b82b0aed6e01ef82d3b|15 M1 activation_1920.png|/home/pi/screenly_assets/ba6e417c391c4b82b0aed6e01ef82d3b||2016-06-01 15:29:00|2020-06-08 15:29:00|10|image|1|0|14
37b062c21cfb45de9d9e07d53050421c|16 V5 activation_1920.png|/home/pi/screenly_assets/37b062c21cfb45de9d9e07d53050421c||2016-06-01 15:30:00|2020-06-08 15:30:00|10|image|1|0|15
9067340eddb946a39f4538505b937b37|03 Highres brain segmentation.gif|/home/pi/screenly_assets/9067340eddb946a39f4538505b937b37||2016-06-02 11:28:00|2020-06-09 11:28:00|15|image|1|0|2
3371a23e03234c48954011b9dbbcb61a|07 RESOLVE_1920.gif|/home/pi/screenly_assets/3371a23e03234c48954011b9dbbcb61a||2016-06-02 11:53:00|2020-06-09 11:53:00|15|image|1|0|6
ee01f86193f7463796961690756900ca|08 DWI_1920.gif|/home/pi/screenly_assets/ee01f86193f7463796961690756900ca||2016-06-02 11:55:00|2020-06-09 11:55:00|15|image|1|0|7
27f0228881b444d19f3ae2e73b7bd451|09 ZOOPPA_1920.gif|/home/pi/screenly_assets/27f0228881b444d19f3ae2e73b7bd451||2016-06-02 11:55:00|2020-06-09 11:55:00|15|image|1|0|8
8c80ca9452c24b998184e9a2e1d35aec|12 invivo.png|/home/pi/screenly_assets/8c80ca9452c24b998184e9a2e1d35aec||2016-06-14 11:35:00|2016-06-21 11:35:00|15|image|1||11
27c8d3592815496ab5ca9c79bfc6b675|13 genari.png|/home/pi/screenly_assets/27c8d3592815496ab5ca9c79bfc6b675||2016-06-14 11:37:00|2016-06-21 11:37:00|15|image|1||12

The file can't be found in the webinterface aswell.
Best,
Lil briX

[jscaptura]

Good catch. I just found and cleaned out 19 orphaned assets, totalling 3.5GB.

[Lil-briX]

Some suggestions here?
I still have the problem with a 550mb .mp4 file

Best,
Lil briX

[vpetersson]

No work has been done on this. The easiest quick fix is to bump up the timeout here.

[Lil-briX]

Bumed it up to 800, and still the error message "Operation failed. Please reload the page and try it again"

The video im tryin to upload is 559mb.

[elpollodiabl0]

I experienced the same problem with a similarly big mp4 file not being uploaded to a pi zero via weak wireless signal.

Bumping up the timeout to 5000 fixed the issue...

[By the way: Thanks for the great piece of software, Wireload & Contributors!]

[dpohlman]

I am experiencing the same thing, if I up the time out and hard wire or get better wireless speed my 200 Mb video file will upload fine. Has there been any update on this bug?

[renowden]

I came across this problem which was not a timeout (as it was a fast network) but just the size of the file. It would not load a JPG of 2MB. Fixed by adding

    client_max_body_size 200m;

to the http section of /etc/nginx/nginx.conf
Reading elsewhere http://stackoverflow.com/questions/2056124/nginx-client-max-body-size-has-no-effect suggests that the limit is 1MB by default.

[vpetersson]

Should be fixed in master now.