Auto Upload: make multiple uploads at once possible #6794
Comments
|
I had the same experience, when I did my initial upload with my Android phone. I'd tried to sync ~2k files with an overall size of ~1GB over local 5G Wifi and it took over an hour to sync completely. Assumption |
This should be unrelated, as the upload does not rely on previews…
--> what you can do is to go to upload view, or keep the app in background to see if it faster
That is true, but this is a cheap UI operation, which does not result in 20s delay as @francescocarzaniga is describing. |
tobiasKaminsky
added
the
needs info
|
@tobiasKaminsky Thank you for your answer!
I haven't checked the code so I do not presume to know what is going on under the hood, I simply reported my observations that there is delay between these operations, though I do not know why. Does the web client perform more then one upload at a time? If so, why don't the apps as well?
My little test has been performed while in the upload view. I have tried every possible combination (background, foreground, different views) and it's always this slow. Just to be clear, this is not something that only affects Android (on the contrary it seems to be not afflicted as badly), but the iOS app as well. |
Yeah, you should see my post more with a fan boy point of view than as a technical analysis. I then started to search for the bottleneck and the Server idled, the Network idled, only my phone was getting warmer and so I thought: Maybe there is too much overhead on the Client Site in the process of syncing a file, and if you sync thousand of small files these overhead is getting an issue. Tough I'm a developer, I don't have any knowledge of Android development, so just see my comment as an outsiders view on the problem. 🙂 |
|
:-D We will have to do some sort of timing analysis to see where the bottleneck is. |
|
When doing initial auto-upload after installing nextcloud app this past week the slowness for me seemed to just be a consistent delay between files being uploaded. It seemed to be close to 25s between files being uploaded, very periodic when looking at the network bandwidth vs time on my router. when doing an initial sync of over 9000 photos that'll take days to run. The speed of each individual file seemed normal; however, they were in the MB range so not really able to get a good sense on what the actual Mb/sec was. |
|
Just a quick update since I had not had the chance to the test the iOS client in months. It seems that somewhere along the way (Nextcloud 3.0 perhaps?) things have gotten much much better. Here is the log of an auto-upload from iOS: I guess the iOS client has implemented the parallel upload you were talking about @tobiasKaminsky, even though there is no mention of it anywhere in the changelogs. In any case the iOS client is now performing adequately to upload even larger libraries relatively quickly, so I believe the problem I described at the beginning is restricted to Android only. |
This comment was marked as outdated.
This comment was marked as outdated.
|
Well, yes. Currently upload is done one by one. |
tobiasKaminsky
added
enhancement
and removed
bug
needs info
tobiasKaminsky
changed the title
|
Hate to be "that asshat that revives ancient reports" but, an option to allow at the very least 2 uploads at the same time (make it configurable for the user, they can decide the best number themselves based on issues/performance). Android client takes a piss long time to upload a lot of images at once (currently processing 2500 of them.. one at a time). The issue is less visible when using the website upload. That's still much faster. |
|
@tobiasKaminsky IMHO this is a BUG not an enhancement. The bug affects all android devices. It is so slow that NextCloud is unusable on Android. That makes it a bug. Enhancement implies that the app works and it is just a additional feature. Bug implies that it should work but it doesn't. This is a bug. This is a BUG and should receive the same attention as any other bug. Otherwise we will be sitting with slow sync on Android until were all 99 years old and our files will still not be sunc (new word) P.S. even after installing redis for file locking, tuning mysql, enabling http2 it is still slower than molassis. |
|
This would be so useful and make the app much better. It takes ages to upload multiple files, especially when you upload many pictures via Auto Upload. How is this issue still not getting any attention? |
|
Please improve this because multishot take more than 1 hour uploading.... |
|
I Think I resolved the problem, I'm uploading my entire google photos library to Nextcloud +20k images, and I was in my second day of sync, but the last 6k were uploaded in a couple of hours. All I did was disable the permission to show notifications with the option “envios” (I am Portuguese) in English should be uploads. |
joshtrichards
changed the title
joshtrichards
added
feature: auto upload
performance: general/non-specific
|
Let's consolidate tracking of progress and interest in this into #10124 |
Steps to reproduce
Expected behaviour
Actual behaviour
Environment data
Android version: 10
Device model: OnePlus 7
Stock or customized system: Stock
Nextcloud app version: 3.13.0
Nextcloud server version: 18 and 19
Reverse proxy: NGINX
Logs
Web server error log
Logs for photo upload from phone:
Logs for photo upload from laptop, 96 medium-size random pictures totalling 480MB:
Logs for photo upload from laptop, 1092 small-size random pictures totalling 124MB:
Nextcloud log (data/nextcloud.log)
The medium pictures finished uploading in approximately 3-4 minutes, with a throughput of 2-3MB/s or 20-30 pictures per minute. The small pictures finished uploading in approximately 2 minutes, with a throughput of ~1MB/s or ~500 pictures per minute. There is more than one PUT per second, with the GETs being the native client on the laptop automatically syncing the new pictures.
The testing has been performed on a WAN connection to keep things realistic, so the performance in this situation is absolutely satisfactory.
The phone upload is staggeringly slow, since there is a 20 seconds delay from getting the preview of the old image to checking the existence of the new image, and then another 20 seconds to actually performing the PUT.
The text was updated successfully, but these errors were encountered: