Uploading to OneDrive Cryptomator vault is extremely slow if there are many files

[cyberduck]

ce6d08c created the issue

Hi,

there seems to be a bug in Cyberduck when lots of fairly small files are uploaded to a Cryptomator vault located in OneDrive. I've attached an example. Just unpack it and drag and drop it to the vault. It will be extremely slow.

---

Attachments

• Example.zip (584.0 KiB)

[cyberduck]

ce6d08c commented

Hi,

what's the status on this? It still takes a very long time to browse through directories. Please give an update.

Thanks and best regards
rootie

[cyberduck]

ce6d08c commented

Hi,

is there any plan in improving this? This is clearly a bug and can most likely fixed fairly easily.

Please give an update.

Thanks and best regards
rootie

[cyberduck]

f108a4e commented

Can confirm, super slow, using Windows 10 latest ver Cyberduck 7.8.3 and Mountain duck 4.5.0, I suspect there also is something to do with Ticket [[11582]] because you get rate limited very often uploading. Tried uploading 20,000 small files and it took around 6 hours for even 2000 files to be uploaded! It's quite frustrating to be honest. I hope there is a fix soon. It doesn't look good that this has been open for over 2 years now with no resolution especially after i went out and just purchased this software for basically this reason :/

[cyberduck]

ce6d08c commented

Yes unfortunately it seems that the developers are not interested in such things and this is very disappointing. The most disturbing thing is that there's not even a reaction - nothing.

As they call it 'enhancement' it's not declared a bug and hence won't be fixed anytime soon.

[cyberduck]

f108a4e commented

Yep, this is absolutely a defect, not a "enhancement". The software is unusable at this moment with OneDrive and basically over 1000 smallish files. Just went back and used the original OneDrive desktop client and Cryptomator desktop and it got my 20,000 files synced within an hour or two while Mountainduck and Cyberduck get stuck uploading for several more hours even if you have as few as 500 files! It's insane, absolutely is an issue with OneDrive syncing. I don't know whether to blame how slow it is to transfer many small files into a cryptomator vault on my awful computer or on Mountain Duck/Cyberduck though, Cryptomator desktop app seems to be blazing fast though. It's so weird. I was trialing this software for 2 weeks, it seemed alright with smaller things but it just can't handle a photo library, and I still went out and bought it today. I really didn't expect it to fail so hard today :/

[cyberduck]

@dkocher commented

Related to #10990.

[cyberduck]

@dkocher commented

Fix in 5980dd7 with missing cache.

[cyberduck]

@dkocher commented

Optimization in 78b0f34 to add newly created directories to cache for later lookup of parent directory id when uploading containing files.

[cyberduck]

@dkocher commented

Allow reuse of cached file id for previously created directory in 89770cd.

[cyberduck]

@dkocher commented

New Auto default allows up to 20 concurrent transfers for an upload in 013a4b9 to embrace parallelism when uploading many small files. The auto setting for FTP is at 1 connection (in 9a37e24) because FTP is stateful and the number of allowed connections is usually limited by the server.

[cyberduck]

@dkocher commented

The same issue affects all protocols where we have a file id to filename mapping.

• Google Drive
• OneDrive
• B2
• DRACOON

[cyberduck]

@dkocher commented

Optimization to reuse known file id when setting timestamp in 487a309.

[cyberduck]

@dkocher commented

Fix mapping of file id in decrypted file reference in c43add0.

[cyberduck]

@dkocher commented

Add internal cache in file id providers in d07148a.

[cyberduck]

@dkocher commented

Refactored in 77a74af and a18313f.