resumable dataset upload

[youri-k]

URL of deployed dev instance (used for testing):

• https://___.webknossos.xyz

Steps to test:

• upload dataset => should work

Issues:

• fixes Make dataset upload robust #4812

---

• Updated (unreleased) changelog
• [ ] Updated (unreleased) migration guide if applicable
• [ ] Updated documentation if applicable
• [ ] Adapted wk-connect if datastore API changes
• Needs datastore update after deployment
• Ready for review

[youri-k]

ToDo:

• check how unique resumable uniqueIdentifieris
• synchronize map accesses

[youri-k]

Regarding this ToDo:

check how unique resumable uniqueIdentifier is

resumable uses this code to generate it:

var relativePath = file.webkitRelativePath||file.relativePath||file.fileName||file.name;
var size = file.size;
return(size + '-' + relativePath.replace(/[^0-9a-zA-Z_-]/img, ''));

In my opinion, this isn't unique enough, but I am not sure if it would be enough to add a time code or if we should add the user token in the backend, but that could break it, if the token changes while uploading.

Any ideas @philippotto, @fm3 ?

[philippotto]

Awesome stuff! Just left a few small comments and you probably want to change the unique identifier thing, right?

[philippotto]

Looks good from my side 🕺

[fm3]

I like the approach and separating uploading of chunks and a final finish request seems fair. To be honest I had some troubles understanding some of the semantics of pendingUploadChunks and needsWrite in the DataSourceService. Maybe we can figure something out for that (or maybe I just need a verbal explanation)

[fm3]

Thanks, with the renamings everything gets clear 👍 Works for me and Backend LGTM

[philippotto]

🕺 🎉