[Feature request] Chunked file upload

[Ghoelian]

It seems that right now, the entire file is being sent through a POST request.

I'm running MyDrive behind NGINX, which means I had to increase the client_max_body_size to be able to upload any kind of larger file. This is of course not a great solution, as this will quickly clog up the webserver when multiple users are trying to upload a file simultaneously.

I think having files be uploaded in smaller chunks would allow for better multi-user performance, as well as just not having to mess with your NGINX, or whatever other reverse proxy, configuration.

[subnub]

That is odd, while it does use a single POST request, myDrive will serve up the files in chunks instead of the whole file at once, the file should never be fully loaded into memory, for example I am able to upload 100gb+ files on a server with 1gb of RAM. Although I have not tested myDrive behind NGINX so It could be an issue with both of them in combination. I will have to do some further research on this issue, but could I have more information such as your Node version, ect? Thank you.

[Ghoelian]

My Node version is 13.14.0, running on a Raspberry Pi 4b on Ubuntu 20.04 64 bit.

[Ghoelian]

Oh also, I just opened a different issue, #24 . I guess that one might have something to do with this issue then as well.

[subnub]

Okay I would try to run the LTS version of Node, before I ran into similar issues when running 13.14 (Specifically on Linux too), try v12 for the best results (NVM can help with this), please make sure to remove and reinstall your node modules before rebuilding and testing.

If after you change your node version and reinstall/rebuild, and you still run into the same issue, It would be very helpful if you could attempt to use myDrive without NGINX and see if the same issue arises.

[Ghoelian]

I will try this in a minute, thanks for the help. I used v13.14.0 because it says in the guided setup that v13 works best.

[subnub]

You're right it does say that in the guide, thank you for bringing that to my attention I need to change that lol. And of course, let me know how it goes!

[Ghoelian]

I'm running into a different issue now. I can't log in to the account I previously created, it says my login details are incorrect, but I also can't create a new user, as it already exists. Is there any way of clearing the database of users?

[subnub]

What is it saying in the terminal when you attempt to log in? Like not the client side inside the terminal it should be displaying some type of error

And the easiest way to clear data would be just to change the mongoDB url at the end, there’s currently no way to clear all user accounts.

[Ghoelian]

Ah I see what the issue is here. It's sending a request to drive.mytld.com/drive.mytld.com/user-service/login, which results in a 404 from NGINX. I'm guessing this has something to do with the REMOTE_URL, but I'm not sure why it would change behaviour all of a sudden now that I'm on v12.

Edit: It is not caused by REMOTE_URL, changing this to just / doesn't change anything for the login.

[subnub]

Make sure your client URL is the correct format, it needs to be like so http://myDrive.com , make sure it starts with http/https, and also that it does not end in a / or anything like that.

[subnub]

Don’t forget to rebuild after changing environment variables

[Ghoelian]

Changing the env variable and rebuilding seems to have fixed that problem. Trying the file upload now.

[Ghoelian]

Downgrading to Node v12.18.3 seems to have fixed the file uploading problems as well. I still need to increase the client_max_body_size in NGINX, but after doing that I am able to upload the larger files.
Thanks for helping me figure this out!

[subnub]

You’re very welcome! Thank you for having an interest in myDrive, stay updated a big update including a complete UI overhaul is coming soon.

[Ghoelian]

Ooh very nice! Definitely looking forward to that.