Large files uploads

[klakhov]

Motivation and context

Resolved #956
Resolved #2332
Resolved #1616
Fixed large files uploading via webpage, implemented using chunk file uploads with upload progress bar.

How has this been tested?

Manually tested with large videos, archives, 20k images

Checklist

• I submit my changes into the develop branch
• I have added description of my changes into CHANGELOG file
• I have updated the documentation accordingly
• I have added tests to cover my changes
• I have linked related issues (read github docs)
• I have increased versions of npm packages if it is necessary (cvat-canvas,
  cvat-core, cvat-data and cvat-ui)

License

• I submit my code changes under the same MIT License that covers the project.
  Feel free to contact the maintainers if that's a concern.
• I have updated the license header for each file (see an example below)

# Copyright (C) 2021 Intel Corporation
#
# SPDX-License-Identifier: MIT

[nmanovic]

Some resources and ideas:

• https://docs.aws.amazon.com/AmazonS3/latest/userguide/mpuoverview.html
• https://github.com/juliomalegria/django-chunked-upload

[nmanovic]

@klakhov , I see that you have fixed only uploading data for tasks. I'm fine if you want to split the PR on several parts and implement huge file upload for other endpoints later.

[nmanovic]

@klakhov ,

• the known list of files can be a good heuristics. But in this case we cannot finish upload if by a reason it is not possible to send a file from the list.
• We can also use special acceptance headers (e.g. Upload-Start and Upload-Finish) like Tus does. It will be a part of our protocol. To support the legacy behaviors we can omit both headers and in the case the single request will be start and end at the same time.

What do you think?

[klakhov]

@nmanovic
I'm fine with that. Lets use headers.

[klakhov]

@nmanovic Could you please take a look on updated pr. I've removed /append, /tus, /finished endpoints and used Upload-Start, Upload-Finish, Upload-Multiple headers to indicate request purpose

[nmanovic]

@klakhov , could you please resolve conflicts?

[nmanovic]

Great job!

Comments for the future:

1. I saw that in case of bulk upload we have the following operation for each file in python code: open, read, write, close. Probably we can just move them from a temporary location to upload directory. Probably it will be an optimization in the future.
2. Need to update CLI to support the new protocol
3. Need to add the protocol for all other upload operations (import task, import dataset, import project).