allow >5GB and turn on multi-part uploads for AWS

[kbuma]

Change AWS upload code to allow >5GB objects to be saved and to turn on multi-part uploads

Contributor Checklist

• I have run the tests locally and they passed.

[munrojm]

@kbuma thanks again for taking a look at this and making the necessary changes.

[munrojm]

Do you have any concerns with the upstream multithreading of that function? In other words, does it even make sense to keep the threadpool upstream, or should we just let boto3 handle concurrency via multithreaded chunk uploads?

[kbuma]

from my understanding the upstream multithreading is dealing with parallelizing multiple file uploads. this update will parallelize the multiple parts of a single file upload. so they could be complimentary.
the updated code improves performance for large files whereas the upstream multithreading could provide you performance enhancements in situations where you have a lot of small files.

[codecov]

Codecov Report

Patch coverage: 100.00% and no project coverage change.

Comparison is base (5f8074f) 88.13% compared to head (5376217) 88.14%.

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #829   +/-   ##
=======================================
  Coverage   88.13%   88.14%           
=======================================
  Files          44       44           
  Lines        3590     3592    +2     
=======================================
+ Hits         3164     3166    +2     
  Misses        426      426           

Files Changed	Coverage Δ
src/maggma/stores/aws.py	91.30% <100.00%> (+0.08%)	⬆️

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.