--expected-size (string) This argument specifies the expected size of a stream in terms of bytes. Note that this argument is needed only when a stream is being uploaded to s3 and the size is larger than 5GB. Failure to include this argument under these conditions may result in a failed upload. due to too many parts in upload.
I noticed something was wrong after bytes transmitted out of the network interface dropped to 0. I manually stopped the process about 15 minutes after traffic stopped. I saw the above error in the logs but everything before aws in the pipeline was still chugging along.
With the release of 1.10.57, we replaced the underlying architecture for streaming and this should no longer be an issue as errors handling is much resilient and quicker in shutting down the entire transfer process. There should be no more hanging as well. Resolving issue.