TransferManager download does not complete until after SocketTimeoutException is encountered

[bakkenlab]

We are trying to download a large (6GB) file from S3. On our slow connection, this takes about 20 minutes. Initially, we were seeing intermittent SocketTimeoutExceptions, although when we didn't see that issue, things worked fine. We raised the socket timeout to 25 minutes in response.

Now the download seems to hang after actually pulling down all the data. Initially, we were just using the Download.waitForCompletion(), but we wanted more feedback. We added a loop to report the progress, and it showed the percentage as '99.98596920967103' for 12 minutes. Then it encountered "SocketTimeoutException: Read timed out" and indicated that it would retry the download. The completion percentage showed as '100' for about 20 seconds. Then it completed and things worked correctly from there.

[varunnvs92]

When downloading large files, networking issues can occur often and we added retry logic. Thats why you see message that we would retry the download. This is a networking issue and not an SDK bug. I am not sure what is the ask here from our end?

[bakkenlab]

To better phrase my concern, when we tried downloading the large file on a slow connection, it would download with the retry logic included (as you stated). What we noticed though was that our download would "hang" at 99.8% until the socketTimeout completed the time we passed to it.

For example, our file took around 15 minutes to download to 99.8%, but then it hanged on that 99.8% for another 10 minutes, reaching the total 25 minute socket timeout we passed to the config. So it seems as though the socketTimeout blocked the download from completing until it reached that time limit. After that 25 minutes passed, we saw the download reach 100% and it continued to do the next steps in our program around 20 seconds after the download finalized.

We're curious if that is the behavior we should be seeing from the socketTimeout when we originally expected it to keep our connection open for max 25 minutes and then close once the download completed (around the 15 minute mark) versus staying attached to the timeout.

[dagnir]

Do you have parallel downloads enabled? If the the TransferManager spawns multiple connections to download parts of your object in parallel and one of those connections gets timed out on reading a response from S3, then it will take the full 25 minutes until the read() on the connection throws SocketTimeoutException and the client tries again.

So for example, if two connections are created, and the first one completes downloading the first half of the object but the second connection gets timed out, you'd see the download hang at 50% until 25 minutes is reached, after which the second connection throws SocketTimeoutException and then request for the second half gets retried again.

[dagnir]

Closing due to no response. Please feel free to reopen if you're still experiencing this issue or have further questions.