Fix concurrency bug in S3 parallel multipart download

[davidh44]

Motivation and Context

S3 parallel multipart downloads with MultipartS3AsyncClient can enter an unrecoverable infinite loop, causing threads to spin at 100% CPU indefinitely

MultipartDownloadResumeContext uses a non-thread-safe TreeSet that is accessed concurrently from CompletableFuture.whenComplete() callbacks in the parallel download path. Concurrent TreeSet.add() corrupts the red-black tree, creating circular references that cause infinite loops in TreeMap.put().

Modifications

• Replace TreeSet with ConcurrentSkipListSet
• Replace Long with AtomicLong for bytesToLastCompletedParts

Testing

Added unit tests

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)

Checklist

• I have read the CONTRIBUTING document
• Local run of mvn install succeeds
• My code follows the code style of this project
• My change requires a change to the Javadoc documentation
• I have updated the Javadoc documentation accordingly
• I have added tests to cover my changes
• All new and existing tests passed
• I have added a changelog entry. Adding a new entry must be accomplished by running the scripts/new-change script and following the instructions. Commit the new file created by the script in .changes/next-release with your changes.
• My change is to implement 1.11 parity feature and I have updated LaunchChangelog

License

• I confirm that this pull request can be released under the Apache 2 license

[github-actions]

This pull request has been closed and the conversation has been locked. Comments on closed PRs are hard for our team to see. If you need more assistance, please open a new issue that references this one.