-
Notifications
You must be signed in to change notification settings - Fork 826
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix request cancellation issue in the AWS CRT-based S3 client that co… #4955
Conversation
…uld lead to memory leak
@@ -18,25 +18,24 @@ | |||
import java.util.function.Function; | |||
import software.amazon.awssdk.annotations.SdkInternalApi; | |||
import software.amazon.awssdk.crt.s3.ResumeToken; | |||
import software.amazon.awssdk.crt.s3.S3MetaRequest; | |||
|
|||
/** | |||
* An observable that notifies the observer {@link S3CrtAsyncHttpClient} to pause the request. | |||
*/ | |||
@SdkInternalApi |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This class should've been a protected API. I will create a separate PR to fix it.
System.out.printf("%s %s%n", sdkHttpRequest.method(), sdkHttpRequest.encodedPath()); | ||
headers.forEach((k, strings) -> System.out.printf("%s, %s%n", k, strings)); | ||
System.out.println(); | ||
log.debug(() -> String.format("%s %s%n", sdkHttpRequest.method(), sdkHttpRequest.encodedPath())); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I changed this because I noticed it produced a lot of (unnecessary) logs.
@@ -215,23 +219,6 @@ private static S3MetaRequestOptions.MetaRequestType requestType(AsyncExecuteRequ | |||
return S3MetaRequestOptions.MetaRequestType.DEFAULT; | |||
} | |||
|
|||
private static void closeResourceCallback(CompletableFuture<Void> executeFuture, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I moved this to S3CrtResponseHandlerAdapter because I figured it makes more sense to manage the closure of the S3MetaRequest in one place.
...c/main/java/software/amazon/awssdk/services/s3/internal/crt/S3CrtResponseHandlerAdapter.java
Outdated
Show resolved
Hide resolved
...c/main/java/software/amazon/awssdk/services/s3/internal/crt/S3CrtResponseHandlerAdapter.java
Outdated
Show resolved
Hide resolved
@@ -160,7 +160,9 @@ void listeners_reports_ErrorsWhenCancelled(WireMockRuntimeInfo wm) throws Interr | |||
assertThat(transferListener.getExceptionCaught()).isInstanceOf(CancellationException.class); | |||
assertThat(transferListener.isTransferComplete()).isFalse(); | |||
assertThat(transferListener.isTransferInitiated()).isTrue(); | |||
assertMockOnFailure(transferListenerMock); | |||
Mockito.verify(transferListenerMock, times(1)).transferFailed(ArgumentMatchers.any()); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The failure of the test actually verified this fix. Before the fix, the request would never cancelled, so we'd still get data after cancelling future. After the fix, the request would get cancelled properly, so Mockito.verify(transferListenerMock, times(1)).bytesTransferred(ArgumentMatchers.any());
may not be invoked at all.
9e797ee
to
4679773
Compare
4679773
to
183a0d8
Compare
|
* Update to next snapshot version: 2.23.10-SNAPSHOT * Amazon EC2 Container Service Update: Documentation updates for Amazon ECS. * AWS Outposts Update: DeviceSerialNumber parameter is now optional in StartConnection API * Amazon Relational Database Service Update: This release adds support for Aurora Limitless Database. * Amazon Elastic Compute Cloud Update: Introduced a new clientToken request parameter on CreateNetworkAcl and CreateRouteTable APIs. The clientToken parameter allows idempotent operations on the APIs. * AWS Storage Gateway Update: Add DeprecationDate and SoftwareVersion to response of ListGateways. * Release 2.23.10. Updated CHANGELOG.md, README.md and all pom.xml. * Update to next snapshot version: 2.23.11-SNAPSHOT * Endpoint based auth scheme resolver should honor endpoint overrides (#4838) * Amazon Lightsail Update: This release adds support for IPv6-only instance plans. * AWS Certificate Manager Private Certificate Authority Update: AWS Private CA now supports an option to omit the CDP extension from issued certificates, when CRL revocation is enabled. * Release 2.23.11. Updated CHANGELOG.md, README.md and all pom.xml. * Update to next snapshot version: 2.23.12-SNAPSHOT * A few javadoc corrections (#4848) * Performance improvements (#4850) * Amazon Connect Service Update: Update list and string length limits for predefined attributes. * Inspector2 Update: This release adds ECR container image scanning based on their lastRecordedPullTime. * Amazon SageMaker Service Update: Amazon SageMaker Automatic Model Tuning now provides an API to programmatically delete tuning jobs. * Release 2.23.12. Updated CHANGELOG.md, README.md and all pom.xml. * Update to next snapshot version: 2.23.13-SNAPSHOT * Add option to disable IMDS v1 fallback when token is not returned (#4840) * Add option to disable IMDS v1 fallback when token is not returned for IMDS credential calls * updating IMDS fallback message * Update requestcompression codegen name (#4851) * Update codegen to get requestcompression instead of requestCompression * Update test models * Fix for issue #4720 , Cross Region enabled Clients created in US-EAST-1 will by internally disable global endpoint and do a regional endpoint call. (#4849) * Fix for issue #4720 , Cross Region enabled Clients created in US-EAST-1 will by internally disable global endpoint and do a regional endpoint call. * Handled Zoe's comments * Update test case where endppoint resolution is checked for atleast once (#4854) * Auto Scaling Update: EC2 Auto Scaling customers who use attribute based instance-type selection can now intuitively define their Spot instances price protection limit as a percentage of the lowest priced On-Demand instance type. * Amazon Relational Database Service Update: Introduced support for the InsufficientDBInstanceCapacityFault error in the RDS RestoreDBClusterFromSnapshot and RestoreDBClusterToPointInTime API methods. This provides enhanced error handling, ensuring a more robust experience. * Amazon Elastic Compute Cloud Update: EC2 Fleet customers who use attribute based instance-type selection can now intuitively define their Spot instances price protection limit as a percentage of the lowest priced On-Demand instance type. * AmazonMWAA Update: This release adds MAINTENANCE environment status for Amazon MWAA environments. * Amazon Import/Export Snowball Update: Modified description of createaddress to include direction to add path when providing a JSON file. * Amazon Comprehend Update: Comprehend PII analysis now supports Spanish input documents. * Updated endpoints.json and partitions.json. * Release 2.23.13. Updated CHANGELOG.md, README.md and all pom.xml. * Update to next snapshot version: 2.23.14-SNAPSHOT * Minor GitHub workflow changes (#4847) * chore: a more polite closed issue message * chore: relax stale issue timing * Reduce memory usage in S3 when plugins aren't used. (#4857) Modifying the SDK client configuration at the request level uses extra memory, because it requires copying the existing configuration map. Before this change, we always modify the client configuration with each request to add the client instance. This change moves this modification to the client-creation, so that the configuration doesn't need to be copied unless plugins also modify the configuration. This change also removes the conditional logic that only added the client for S3, which simplifies the code generator. * Amazon DataZone Update: Add new skipDeletionCheck to DeleteDomain. Add new skipDeletionCheck to DeleteProject which also automatically deletes dependent objects * Amazon Route 53 Update: Update the SDKs for text changes in the APIs. * Release 2.23.14. Updated CHANGELOG.md, README.md and all pom.xml. * Update to next snapshot version: 2.23.15-SNAPSHOT * Performance improvements for SRA changes. (#4864) 1. Do not call SignerProperty.toString() whenever required properties are read. 2. Check if an SdkHttpRequest is an SdkHttpFullRequest before performing a full conversion to the latter. * Performance optimizations in ExecutionInterceptorChain (#4863) 1. Removed apply*Hack method. This method wasn't doing anything, because its side effects were immediately overridden. 2. Do not modify the execution context in the modify* methods unless the interceptor requests a modification. The time added by the comparisons are less than the cost of always modifying the interceptor context, because modifying the execution context is relatively rare. * Reduce memory usage for chunk-encoded streaming uploads, like those used by flexible checksums in S3. (#4858) Before this change, our chunk encoding logic would copy customer data five times: 1. [From the customer's stream into a byte array.](https://github.com/aws/aws-sdk-java-v2/blob/6040b2be6731e4b5ef64e775a2cfffb07d76766c/core/http-auth-aws/src/main/java/software/amazon/awssdk/http/auth/aws/internal/signer/chunkedencoding/ChunkedEncodedInputStream.java#L106-L107) 2. [From the byte array into a slightly smaller byte array.](https://github.com/aws/aws-sdk-java-v2/blob/6040b2be6731e4b5ef64e775a2cfffb07d76766c/core/http-auth-aws/src/main/java/software/amazon/awssdk/http/auth/aws/internal/signer/chunkedencoding/ChunkedEncodedInputStream.java#L111) 3. [From the smaller byte array into a byte array output stream.](https://github.com/aws/aws-sdk-java-v2/blob/6040b2be6731e4b5ef64e775a2cfffb07d76766c/core/http-auth-aws/src/main/java/software/amazon/awssdk/http/auth/aws/internal/signer/chunkedencoding/ChunkedEncodedInputStream.java#L171) 4. [From the byte array output stream into an array.](https://github.com/aws/aws-sdk-java-v2/blob/6040b2be6731e4b5ef64e775a2cfffb07d76766c/core/http-auth-aws/src/main/java/software/amazon/awssdk/http/auth/aws/internal/signer/chunkedencoding/ChunkedEncodedInputStream.java#L149) 5. [From the array into the output array.](https://github.com/aws/aws-sdk-java-v2/blob/6040b2be6731e4b5ef64e775a2cfffb07d76766c/core/http-auth-aws/src/main/java/software/amazon/awssdk/http/auth/aws/internal/signer/chunkedencoding/ChunkedEncodedInputStream.java#L85) After this change, the logic will copy the data twice: 1. From the customer's stream into a byte array. 2. From the byte array into the output array. There's a path to make it only one copy, but it requires the chunk encoded input stream to know the length of the underlying stream so that it can detect when the last chunk will be encountered. This will require additional piping, so we can do it in a follow-up PR. * Add support for disable IMDS v1 fallback for regions * Various Maven fixes: 1. consume DynamoDB Local from Maven instead of S3 2. Fix Maven warnings (#4859) * Optimizing profile file loading * Elastic Load Balancing Update: This release enables unhealthy target draining intervals for Network Load Balancers. * AWS CloudFormation Update: CloudFormation IaC generator allows you to scan existing resources in your account and select resources to generate a template for a new or existing CloudFormation stack. * AWS Glue Update: Update page size limits for GetJobRuns and GetTriggers APIs. * Amazon Simple Systems Manager (SSM) Update: This release adds an optional Duration parameter to StateManager Associations. This allows customers to specify how long an apply-only-on-cron association execution should run. Once the specified Duration is out all the ongoing cancellable commands or automations are cancelled. * Updated endpoints.json and partitions.json. * Release 2.23.15. Updated CHANGELOG.md, README.md and all pom.xml. * Update to next snapshot version: 2.23.16-SNAPSHOT * Fix check for release tag (#4871) The original check checks whether the tag *exists* in the local and then tries to remove it from the public repo. However, it's possible that the tag is in the private repo, where it might not exists (refs from both remotes are present locally). This commit fixes this by explicitly check if the tag exists in the public repo and only then will it try to delete it. * Exclude endpoint provider tests to speed up the build (#4860) * Exclude endpoint provider tests to speed up the build * Update CI and add buildspec for endpoints test * Fix cloudformation template * Update japicmp config (#4872) * caching resolved values * adding back commented test * Changed to use lazy value for credential provider * Activating SRA auth for 25 services with control plane type APIs (#4874) * Amazon Cognito Identity Provider Update: Added CreateIdentityProvider and UpdateIdentityProvider details for new SAML IdP features * Amazon Neptune Graph Update: Adding new APIs in SDK for Amazon Neptune Analytics. These APIs include operations to execute, cancel, list queries and get the graph summary. * AWS Elemental MediaConvert Update: This release includes support for broadcast-mixed audio description tracks. * Amazon Interactive Video Service Update: This release introduces a new resource Playback Restriction Policy which can be used to geo-restrict or domain-restrict channel stream playback when associated with a channel. New APIs to support this resource were introduced in the form of Create/Delete/Get/Update/List. * Amazon Managed Blockchain Query Update: This release adds support for transactions that have not reached finality. It also removes support for the status property from the response of the GetTransaction operation. You can use the confirmationStatus and executionStatus properties to determine the status of the transaction. * Updated endpoints.json and partitions.json. * Release 2.23.16. Updated CHANGELOG.md, README.md and all pom.xml. * Add option to disable EC2 metadata (IMDS) calls without token (#4866) * Add option to disable IMDS v1 fallback when token is not returned (#4840) * Add support for disable IMDS v1 fallback for regions * Cache whether CRT is available in the flexible checksum algorithm. (#4878) * Update to next snapshot version: 2.23.17-SNAPSHOT * Bump maven wrapper plugin version (#4879) * Performance improvement for sigv4 signing. (#4867) 1. When trimming and removing consecutive spaces during sigv4 normalization, copy word-by-word instead of character-by-character. This reduces the overhead of range and encoding checks in string builder. 2. Increase starting string builder size for canonical headers, to limit resizing (2048 worked well for DynamoDB's get-item). 3. Use a switch statement for whitespace checks instead of consecutive if statements. On my compiler, the switch statement compiles to a jump table which runs quicker. * Expose futureCompletionExecutor on S3 CRT client (#4880) * Expose futureCompletionExecutor on S3 CRT client * Adderss feedback and fix build * Amazon SageMaker Service Update: Amazon SageMaker Canvas adds GenerativeAiSettings support for CanvasAppSettings. * Amazon DynamoDB Update: Any number of users can execute up to 50 concurrent restores (any type of restore) in a given account. * Updated endpoints.json and partitions.json. * Release 2.23.17. Updated CHANGELOG.md, README.md and all pom.xml. * Enable compiled endpoint rules for the second wave (#4883) Services included are: backupstorage, codecatalyst, cognitoidentity, kinesis, mediastoredata, transcribe, transcribestreaming * Revert "Performance improvement for sigv4 signing. (#4867)" This reverts commit 0ab7f75. * Update to next snapshot version: 2.23.18-SNAPSHOT * AWS Glue Update: Introduce Catalog Encryption Role within Glue Data Catalog Settings. Introduce SASL/PLAIN as an authentication method for Glue Kafka connections * Amazon WorkSpaces Update: Added definitions of various WorkSpace states * Release 2.23.18. Updated CHANGELOG.md, README.md and all pom.xml. * update aws-sdk-java pom to add imds and dyanmodb-enhanced (#4890) * Performance improvement for sigv4 signing. (#4891) This is a reintroduction of the reverted commit #4867. This includes a fix to the issue that caused the revert: improper handling of empty header values. * Update to next snapshot version: 2.23.19-SNAPSHOT * Delete CloudSearchv2IntegrationTest (#4888) * Fix tag deletion command (#4892) * including S3 Access Grants Plugin as part of Java SDK Bundle (#4881) * including S3 Access Grants Plugin as part of Java SDK Bundle --------- Co-authored-by: Shiva Kumar Mukkapati <mshvkmr@amazon.com> * Archive old changelog entries (< 2.23.0) (#4873) - .json files files for each version are grouped under minor versions in .changes - Markdown files for each minor version are created in changelogs/ directory - Changelog scripts updated to add a link to older version in the generated changelog * Changing indentation of config files to 4 spaces (#4889) * Amazon EC2 Container Service Update: This release is a documentation only update to address customer issues. * AWS WAFV2 Update: You can now delete an API key that you've created for use with your CAPTCHA JavaScript integration API. * Amazon OpenSearch Service Update: This release adds clear visibility to the customers on the changes that they make on the domain. * AWS AppSync Update: Support for environment variables in AppSync GraphQL APIs * Amazon CloudWatch Logs Update: This release adds a new field, logGroupArn, to the response of the logs:DescribeLogGroups action. * Amazon Elasticsearch Service Update: This release adds clear visibility to the customers on the changes that they make on the domain. * Release 2.23.19. Updated CHANGELOG.md, README.md and all pom.xml. * Update to next snapshot version: 2.23.20-SNAPSHOT * Bump CRT version and expose setting memory limits for S3 calls (#4885) * Bump aws-crt version to 0.29.9 * Exposes a setting to set the memory limit when making asynchronous calls with the CRT-based S3 client * Activating SRA for this service (#4896) * Fix request cancellation logic in the AWS CRT Sync HTTP client (#4887) * Fix request cancellation logic in the AWS CRT Sync HTTP client * Address feedback * AWS DataSync Update: AWS DataSync now supports manifests for specifying files or objects to transfer. * Amazon Redshift Update: LisRecommendations API to fetch Amazon Redshift Advisor recommendations. * Amazon Lex Model Building V2 Update: This release introduces a new bot replication feature as part of Lex Global Resiliency offering. This feature leverages a new set of APIs that allow customers to create bot replicas and replicate changes to bots across regions. * Updated endpoints.json and partitions.json. * Release 2.23.20. Updated CHANGELOG.md, README.md and all pom.xml. * Update to next snapshot version: 2.23.21-SNAPSHOT * Amazon QuickSight Update: General Interactions for Visuals; Waterfall Chart Color Configuration; Documentation Update * AWS CodePipeline Update: Add ability to execute pipelines with new parallel & queued execution modes and add support for triggers with filtering on branches and file paths. * Amazon WorkSpaces Update: This release introduces User-Decoupling feature. This feature allows Workspaces Core customers to provision workspaces without providing users. CreateWorkspaces and DescribeWorkspaces APIs will now take a new optional parameter "WorkspaceName". * Release 2.23.21. Updated CHANGELOG.md, README.md and all pom.xml. * Update to next snapshot version: 2.23.22-SNAPSHOT * S3 should configure signer properties (#4856) * Configure modeled signer properties for endpoint based auth scheme resolver * Refactor the logic to use Knowledge Indexes * Enable CRC32 for PUT for MultipartS3AsyncClient (#4898) * Fix bug to allow MpuS3Client to PUT COPY with SSE-C and Checksum * Enable CRC32 for Multipart PUT COPY * Address comments * add changelog and update checksum check * Address comments * update javadocs * Add unit tests * Amazon Prometheus Service Update: Overall documentation updates. * Braket Update: Creating a job will result in DeviceOfflineException when using an offline device, and DeviceRetiredException when using a retired device. * AWS IoT Update: This release allows AWS IoT Core users to enable Online Certificate Status Protocol (OCSP) Stapling for TLS X.509 Server Certificates when creating and updating AWS IoT Domain Configurations with Custom Domain. * AWS Batch Update: This feature allows Batch to support configuration of repository credentials for jobs running on ECS * Cost Optimization Hub Update: Adding includeMemberAccounts field to the response of ListEnrollmentStatuses API. * Amazon EC2 Container Service Update: Documentation only update for Amazon ECS. * AWS Price List Service Update: Add Throttling Exception to all APIs. * Updated endpoints.json and partitions.json. * Release 2.23.22. Updated CHANGELOG.md, README.md and all pom.xml. * Bumping minor version to 2.24.0 due to core refactoring and moving away from S3 signing attributes * Update to next snapshot version: 2.24.1-SNAPSHOT * Fix changelogs for 2.24.0 (#4913) Changes were incorrectly marked as 2.23.22. Arhived changes up to and including 2.23.21, and changes 2.23.22 changes to 2.24.0. * Correct the javadoc for the main class (#4915) * Remove uses of legacy singer attributes (#4914) * AWS AppSync Update: Adds support for new options on GraphqlAPIs, Resolvers and Data Sources for emitting Amazon CloudWatch metrics for enhanced monitoring of AppSync APIs. * Amazon CloudWatch Update: This release enables PutMetricData API request payload compression by default. * Amazon Neptune Graph Update: Adding a new option "parameters" for data plane api ExecuteQuery to support running parameterized query via SDK. * Amazon Route 53 Domains Update: This release adds bill contact support for RegisterDomain, TransferDomain, UpdateDomainContact and GetDomainDetail API. * Release 2.24.1. Updated CHANGELOG.md, README.md and all pom.xml. * Update to next snapshot version: 2.24.2-SNAPSHOT * Handle header lists in URLConnection client (#4897) * Add additional helper methods to DynamoDB Expression (#4765) These helpers make it easier to write complex queries without having to use the existing overly verbose static methods. This lets users write more clear and concise code that makes is obvious how the expressions are logically combined together. #4764 Co-authored-by: Andy Kiesler <kiesler@amazon.com> Co-authored-by: Dongie Agnir <261310+dagnir@users.noreply.github.com> * Trim object metadata keys for PUT and MPU (#4902) This brings the V2's behavior in line with 1.x * Backfilling changelog to include signing attribute changes (#4919) * Backfilling changelog to include signing attribute changes * Moved changes to release .json file and regenerated changelog * AWS Resource Explorer Update: Resource Explorer now uses newly supported IPv4 'amazonaws.com' endpoints by default. * AWS Marketplace Catalog Service Update: AWS Marketplace Catalog API now supports setting intent on requests * Amazon Security Lake Update: Documentation updates for Security Lake * Amazon Lightsail Update: This release adds support to upgrade the major version of a database. * Updated endpoints.json and partitions.json. * Release 2.24.2. Updated CHANGELOG.md, README.md and all pom.xml. * Update CloudFront test resource name (#4925) Make it clear that the resource is not to be deleted. * Update to next snapshot version: 2.24.3-SNAPSHOT * Fixed an issue where NPE would be thrown for empty event (#4922) * Fix for Issue #4912 where Cross Region S3 Access using AWS_GLOBAL Endpoint failed. (#4921) * Activating SRA auth for approximately half of remaining services APIs (#4926) * Enable compiled endpoint rules for the third wave (#4927) Services included are: - accessanalyzer - account - acm - acmpca - alexaforbusiness - amp - amplify - amplifybackend - amplifyuibuilder - apigatewaymanagementapi - appconfig - appfabric - appflow - appintegrations - applicationautoscaling - applicationcostprofiler - applicationdiscovery - applicationinsights - cloudfrontkeyvaluestore - cognitoidentityprovider - eks - emrcontainers - iam - iot - marketplacemetering - paymentcryptography - securityhub * QBusiness Update: This release adds the metadata-boosting feature, which allows customers to easily fine-tune the underlying ranking of retrieved RAG passages in order to optimize Q&A answer relevance. It also adds new feedback reasons for the PutFeedback API. * Amazon Lookout for Equipment Update: This feature allows customers to see pointwise model diagnostics results for their models. * AWS Control Tower Update: Adds support for new Baseline and EnabledBaseline APIs for automating multi-account governance. * Updated endpoints.json and partitions.json. * Release 2.24.3. Updated CHANGELOG.md, README.md and all pom.xml. * Update to next snapshot version: 2.24.4-SNAPSHOT * Amazon Polly Update: Amazon Polly adds 1 new voice - Burcu (tr-TR) * Amazon OpenSearch Service Update: Adds additional supported instance types. * AWS Secrets Manager Update: Doc only update for Secrets Manager * Amazon GuardDuty Update: Marked fields IpAddressV4, PrivateIpAddress, Email as Sensitive. * AWS Artifact Update: This is the initial SDK release for AWS Artifact. AWS Artifact provides on-demand access to compliance and third-party compliance reports. This release includes access to List and Get reports, along with their metadata. This release also includes access to AWS Artifact notifications settings. * Amazon HealthLake Update: This release adds a new response parameter, JobProgressReport, to the DescribeFHIRImportJob and ListFHIRImportJobs API operation. JobProgressReport provides details on the progress of the import job on the server. * AWS CodePipeline Update: Add ability to override timeout on action level. * Amazon Detective Update: Doc only updates for content enhancement * Amazon SageMaker Service Update: This release adds a new API UpdateClusterSoftware for SageMaker HyperPod. This API allows users to patch HyperPod clusters with latest platform softwares. * Updated endpoints.json and partitions.json. * Release 2.24.4. Updated CHANGELOG.md, README.md and all pom.xml. * Update to next snapshot version: 2.24.5-SNAPSHOT * update all contributors hall (#4936) * Update codebuild-ci.yml to trigger CI on doc changes (#4928) * Amazon Connect Participant Service Update: Doc only update to GetTranscript API reference guide to inform users about presence of events in the chat transcript. * Amazon Kinesis Firehose Update: This release adds support for Data Message Extraction for decompressed CloudWatch logs, and to use a custom file extension or time zone for S3 destinations. * Amazon Simple Notification Service Update: This release marks phone numbers as sensitive inputs. * Amazon EMR Update: adds fine grained control over Unhealthy Node Replacement to Amazon ElasticMapReduce * AWS Lambda Update: Documentation-only updates for Lambda to clarify a number of existing actions and properties. * Amazon Relational Database Service Update: Doc only update for a valid option in DB parameter group * Updated endpoints.json and partitions.json. * Release 2.24.5. Updated CHANGELOG.md, README.md and all pom.xml. * Add decision log for migration tool (#4853) * Update to next snapshot version: 2.24.6-SNAPSHOT * Fixed the issue where invoking abort on AsyncResponseTransformer.toBl… (#4932) * Fixed the issue where invoking abort on AsyncResponseTransformer.toBlockingInputStream could lead to memory leak * Fix build * Remove doAfterAbort and fix build * Make test less flaky * Pause/Resume Upload for Transfer Manager with Java S3Client (#4908) (#4937) * Pause/Resume Upload for Transfer Manager with Java S3Client * Fix checkstyle and equalsVerifier * temporarily disable failing test * Address comments * ifPresent check for PauseObservable * Address comments * Wrap subscriber with PausibleUpload * add changelog * integ test upload resume with different TMs * Rename to PausableUpload * Add unit tests * Refactor KnownLengthUploadHelper * Add unit tests * Move PauseObservable and S3ResumeToken out of internal * Refactor UploadWithKnownContentLengthHelper * Address comments and update tests * Fix import order * Extract Subscriber and MpuRequestContext to separate classes * Update Subscriber and add tests * Create separate tmJava with multipartClient in integ test * Address comments * Remove TestInternalApis and refacor tests * Address comment * AWS Config Update: Documentation updates for the AWS Config CLI * Amazon Interactive Video Service Update: Changed description for latencyMode in Create/UpdateChannel and Channel/ChannelSummary. * Amazon Keyspaces Update: Documentation updates for Amazon Keyspaces * AWS Amplify Update: This release contains API changes that enable users to configure their Amplify domains with their own custom SSL/TLS certificate. * AWS MediaTailor Update: MediaTailor: marking #AdBreak.OffsetMillis as required. * chatbot Update: This release adds support for AWS Chatbot. You can now monitor, operate, and troubleshoot your AWS resources with interactive ChatOps using the AWS SDK. * Release 2.24.6. Updated CHANGELOG.md, README.md and all pom.xml. * Update to next snapshot version: 2.24.7-SNAPSHOT * changelog * Fix codebuild-ci.yml (#4946) * AWS Lambda Update: Add .NET 8 (dotnet8) Runtime support to AWS Lambda. * Amazon Kinesis Firehose Update: This release updates a few Firehose related APIs. * Amazon DynamoDB Update: Publishing quick fix for doc only update. * Release 2.24.7. Updated CHANGELOG.md, README.md and all pom.xml. * Update to next snapshot version: 2.24.8-SNAPSHOT * Fix flaky tests (#4947) * Also delete local release tag (#4943) If the tag exists in public, then it exists locally too since we fetched all of public when adding it as a remote. Also delete this local tag so it doesn't fail when we recreate it. * Update documentation for FUTURE_COMPLETION_EXECUTOR (#4950) * Add content header if the Request doesnot has one for CRT sync Http clients (#4920) * Add content header if the Request doesnot has one for CRT sync Http clients * Content length updated in the Marshallers * Content length updation done in the Marshallers after internal comment * Handled Matts comments on PR * Updated MarshallersAddContentLengthTest to Junit5 * Updated to initiate theCheckbuilds * Handle sonar raised issues * Amazon Simple Systems Manager (SSM) Update: This release adds support for sharing Systems Manager parameters with other AWS accounts. * AWS Elemental MediaLive Update: MediaLive now supports the ability to restart pipelines in a running channel. * AWS IoT Events Update: Increase the maximum length of descriptions for Inputs, Detector Models, and Alarm Models * Amazon Lookout for Equipment Update: This release adds a field exposing model quality to read APIs for models. It also adds a model quality field to the API response when creating an inference scheduler. * Release 2.24.8. Updated CHANGELOG.md, README.md and all pom.xml. * Update to next snapshot version: 2.24.9-SNAPSHOT * Fix request cancellation issue in the AWS CRT-based S3 client that co… (#4955) * Fix request cancellation issue in the AWS CRT-based S3 client that could lead to memory leak * Fix build and address feedback * Fix test * Fix build * Amazon CloudWatch Internet Monitor Update: This release adds IPv4 prefixes to health events * Amazon Kinesis Video Streams Update: Increasing NextToken parameter length restriction for List APIs from 512 to 1024. * Release 2.24.9. Updated CHANGELOG.md, README.md and all pom.xml. * Update to next snapshot version: 2.24.10-SNAPSHOT * Choose next potential scheme if signer not found (#4952) * Choose next potential scheme if signer not found This commit fixes the generated auth scheme selection mechanism by ensuring that auth scheme is able to provide a signer implementation (i.e. does not throw an exception) before selecting the auth scheme. This is useful in situations where the signer implementation relies on components thay may not be available at runtime, such as the AwsV4aAuthScheme, which relies on CRT which is an optional dependency. Additional changes: - introduce crt-unavailable-tests which is a good candidate for testing behavior of components that use CRT when CRT is not present. - modify DefaultAwsV4aAuthScheme so that errors thrown during static initialization of the singleton holder for the Sigv4a signer are caught and thrown directly instead of resulting in ExceptionInInitializerError * Review comments * Fix module version (#4961) * Revert "Handle header lists in URLConnection client (#4897)" (#4960) This reverts commit b51a51c. This change may be causing issues with propagation of trace IDs. * Mulitpart download workaround (#4945) * Make Transfer Manager work by default with MultipartS3AsyncClient. * checkstyle --------- Co-authored-by: AWS <> Co-authored-by: aws-sdk-java-automation <43143862+aws-sdk-java-automation@users.noreply.github.com> Co-authored-by: Manuel Sugawara <sugmanue@amazon.com> Co-authored-by: Debora N. Ito <476307+debora-ito@users.noreply.github.com> Co-authored-by: Matthew Miller <millem@amazon.com> Co-authored-by: Anna-Karin Salander <salande@amazon.com> Co-authored-by: David Ho <70000000+davidh44@users.noreply.github.com> Co-authored-by: John Viegas <70235430+joviegas@users.noreply.github.com> Co-authored-by: Tom Keller <1083460+kellertk@users.noreply.github.com> Co-authored-by: Zoe Wang <33073555+zoewangg@users.noreply.github.com> Co-authored-by: Dongie Agnir <261310+dagnir@users.noreply.github.com> Co-authored-by: Dongie Agnir <dongie@amazon.com> Co-authored-by: shiva kumar <62441208+shiva958@users.noreply.github.com> Co-authored-by: Shiva Kumar Mukkapati <mshvkmr@amazon.com> Co-authored-by: Andy Kiesler <4186292+akiesler@users.noreply.github.com> Co-authored-by: Andy Kiesler <kiesler@amazon.com>
* AmazonMWAA Update: This release adds MAINTENANCE environment status for Amazon MWAA environments. * Amazon Import/Export Snowball Update: Modified description of createaddress to include direction to add path when providing a JSON file. * Amazon Comprehend Update: Comprehend PII analysis now supports Spanish input documents. * Updated endpoints.json and partitions.json. * Release 2.23.13. Updated CHANGELOG.md, README.md and all pom.xml. * Update to next snapshot version: 2.23.14-SNAPSHOT * Minor GitHub workflow changes (#4847) * chore: a more polite closed issue message * chore: relax stale issue timing * Reduce memory usage in S3 when plugins aren't used. (#4857) Modifying the SDK client configuration at the request level uses extra memory, because it requires copying the existing configuration map. Before this change, we always modify the client configuration with each request to add the client instance. This change moves this modification to the client-creation, so that the configuration doesn't need to be copied unless plugins also modify the configuration. This change also removes the conditional logic that only added the client for S3, which simplifies the code generator. * Amazon DataZone Update: Add new skipDeletionCheck to DeleteDomain. Add new skipDeletionCheck to DeleteProject which also automatically deletes dependent objects * Amazon Route 53 Update: Update the SDKs for text changes in the APIs. * Release 2.23.14. Updated CHANGELOG.md, README.md and all pom.xml. * Update to next snapshot version: 2.23.15-SNAPSHOT * Performance improvements for SRA changes. (#4864) 1. Do not call SignerProperty.toString() whenever required properties are read. 2. Check if an SdkHttpRequest is an SdkHttpFullRequest before performing a full conversion to the latter. * Performance optimizations in ExecutionInterceptorChain (#4863) 1. Removed apply*Hack method. This method wasn't doing anything, because its side effects were immediately overridden. 2. Do not modify the execution context in the modify* methods unless the interceptor requests a modification. The time added by the comparisons are less than the cost of always modifying the interceptor context, because modifying the execution context is relatively rare. * Reduce memory usage for chunk-encoded streaming uploads, like those used by flexible checksums in S3. (#4858) Before this change, our chunk encoding logic would copy customer data five times: 1. [From the customer's stream into a byte array.](https://github.com/aws/aws-sdk-java-v2/blob/6040b2be6731e4b5ef64e775a2cfffb07d76766c/core/http-auth-aws/src/main/java/software/amazon/awssdk/http/auth/aws/internal/signer/chunkedencoding/ChunkedEncodedInputStream.java#L106-L107) 2. [From the byte array into a slightly smaller byte array.](https://github.com/aws/aws-sdk-java-v2/blob/6040b2be6731e4b5ef64e775a2cfffb07d76766c/core/http-auth-aws/src/main/java/software/amazon/awssdk/http/auth/aws/internal/signer/chunkedencoding/ChunkedEncodedInputStream.java#L111) 3. [From the smaller byte array into a byte array output stream.](https://github.com/aws/aws-sdk-java-v2/blob/6040b2be6731e4b5ef64e775a2cfffb07d76766c/core/http-auth-aws/src/main/java/software/amazon/awssdk/http/auth/aws/internal/signer/chunkedencoding/ChunkedEncodedInputStream.java#L171) 4. [From the byte array output stream into an array.](https://github.com/aws/aws-sdk-java-v2/blob/6040b2be6731e4b5ef64e775a2cfffb07d76766c/core/http-auth-aws/src/main/java/software/amazon/awssdk/http/auth/aws/internal/signer/chunkedencoding/ChunkedEncodedInputStream.java#L149) 5. [From the array into the output array.](https://github.com/aws/aws-sdk-java-v2/blob/6040b2be6731e4b5ef64e775a2cfffb07d76766c/core/http-auth-aws/src/main/java/software/amazon/awssdk/http/auth/aws/internal/signer/chunkedencoding/ChunkedEncodedInputStream.java#L85) After this change, the logic will copy the data twice: 1. From the customer's stream into a byte array. 2. From the byte array into the output array. There's a path to make it only one copy, but it requires the chunk encoded input stream to know the length of the underlying stream so that it can detect when the last chunk will be encountered. This will require additional piping, so we can do it in a follow-up PR. * Add support for disable IMDS v1 fallback for regions * Various Maven fixes: 1. consume DynamoDB Local from Maven instead of S3 2. Fix Maven warnings (#4859) * Optimizing profile file loading * Elastic Load Balancing Update: This release enables unhealthy target draining intervals for Network Load Balancers. * AWS CloudFormation Update: CloudFormation IaC generator allows you to scan existing resources in your account and select resources to generate a template for a new or existing CloudFormation stack. * AWS Glue Update: Update page size limits for GetJobRuns and GetTriggers APIs. * Amazon Simple Systems Manager (SSM) Update: This release adds an optional Duration parameter to StateManager Associations. This allows customers to specify how long an apply-only-on-cron association execution should run. Once the specified Duration is out all the ongoing cancellable commands or automations are cancelled. * Updated endpoints.json and partitions.json. * Release 2.23.15. Updated CHANGELOG.md, README.md and all pom.xml. * Update to next snapshot version: 2.23.16-SNAPSHOT * Fix check for release tag (#4871) The original check checks whether the tag *exists* in the local and then tries to remove it from the public repo. However, it's possible that the tag is in the private repo, where it might not exists (refs from both remotes are present locally). This commit fixes this by explicitly check if the tag exists in the public repo and only then will it try to delete it. * Exclude endpoint provider tests to speed up the build (#4860) * Exclude endpoint provider tests to speed up the build * Update CI and add buildspec for endpoints test * Fix cloudformation template * Update japicmp config (#4872) * caching resolved values * adding back commented test * Changed to use lazy value for credential provider * Activating SRA auth for 25 services with control plane type APIs (#4874) * Amazon Cognito Identity Provider Update: Added CreateIdentityProvider and UpdateIdentityProvider details for new SAML IdP features * Amazon Neptune Graph Update: Adding new APIs in SDK for Amazon Neptune Analytics. These APIs include operations to execute, cancel, list queries and get the graph summary. * AWS Elemental MediaConvert Update: This release includes support for broadcast-mixed audio description tracks. * Amazon Interactive Video Service Update: This release introduces a new resource Playback Restriction Policy which can be used to geo-restrict or domain-restrict channel stream playback when associated with a channel. New APIs to support this resource were introduced in the form of Create/Delete/Get/Update/List. * Amazon Managed Blockchain Query Update: This release adds support for transactions that have not reached finality. It also removes support for the status property from the response of the GetTransaction operation. You can use the confirmationStatus and executionStatus properties to determine the status of the transaction. * Updated endpoints.json and partitions.json. * Release 2.23.16. Updated CHANGELOG.md, README.md and all pom.xml. * Add option to disable EC2 metadata (IMDS) calls without token (#4866) * Add option to disable IMDS v1 fallback when token is not returned (#4840) * Add support for disable IMDS v1 fallback for regions * Cache whether CRT is available in the flexible checksum algorithm. (#4878) * Update to next snapshot version: 2.23.17-SNAPSHOT * Bump maven wrapper plugin version (#4879) * Performance improvement for sigv4 signing. (#4867) 1. When trimming and removing consecutive spaces during sigv4 normalization, copy word-by-word instead of character-by-character. This reduces the overhead of range and encoding checks in string builder. 2. Increase starting string builder size for canonical headers, to limit resizing (2048 worked well for DynamoDB's get-item). 3. Use a switch statement for whitespace checks instead of consecutive if statements. On my compiler, the switch statement compiles to a jump table which runs quicker. * Expose futureCompletionExecutor on S3 CRT client (#4880) * Expose futureCompletionExecutor on S3 CRT client * Adderss feedback and fix build * Amazon SageMaker Service Update: Amazon SageMaker Canvas adds GenerativeAiSettings support for CanvasAppSettings. * Amazon DynamoDB Update: Any number of users can execute up to 50 concurrent restores (any type of restore) in a given account. * Updated endpoints.json and partitions.json. * Release 2.23.17. Updated CHANGELOG.md, README.md and all pom.xml. * Enable compiled endpoint rules for the second wave (#4883) Services included are: backupstorage, codecatalyst, cognitoidentity, kinesis, mediastoredata, transcribe, transcribestreaming * Revert "Performance improvement for sigv4 signing. (#4867)" This reverts commit 0ab7f75. * Update to next snapshot version: 2.23.18-SNAPSHOT * AWS Glue Update: Introduce Catalog Encryption Role within Glue Data Catalog Settings. Introduce SASL/PLAIN as an authentication method for Glue Kafka connections * Amazon WorkSpaces Update: Added definitions of various WorkSpace states * Release 2.23.18. Updated CHANGELOG.md, README.md and all pom.xml. * update aws-sdk-java pom to add imds and dyanmodb-enhanced (#4890) * Performance improvement for sigv4 signing. (#4891) This is a reintroduction of the reverted commit #4867. This includes a fix to the issue that caused the revert: improper handling of empty header values. * Update to next snapshot version: 2.23.19-SNAPSHOT * Delete CloudSearchv2IntegrationTest (#4888) * Fix tag deletion command (#4892) * including S3 Access Grants Plugin as part of Java SDK Bundle (#4881) * including S3 Access Grants Plugin as part of Java SDK Bundle --------- Co-authored-by: Shiva Kumar Mukkapati <mshvkmr@amazon.com> * Archive old changelog entries (< 2.23.0) (#4873) - .json files files for each version are grouped under minor versions in .changes - Markdown files for each minor version are created in changelogs/ directory - Changelog scripts updated to add a link to older version in the generated changelog * Changing indentation of config files to 4 spaces (#4889) * Amazon EC2 Container Service Update: This release is a documentation only update to address customer issues. * AWS WAFV2 Update: You can now delete an API key that you've created for use with your CAPTCHA JavaScript integration API. * Amazon OpenSearch Service Update: This release adds clear visibility to the customers on the changes that they make on the domain. * AWS AppSync Update: Support for environment variables in AppSync GraphQL APIs * Amazon CloudWatch Logs Update: This release adds a new field, logGroupArn, to the response of the logs:DescribeLogGroups action. * Amazon Elasticsearch Service Update: This release adds clear visibility to the customers on the changes that they make on the domain. * Release 2.23.19. Updated CHANGELOG.md, README.md and all pom.xml. * Update to next snapshot version: 2.23.20-SNAPSHOT * Bump CRT version and expose setting memory limits for S3 calls (#4885) * Bump aws-crt version to 0.29.9 * Exposes a setting to set the memory limit when making asynchronous calls with the CRT-based S3 client * Activating SRA for this service (#4896) * Fix request cancellation logic in the AWS CRT Sync HTTP client (#4887) * Fix request cancellation logic in the AWS CRT Sync HTTP client * Address feedback * AWS DataSync Update: AWS DataSync now supports manifests for specifying files or objects to transfer. * Amazon Redshift Update: LisRecommendations API to fetch Amazon Redshift Advisor recommendations. * Amazon Lex Model Building V2 Update: This release introduces a new bot replication feature as part of Lex Global Resiliency offering. This feature leverages a new set of APIs that allow customers to create bot replicas and replicate changes to bots across regions. * Updated endpoints.json and partitions.json. * Release 2.23.20. Updated CHANGELOG.md, README.md and all pom.xml. * Update to next snapshot version: 2.23.21-SNAPSHOT * Amazon QuickSight Update: General Interactions for Visuals; Waterfall Chart Color Configuration; Documentation Update * AWS CodePipeline Update: Add ability to execute pipelines with new parallel & queued execution modes and add support for triggers with filtering on branches and file paths. * Amazon WorkSpaces Update: This release introduces User-Decoupling feature. This feature allows Workspaces Core customers to provision workspaces without providing users. CreateWorkspaces and DescribeWorkspaces APIs will now take a new optional parameter "WorkspaceName". * Release 2.23.21. Updated CHANGELOG.md, README.md and all pom.xml. * Update to next snapshot version: 2.23.22-SNAPSHOT * S3 should configure signer properties (#4856) * Configure modeled signer properties for endpoint based auth scheme resolver * Refactor the logic to use Knowledge Indexes * Enable CRC32 for PUT for MultipartS3AsyncClient (#4898) * Fix bug to allow MpuS3Client to PUT COPY with SSE-C and Checksum * Enable CRC32 for Multipart PUT COPY * Address comments * add changelog and update checksum check * Address comments * update javadocs * Add unit tests * Amazon Prometheus Service Update: Overall documentation updates. * Braket Update: Creating a job will result in DeviceOfflineException when using an offline device, and DeviceRetiredException when using a retired device. * AWS IoT Update: This release allows AWS IoT Core users to enable Online Certificate Status Protocol (OCSP) Stapling for TLS X.509 Server Certificates when creating and updating AWS IoT Domain Configurations with Custom Domain. * AWS Batch Update: This feature allows Batch to support configuration of repository credentials for jobs running on ECS * Cost Optimization Hub Update: Adding includeMemberAccounts field to the response of ListEnrollmentStatuses API. * Amazon EC2 Container Service Update: Documentation only update for Amazon ECS. * AWS Price List Service Update: Add Throttling Exception to all APIs. * Updated endpoints.json and partitions.json. * Release 2.23.22. Updated CHANGELOG.md, README.md and all pom.xml. * Bumping minor version to 2.24.0 due to core refactoring and moving away from S3 signing attributes * Update to next snapshot version: 2.24.1-SNAPSHOT * Fix changelogs for 2.24.0 (#4913) Changes were incorrectly marked as 2.23.22. Arhived changes up to and including 2.23.21, and changes 2.23.22 changes to 2.24.0. * Correct the javadoc for the main class (#4915) * Remove uses of legacy singer attributes (#4914) * AWS AppSync Update: Adds support for new options on GraphqlAPIs, Resolvers and Data Sources for emitting Amazon CloudWatch metrics for enhanced monitoring of AppSync APIs. * Amazon CloudWatch Update: This release enables PutMetricData API request payload compression by default. * Amazon Neptune Graph Update: Adding a new option "parameters" for data plane api ExecuteQuery to support running parameterized query via SDK. * Amazon Route 53 Domains Update: This release adds bill contact support for RegisterDomain, TransferDomain, UpdateDomainContact and GetDomainDetail API. * Release 2.24.1. Updated CHANGELOG.md, README.md and all pom.xml. * Update to next snapshot version: 2.24.2-SNAPSHOT * Handle header lists in URLConnection client (#4897) * Add additional helper methods to DynamoDB Expression (#4765) These helpers make it easier to write complex queries without having to use the existing overly verbose static methods. This lets users write more clear and concise code that makes is obvious how the expressions are logically combined together. #4764 Co-authored-by: Andy Kiesler <kiesler@amazon.com> Co-authored-by: Dongie Agnir <261310+dagnir@users.noreply.github.com> * Trim object metadata keys for PUT and MPU (#4902) This brings the V2's behavior in line with 1.x * Backfilling changelog to include signing attribute changes (#4919) * Backfilling changelog to include signing attribute changes * Moved changes to release .json file and regenerated changelog * AWS Resource Explorer Update: Resource Explorer now uses newly supported IPv4 'amazonaws.com' endpoints by default. * AWS Marketplace Catalog Service Update: AWS Marketplace Catalog API now supports setting intent on requests * Amazon Security Lake Update: Documentation updates for Security Lake * Amazon Lightsail Update: This release adds support to upgrade the major version of a database. * Updated endpoints.json and partitions.json. * Release 2.24.2. Updated CHANGELOG.md, README.md and all pom.xml. * Update CloudFront test resource name (#4925) Make it clear that the resource is not to be deleted. * Update to next snapshot version: 2.24.3-SNAPSHOT * Fixed an issue where NPE would be thrown for empty event (#4922) * Fix for Issue #4912 where Cross Region S3 Access using AWS_GLOBAL Endpoint failed. (#4921) * Activating SRA auth for approximately half of remaining services APIs (#4926) * Enable compiled endpoint rules for the third wave (#4927) Services included are: - accessanalyzer - account - acm - acmpca - alexaforbusiness - amp - amplify - amplifybackend - amplifyuibuilder - apigatewaymanagementapi - appconfig - appfabric - appflow - appintegrations - applicationautoscaling - applicationcostprofiler - applicationdiscovery - applicationinsights - cloudfrontkeyvaluestore - cognitoidentityprovider - eks - emrcontainers - iam - iot - marketplacemetering - paymentcryptography - securityhub * QBusiness Update: This release adds the metadata-boosting feature, which allows customers to easily fine-tune the underlying ranking of retrieved RAG passages in order to optimize Q&A answer relevance. It also adds new feedback reasons for the PutFeedback API. * Amazon Lookout for Equipment Update: This feature allows customers to see pointwise model diagnostics results for their models. * AWS Control Tower Update: Adds support for new Baseline and EnabledBaseline APIs for automating multi-account governance. * Updated endpoints.json and partitions.json. * Release 2.24.3. Updated CHANGELOG.md, README.md and all pom.xml. * Update to next snapshot version: 2.24.4-SNAPSHOT * Amazon Polly Update: Amazon Polly adds 1 new voice - Burcu (tr-TR) * Amazon OpenSearch Service Update: Adds additional supported instance types. * AWS Secrets Manager Update: Doc only update for Secrets Manager * Amazon GuardDuty Update: Marked fields IpAddressV4, PrivateIpAddress, Email as Sensitive. * AWS Artifact Update: This is the initial SDK release for AWS Artifact. AWS Artifact provides on-demand access to compliance and third-party compliance reports. This release includes access to List and Get reports, along with their metadata. This release also includes access to AWS Artifact notifications settings. * Amazon HealthLake Update: This release adds a new response parameter, JobProgressReport, to the DescribeFHIRImportJob and ListFHIRImportJobs API operation. JobProgressReport provides details on the progress of the import job on the server. * AWS CodePipeline Update: Add ability to override timeout on action level. * Amazon Detective Update: Doc only updates for content enhancement * Amazon SageMaker Service Update: This release adds a new API UpdateClusterSoftware for SageMaker HyperPod. This API allows users to patch HyperPod clusters with latest platform softwares. * Updated endpoints.json and partitions.json. * Release 2.24.4. Updated CHANGELOG.md, README.md and all pom.xml. * Update to next snapshot version: 2.24.5-SNAPSHOT * update all contributors hall (#4936) * Update codebuild-ci.yml to trigger CI on doc changes (#4928) * Amazon Connect Participant Service Update: Doc only update to GetTranscript API reference guide to inform users about presence of events in the chat transcript. * Amazon Kinesis Firehose Update: This release adds support for Data Message Extraction for decompressed CloudWatch logs, and to use a custom file extension or time zone for S3 destinations. * Amazon Simple Notification Service Update: This release marks phone numbers as sensitive inputs. * Amazon EMR Update: adds fine grained control over Unhealthy Node Replacement to Amazon ElasticMapReduce * AWS Lambda Update: Documentation-only updates for Lambda to clarify a number of existing actions and properties. * Amazon Relational Database Service Update: Doc only update for a valid option in DB parameter group * Updated endpoints.json and partitions.json. * Release 2.24.5. Updated CHANGELOG.md, README.md and all pom.xml. * Add decision log for migration tool (#4853) * Update to next snapshot version: 2.24.6-SNAPSHOT * Fixed the issue where invoking abort on AsyncResponseTransformer.toBl… (#4932) * Fixed the issue where invoking abort on AsyncResponseTransformer.toBlockingInputStream could lead to memory leak * Fix build * Remove doAfterAbort and fix build * Make test less flaky * Pause/Resume Upload for Transfer Manager with Java S3Client (#4908) (#4937) * Pause/Resume Upload for Transfer Manager with Java S3Client * Fix checkstyle and equalsVerifier * temporarily disable failing test * Address comments * ifPresent check for PauseObservable * Address comments * Wrap subscriber with PausibleUpload * add changelog * integ test upload resume with different TMs * Rename to PausableUpload * Add unit tests * Refactor KnownLengthUploadHelper * Add unit tests * Move PauseObservable and S3ResumeToken out of internal * Refactor UploadWithKnownContentLengthHelper * Address comments and update tests * Fix import order * Extract Subscriber and MpuRequestContext to separate classes * Update Subscriber and add tests * Create separate tmJava with multipartClient in integ test * Address comments * Remove TestInternalApis and refacor tests * Address comment * AWS Config Update: Documentation updates for the AWS Config CLI * Amazon Interactive Video Service Update: Changed description for latencyMode in Create/UpdateChannel and Channel/ChannelSummary. * Amazon Keyspaces Update: Documentation updates for Amazon Keyspaces * AWS Amplify Update: This release contains API changes that enable users to configure their Amplify domains with their own custom SSL/TLS certificate. * AWS MediaTailor Update: MediaTailor: marking #AdBreak.OffsetMillis as required. * chatbot Update: This release adds support for AWS Chatbot. You can now monitor, operate, and troubleshoot your AWS resources with interactive ChatOps using the AWS SDK. * Release 2.24.6. Updated CHANGELOG.md, README.md and all pom.xml. * Update to next snapshot version: 2.24.7-SNAPSHOT * Fix codebuild-ci.yml (#4946) * AWS Lambda Update: Add .NET 8 (dotnet8) Runtime support to AWS Lambda. * Amazon Kinesis Firehose Update: This release updates a few Firehose related APIs. * Amazon DynamoDB Update: Publishing quick fix for doc only update. * Release 2.24.7. Updated CHANGELOG.md, README.md and all pom.xml. * Update to next snapshot version: 2.24.8-SNAPSHOT * Fix flaky tests (#4947) * Also delete local release tag (#4943) If the tag exists in public, then it exists locally too since we fetched all of public when adding it as a remote. Also delete this local tag so it doesn't fail when we recreate it. * Update documentation for FUTURE_COMPLETION_EXECUTOR (#4950) * Add content header if the Request doesnot has one for CRT sync Http clients (#4920) * Add content header if the Request doesnot has one for CRT sync Http clients * Content length updated in the Marshallers * Content length updation done in the Marshallers after internal comment * Handled Matts comments on PR * Updated MarshallersAddContentLengthTest to Junit5 * Updated to initiate theCheckbuilds * Handle sonar raised issues * Amazon Simple Systems Manager (SSM) Update: This release adds support for sharing Systems Manager parameters with other AWS accounts. * AWS Elemental MediaLive Update: MediaLive now supports the ability to restart pipelines in a running channel. * AWS IoT Events Update: Increase the maximum length of descriptions for Inputs, Detector Models, and Alarm Models * Amazon Lookout for Equipment Update: This release adds a field exposing model quality to read APIs for models. It also adds a model quality field to the API response when creating an inference scheduler. * Release 2.24.8. Updated CHANGELOG.md, README.md and all pom.xml. * Update to next snapshot version: 2.24.9-SNAPSHOT * Fix request cancellation issue in the AWS CRT-based S3 client that co… (#4955) * Fix request cancellation issue in the AWS CRT-based S3 client that could lead to memory leak * Fix build and address feedback * Fix test * Fix build * Amazon CloudWatch Internet Monitor Update: This release adds IPv4 prefixes to health events * Amazon Kinesis Video Streams Update: Increasing NextToken parameter length restriction for List APIs from 512 to 1024. * Release 2.24.9. Updated CHANGELOG.md, README.md and all pom.xml. * Update to next snapshot version: 2.24.10-SNAPSHOT * Choose next potential scheme if signer not found (#4952) * Choose next potential scheme if signer not found This commit fixes the generated auth scheme selection mechanism by ensuring that auth scheme is able to provide a signer implementation (i.e. does not throw an exception) before selecting the auth scheme. This is useful in situations where the signer implementation relies on components thay may not be available at runtime, such as the AwsV4aAuthScheme, which relies on CRT which is an optional dependency. Additional changes: - introduce crt-unavailable-tests which is a good candidate for testing behavior of components that use CRT when CRT is not present. - modify DefaultAwsV4aAuthScheme so that errors thrown during static initialization of the singleton holder for the Sigv4a signer are caught and thrown directly instead of resulting in ExceptionInInitializerError * Review comments * Fix module version (#4961) * Revert "Handle header lists in URLConnection client (#4897)" (#4960) This reverts commit b51a51c. This change may be causing issues with propagation of trace IDs. * Mulitpart download workaround (#4945) * Make Transfer Manager work by default with MultipartS3AsyncClient. * Amazon QLDB Update: Clarify possible values for KmsKeyArn and EncryptionDescription. * AWS AppSync Update: Documentation only updates for AppSync * CloudWatch RUM Update: Doc-only update for new RUM metrics that were added * Amazon Relational Database Service Update: Add pattern and length based validations for DBShardGroupIdentifier * Release 2.24.10. Updated CHANGELOG.md, README.md and all pom.xml. * Update to next snapshot version: 2.24.11-SNAPSHOT * Remove the default use of the legacy signer (#4956) * Remove the default use of the legacy signer * Address PR comments * Fix a small typo * Elastic Disaster Recovery Service Update: Added volume status to DescribeSourceServer replicated volumes. * Amazon API Gateway Update: Documentation updates for Amazon API Gateway. * Amazon Relational Database Service Update: This release adds support for gp3 data volumes for Multi-AZ DB Clusters. * Managed Streaming for Kafka Connect Update: Adds support for tagging, with new TagResource, UntagResource and ListTagsForResource APIs to manage tags and updates to existing APIs to allow tag on create. This release also adds support for the new DeleteWorkerConfiguration API. * Release 2.24.11. Updated CHANGELOG.md, README.md and all pom.xml. * Update to next snapshot version: 2.24.12-SNAPSHOT * upgrade to netty 4.1.107.Final (#4966) * Remove the use of the legacy signer (#4941) * Remove the use of the legacy signer * Remove extra whitespace * Small commit to kick the build * Remove the profiles dependency that is not needed anymore * Enable transfer listener for Java-based TransferManager multipart upload (#4951) * Enable transfer listener for Java-based TransferManager multipart upload * Add Java Progress Listener and refactor * Refactoring * Fix merge * Address comments * Address comments * Add changelog entry * Make NoOpPublisherListener static * Remove static modifier * Support Non proxy host settings in the ProxyConfiguration for Crt http client. (#4962) * Support Non proxy host settings in the ProxyConfiguration for Crt http client. * Handled comments * AWS Amplify UI Builder Update: We have added the ability to tag resources after they are created * Release 2.24.12. Updated CHANGELOG.md, README.md and all pom.xml. * Revert "Support Non proxy host settings in the ProxyConfiguration for Crt http client. (#4962)" This reverts commit 7e9c01d. * Enable compiled endpoint rules for the fourth wave (#4973) * Update to next snapshot version: 2.24.13-SNAPSHOT * Fixed an issue in S3 multipart client where `BlockingInputStreamAsyncRequestBody#writeInputStream` could get stuck if any of the multipart request fails. (#4970) * Activating SRA auth for remaining service APIs (#4967) * Re-enable japicmp for S3 (#4969) Re-enable japicmp for S3 * fix codegen test classes --------- Co-authored-by: AWS <> Co-authored-by: aws-sdk-java-automation <43143862+aws-sdk-java-automation@users.noreply.github.com> Co-authored-by: Tom Keller <1083460+kellertk@users.noreply.github.com> Co-authored-by: Matthew Miller <millem@amazon.com> Co-authored-by: Zoe Wang <33073555+zoewangg@users.noreply.github.com> Co-authored-by: Dongie Agnir <261310+dagnir@users.noreply.github.com> Co-authored-by: Manuel Sugawara <sugmanue@amazon.com> Co-authored-by: Dongie Agnir <dongie@amazon.com> Co-authored-by: John Viegas <70235430+joviegas@users.noreply.github.com> Co-authored-by: shiva kumar <62441208+shiva958@users.noreply.github.com> Co-authored-by: Shiva Kumar Mukkapati <mshvkmr@amazon.com> Co-authored-by: David Ho <70000000+davidh44@users.noreply.github.com> Co-authored-by: Andy Kiesler <4186292+akiesler@users.noreply.github.com> Co-authored-by: Andy Kiesler <kiesler@amazon.com> Co-authored-by: Debora N. Ito <476307+debora-ito@users.noreply.github.com> Co-authored-by: Olivier L Applin <olapplin@amazon.com> Co-authored-by: sullis <github@seansullivan.com> Co-authored-by: John Viegas <joviegas@amazon.com>
…uld lead to memory leak
Motivation and Context
Currently, if a request is cancelled in AWS CRT-based S3 client, CRT resource will not be cancelled and cleaned up properly, leaking to memory leak.
The root cause is that we are calling
S3MetaRequest.close()
beforeS3MetaRequest.cancel()
.Modifications
S3MetaRequestWrapper
class to make sure we don't invoke S3MetaRequest if it's closed. All interactions withS3MetaRequest
must be throughS3MetaRequestWrapper
going forward.S3MetaRequestWrapper
is used instead ofS3MetaRequest
.Testing
Tested it locally and added new functional tests.
Screenshots (if appropriate)
Types of changes
Checklist
mvn install
succeedsscripts/new-change
script and following the instructions. Commit the new file created by the script in.changes/next-release
with your changes.License