Add UploadWithResponseAsync api #4105

GarrettBeatty · 2025-11-06T15:18:40Z

Stacked PRs:

Description

This change creates a new api UploadWithResponseAsync. It is the same as the existing UploadAsync api but it returns the TransferUtilityUploadResponse object.

How

In order to re-use as much as the existing code as possible, I made BaseCommand take in a generic which is the return type of the ExecuteAsync function. All commands UploadCommand, DownloadCommand, etc will pass in their expected return type TransferUtilityUploadResponse, TransferUTilityDownloadResponse, etc.

In order to make this change in one go, I had to create place holder classes for TransferUtilityDownloadDirectoryResponse and other responses. These will eventually be populated in future PRs

Motivation and Context

To adhere to the SEP

Testing

1.dry run - 36f635c4-0fbb-4d2c-9b9c-b977c7906aef in progress
2. Integration tests

Types of changes

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to change)

Checklist

My code follows the code style of this project
My change requires a change to the documentation
I have updated the documentation accordingly
I have read the README document
I have added tests to cover my changes
All new and existing tests passed

License

I confirm that this pull request can be released under the Apache 2 license

Copilot

Pull Request Overview

This PR introduces new UploadWithResponse APIs for the S3 Transfer Utility that return response metadata information. The key difference from existing Upload methods is that these new methods return a TransferUtilityUploadResponse object containing upload metadata like ETag, version ID, checksums, and encryption information.

Key changes:

Added async UploadWithResponseAsync methods with overloads for file path and stream inputs
Added synchronous UploadWithResponse methods that wrap async calls
Refactored internal BaseCommand to be generic BaseCommand<TResponse> to support typed responses
Added comprehensive integration tests covering small files, large files, streams, checksums, and comparison with legacy methods
Introduced placeholder response classes for other Transfer Utility operations (download directory, upload directory, abort multipart uploads)

Reviewed Changes

Copilot reviewed 24 out of 24 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
sdk/test/Services/S3/IntegrationTests/TransferUtilityTests.cs	Added 5 comprehensive integration tests for new UploadWithResponse API variants
sdk/src/Services/S3/Custom/Transfer/_bcl+netstandard/TransferUtility.sync.cs	Added 4 synchronous UploadWithResponse method overloads
sdk/src/Services/S3/Custom/Transfer/_async/TransferUtility.async.cs	Added 4 async UploadWithResponseAsync method overloads
sdk/src/Services/S3/Custom/Transfer/TransferUtility.cs	Updated GetUploadCommand signature to return generic BaseCommand
sdk/src/Services/S3/Custom/Transfer/Internal/_async/BaseCommand.async.cs	Made BaseCommand generic to support typed responses
sdk/src/Services/S3/Custom/Transfer/Internal/BaseCommand.cs	Made BaseCommand generic and removed obsolete Return property
sdk/src/Services/S3/Custom/Transfer/Internal/_async/SimpleUploadCommand.async.cs	Updated to return TransferUtilityUploadResponse
sdk/src/Services/S3/Custom/Transfer/Internal/_async/MultipartUploadCommand.async.cs	Updated to return TransferUtilityUploadResponse from both code paths
sdk/src/Services/S3/Custom/Transfer/Internal/_async/OpenStreamCommand.async.cs	Updated to return TransferUtilityOpenStreamResponse
sdk/src/Services/S3/Custom/Transfer/Internal/_async/DownloadCommand.async.cs	Updated to return TransferUtilityDownloadResponse
sdk/src/Services/S3/Custom/Transfer/Internal/_bcl+netstandard/UploadDirectoryCommand.cs	Updated to return TransferUtilityUploadDirectoryResponse
sdk/src/Services/S3/Custom/Transfer/Internal/_bcl+netstandard/DownloadDirectoryCommand.cs	Updated to return TransferUtilityDownloadDirectoryResponse
sdk/src/Services/S3/Custom/Transfer/Internal/_async/AbortMultipartUploadsCommand.async.cs	Updated to return TransferUtilityAbortMultipartUploadsResponse
sdk/src/Services/S3/Custom/Transfer/TransferUtilityUploadDirectoryResponse.cs	Added placeholder response class for upload directory operations
sdk/src/Services/S3/Custom/Transfer/TransferUtilityDownloadDirectoryResponse.cs	Added placeholder response class for download directory operations
sdk/src/Services/S3/Custom/Transfer/TransferUtilityAbortMultipartUploadsResponse.cs	Added placeholder response class for abort multipart uploads operations
generator/.DevConfigs/77d980ad-8f58-4f2e-97f8-d2c8c5ba3732.json	Added dev config marking this as a minor version change

sdk/src/Services/S3/Custom/Transfer/Internal/BaseCommand.cs

Copilot · 2025-11-06T15:29:55Z

sdk/src/Services/S3/Custom/Transfer/Internal/_async/OpenStreamCommand.async.cs

+            // TODO map and return response
+            return new TransferUtilityOpenStreamResponse();


TODO comment indicates incomplete implementation. The response from GetObjectAsync should be properly mapped to the TransferUtilityOpenStreamResponse to ensure consumers receive complete metadata. Complete the response mapping or document why it's deferred.

will be done in future pr

Copilot · 2025-11-06T15:29:55Z

sdk/src/Services/S3/Custom/Transfer/Internal/_async/DownloadCommand.async.cs

+            // TODO map and return response
+            return new TransferUtilityDownloadResponse();


TODO comment indicates incomplete implementation. The response from the download operation should be properly mapped to the TransferUtilityDownloadResponse to provide consumers with metadata about the downloaded object. Complete the response mapping or document why it's deferred.

will be done in future pr

Copilot

Pull Request Overview

Copilot reviewed 24 out of 24 changed files in this pull request and generated 2 comments.

Copilot · 2025-11-06T15:50:41Z

sdk/src/Services/S3/Custom/Transfer/Internal/_async/OpenStreamCommand.async.cs

+            // TODO map and return response
+            return new TransferUtilityOpenStreamResponse();


This TODO comment indicates incomplete implementation. The OpenStreamCommand should properly populate the TransferUtilityOpenStreamResponse with data from the GetObjectResponse, including setting the ResponseStream property and any other relevant metadata. Consider completing this implementation or creating a tracking issue.

Suggested change

// TODO map and return response

return new TransferUtilityOpenStreamResponse();

// Map and return response

return new TransferUtilityOpenStreamResponse

{

ResponseStream = response.ResponseStream,

// Optionally map additional properties if available

// ContentType = response.Headers.ContentType,

// Metadata = response.Metadata,

// Headers = response.Headers

};

Copilot · 2025-11-06T15:50:42Z

sdk/src/Services/S3/Custom/Transfer/Internal/_async/DownloadCommand.async.cs

+            // TODO map and return response
+            return new TransferUtilityDownloadResponse();


This TODO comment indicates incomplete implementation. The DownloadCommand should properly populate the TransferUtilityDownloadResponse with metadata from the download operation. Consider completing this implementation or creating a tracking issue.

Suggested change

// TODO map and return response

return new TransferUtilityDownloadResponse();

// Map and return response

return new TransferUtilityDownloadResponse

{

ETag = response.ETag,

LastModified = response.LastModified,

ContentType = response.Headers.ContentType,

HttpStatusCode = response.HttpStatusCode,

// Add other relevant properties as needed

};

peterrsongg

approving, since the only change is removing Obsolete (from previous PR)

peterrsongg · 2025-11-06T19:41:00Z

sdk/test/Services/S3/IntegrationTests/TransferUtilityTests.cs

+
+                Console.WriteLine($"UploadWithResponseAsync ETag: {response.ETag}");
+                Console.WriteLine($"Legacy upload ETag: {legacyMetadata.ETag}");
+                Console.WriteLine($"File size: {fileSize}, Response metadata size: {responseMetadata.ContentLength}");


non-blocking: do we also want to validate content of the file is the same and stuff or do you not think that is necessary?

stack-info: PR: #4105, branch: GarrettBeatty/stacked/13

GarrettBeatty force-pushed the GarrettBeatty/stacked/13 branch from a0a62cb to a589ea7 Compare November 6, 2025 15:18

This was referenced Nov 6, 2025

Remove AmazonWebServiceResponse as base class for transfer utility repsonse objects. #4087

Merged

Add GetObjectResponse to TransferUtilityOpenStreamResponse mapping. #4076

Merged

Add DownloadInitiated, Failed and Completed events #4079

Open

GarrettBeatty requested review from Copilot, peterrsongg and philasmar November 6, 2025 15:20

Copilot AI reviewed Nov 6, 2025

View reviewed changes

GarrettBeatty changed the base branch from GarrettBeatty/stacked/9 to feature/transfermanager November 6, 2025 15:32

GarrettBeatty force-pushed the GarrettBeatty/stacked/13 branch from a589ea7 to c46c1f2 Compare November 6, 2025 15:32

GarrettBeatty changed the base branch from feature/transfermanager to GarrettBeatty/stacked/9 November 6, 2025 15:33

GarrettBeatty changed the base branch from GarrettBeatty/stacked/9 to feature/transfermanager November 6, 2025 15:34

GarrettBeatty force-pushed the GarrettBeatty/stacked/13 branch 2 times, most recently from dc392ca to a49039c Compare November 6, 2025 15:38

GarrettBeatty requested a review from Copilot November 6, 2025 15:40

Copilot AI reviewed Nov 6, 2025

View reviewed changes

GarrettBeatty force-pushed the GarrettBeatty/stacked/13 branch from a49039c to e6d37cf Compare November 6, 2025 19:15

GarrettBeatty marked this pull request as ready for review November 6, 2025 19:22

peterrsongg approved these changes Nov 6, 2025

View reviewed changes

philasmar approved these changes Nov 6, 2025

View reviewed changes

Add UploadWithResponseAsync api

5f1fd9a

stack-info: PR: #4105, branch: GarrettBeatty/stacked/13

GarrettBeatty force-pushed the GarrettBeatty/stacked/13 branch from e6d37cf to 5f1fd9a Compare November 6, 2025 21:10

GarrettBeatty mentioned this pull request Nov 6, 2025

Populate TransferUtilityDownloadDirectoryResponse with total objects downloaded #4109

Draft

10 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add UploadWithResponseAsync api #4105

Add UploadWithResponseAsync api #4105

Uh oh!

GarrettBeatty commented Nov 6, 2025 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Copilot AI Nov 6, 2025

Uh oh!

GarrettBeatty Nov 6, 2025

Uh oh!

Copilot AI Nov 6, 2025

Uh oh!

GarrettBeatty Nov 6, 2025

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Nov 6, 2025

Uh oh!

Copilot AI Nov 6, 2025

Uh oh!

peterrsongg left a comment

Uh oh!

peterrsongg Nov 6, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

		// TODO map and return response
		return new TransferUtilityOpenStreamResponse();

		// TODO map and return response
		return new TransferUtilityDownloadResponse();

-            // TODO map and return response
-            return new TransferUtilityOpenStreamResponse();
+            // Map and return response
+            return new TransferUtilityOpenStreamResponse
+            {
+                ResponseStream = response.ResponseStream,
+                // Optionally map additional properties if available
+                // ContentType = response.Headers.ContentType,
+                // Metadata = response.Metadata,
+                // Headers = response.Headers
+            };

Add UploadWithResponseAsync api #4105

Are you sure you want to change the base?

Add UploadWithResponseAsync api #4105

Uh oh!

Conversation

GarrettBeatty commented Nov 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

How

Motivation and Context

Testing

Types of changes

Checklist

License

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Copilot AI Nov 6, 2025

Choose a reason for hiding this comment

Uh oh!

GarrettBeatty Nov 6, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 6, 2025

Choose a reason for hiding this comment

Uh oh!

GarrettBeatty Nov 6, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Uh oh!

Copilot AI Nov 6, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 6, 2025

Choose a reason for hiding this comment

Uh oh!

peterrsongg left a comment

Choose a reason for hiding this comment

Uh oh!

peterrsongg Nov 6, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

GarrettBeatty commented Nov 6, 2025 •

edited

Loading