Skip to content

Commit

Permalink
Address feedback
Browse files Browse the repository at this point in the history
  • Loading branch information
zoewangg committed Dec 7, 2022
1 parent 77203cc commit bd14714
Show file tree
Hide file tree
Showing 3 changed files with 26 additions and 19 deletions.
19 changes: 11 additions & 8 deletions services-custom/s3-transfer-manager/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ S3TransferManager transferManager =

### Transfer a single object

#### Upload a file to S3 and log the upload’s process with a TransferListener
#### Upload a file to S3 and log the upload’s progress with a TransferListener
To upload a file to Amazon S3, you need to provide the source file path and a PutObjectRequest specifying the target bucket and key.

```java
Expand All @@ -60,6 +60,7 @@ S3TransferManager transferManager = S3TransferManager.create();
UploadFileRequest uploadFileRequest =
UploadFileRequest.builder()
.putObjectRequest(req -> req.bucket("bucket").key("key"))
// attaching a LoggingTransferListener that will log the progress
.addTransferListener(LoggingTransferListener.create())
.source(Paths.get("myFile.txt"))
.build();
Expand All @@ -79,10 +80,11 @@ S3TransferManager transferManager = S3TransferManager.create();

DownloadFileRequest downloadFileRequest =
DownloadFileRequest.builder()
.getObjectRequest(req -> req.bucket("bucket").key("key"))
.destination(Paths.get("myFile.txt"))
.addTransferListener(LoggingTransferListener.create())
.build();
.getObjectRequest(req -> req.bucket("bucket").key("key"))
.destination(Paths.get("myFile.txt"))
// attaching a LoggingTransferListener that will log the progress
.addTransferListener(LoggingTransferListener.create())
.build();

FileDownload download = transferManager.downloadFile(downloadFileRequest);

Expand Down Expand Up @@ -127,7 +129,7 @@ DirectoryUpload directoryUpload = transferManager.uploadDirectory(UploadDirector
// Wait for the transfer to complete
CompletedDirectoryUpload completedDirectoryUpload = directoryUpload.completionFuture().join();

// Print out the failed uploads
// Print out any failed uploads
completedDirectoryUpload.failedTransfers().forEach(System.out::println);
```

Expand All @@ -142,11 +144,12 @@ S3TransferManager transferManager = S3TransferManager.create();
DownloadDirectoryRequest.builder()
.destination(Paths.get("destination/directory"))
.bucket("bucket")
.listObjectsV2RequestTransformer(l -> l.prefix("prefix"))
// only download objects with prefix "photos"
.listObjectsV2RequestTransformer(l -> l.prefix("photos"))
.build());
// Wait for the transfer to complete
CompletedDirectoryDownload completedDirectoryDownload = directoryDownload.completionFuture().join();

// Print out the failed downloads
// Print out any failed downloads
completedDirectoryDownload.failedTransfers().forEach(System.out::println);
```
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@
import software.amazon.awssdk.utils.Validate;

/**
* The S3 Transfer Manager offers a simple API to allow you to transfer a single object or a set of objects to and
* The S3 Transfer Manager offers a simple API that allows you to transfer a single object or a set of objects to and
* from Amazon S3 with enhanced throughput and reliability. It leverages Amazon S3 multipart upload and
* byte-range fetches to perform transfers in parallel. In addition, the S3 Transfer Manager also enables you to
* monitor a transfer's progress in real-time, as well as pause the transfer for execution at a later time.
Expand Down Expand Up @@ -122,7 +122,7 @@
* // Wait for the transfer to complete
* CompletedDirectoryUpload completedDirectoryUpload = directoryUpload.completionFuture().join();
*
* // Print out the failed uploads
* // Print out any failed uploads
* completedDirectoryUpload.failedTransfers().forEach(System.out::println);
* }
* <b>Download S3 objects to a local directory</b>
Expand All @@ -138,7 +138,7 @@
* // Wait for the transfer to complete
* CompletedDirectoryDownload completedDirectoryDownload = directoryDownload.completionFuture().join();
*
* // Print out the failed downloads
* // Print out any failed downloads
* completedDirectoryDownload.failedTransfers().forEach(System.out::println);
* }
* <b>Copy an S3 object to a different location in S3</b>
Expand Down Expand Up @@ -500,7 +500,7 @@ default Upload upload(Consumer<UploadRequest.Builder> request) {
* // Wait for the transfer to complete
* CompletedDirectoryUpload completedDirectoryUpload = directoryUpload.completionFuture().join();
*
* // Print out the failed uploads
* // Print out any failed uploads
* completedDirectoryUpload.failedTransfers().forEach(System.out::println);
* }
*
Expand All @@ -523,8 +523,11 @@ default DirectoryUpload uploadDirectory(Consumer<UploadDirectoryRequest.Builder>
}

/**
* Downloads all objects under a specific prefix and bucket to the provided directory. By default, all objects in the entire
* bucket will be downloaded.
* Downloads all objects under a bucket to the provided directory. By default, all objects in the entire
* bucket will be downloaded. You can modify this behavior by providing a
* {@link DownloadDirectoryRequest#listObjectsRequestTransformer()} and/or
* a {@link DownloadDirectoryRequest#filter()} in {@link DownloadDirectoryRequest} to
* limit the S3 objects to download.
*
* <p>
* The downloaded directory structure will match with the provided S3 virtual bucket.
Expand Down Expand Up @@ -575,12 +578,13 @@ default DirectoryUpload uploadDirectory(Consumer<UploadDirectoryRequest.Builder>
* DownloadDirectoryRequest.builder()
* .destination(Paths.get("destination/directory"))
* .bucket("bucket")
* .listObjectsV2RequestTransformer(l -> l.prefix("prefix"))
* // only download objects with prefix "photos"
* .listObjectsV2RequestTransformer(l -> l.prefix("photos"))
* .build());
* // Wait for the transfer to complete
* CompletedDirectoryDownload completedDirectoryDownload = directoryDownload.completionFuture().join();
*
* // Print out the failed downloads
* // Print out any failed downloads
* completedDirectoryDownload.failedTransfers().forEach(System.out::println);
* }
*
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -36,10 +36,10 @@ public interface FileUpload extends ObjectTransfer {
* See {@link ResumableFileUpload} for supported formats.
*
* <p>
* Currently, it's only supported if the underlying {@link S3AsyncClient} is CRT-based created via
* {@link S3AsyncClient#crtBuilder()} or {@link S3AsyncClient#crtCreate()}.
* Currently, it's only supported if the underlying {@link S3AsyncClient} is CRT-based (created via
* {@link S3AsyncClient#crtBuilder()} or {@link S3AsyncClient#crtCreate()}).
* It will throw {@link UnsupportedOperationException} if the {@link S3TransferManager} is created
* with a non CRT-based S3 client, i.e., created by {@link S3AsyncClient#builder()}.
* with a non CRT-based S3 client (created via {@link S3AsyncClient#builder()}).
*
* @return A {@link ResumableFileUpload} that can be used to resume the upload.
*/
Expand Down

0 comments on commit bd14714

Please sign in to comment.