[BEAM-3813] Support encryption for S3FileSystem (SSE-S3, SSE-C and SSE-KMS) #5244
Conversation
|
R: @jbonofre @echauchot |
…ch param - Simplify the logic of default value for s3 upload buffer size in bytes. - Migrate FluentIterable to Java 8 streams.
|
Have not noticed this needed a rebase. @jbonofre PTAL |
|
Thanks for the update. I'm taking a look. |
|
This PR looks great @iemejia . I see a few nice cleanups, too; thanks! |
|
Thanks @jacobmarble. Hoping this gets into the next release. |
|
|
||
| @Description("SSE key for SSE-C encryption, e.g. a base64 encoded key and the algorithm.") | ||
| @Nullable | ||
| SSECustomerKey getSSECustomerKey(); |
There was a problem hiding this comment.
Since this is a custom object type, how does it get serialized/deserialized with Jackson?
There was a problem hiding this comment.
Hi, thanks you have a really valid argument. I hesitated to put the arguments as Strings and build the API objects internally but I thought letting the AWS SDK exposed would move this maintenance to the user. Do you have any suggestion on how to do this properly ?
There was a problem hiding this comment.
I would suggest following the same pattern that was done for the AWSCredentialsProvider. You can see how the class is being converted to and from JSON within AwsModule.java. You effectively define the schema of the object as JSON.
AWSCredentialsProvider to/from JSON had some complexity since it had to deal with inheritance and that it wasn't trivial to convert some credentials providers since they didn't expose all the necessary attributes.
|
|
||
| @Description("KMS key id for SSE-KMS encryption, e.g. \"arn:aws:kms:...\".") | ||
| @Nullable | ||
| SSEAwsKeyManagementParams getSSEAwsKeyManagementParams(); |
There was a problem hiding this comment.
Ditto on SSEAwsKeyManagementParams and Jackson serialization.
No description provided.