-
Notifications
You must be signed in to change notification settings - Fork 3.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
bulkio: allow user to request S3 server-side encryption #31612
Comments
S3 Default Encryption mostly makes this irrelevant. We will reopen if there is significant customer demand to pass custom options for this in the future. |
Fleshing out some details here: KMS requires: AES 256 requires: OR if the customer would like to specify a custom key and MD5 then: No other customer algorithm is supported by AWS according to https://docs.aws.amazon.com/AmazonS3/latest/dev/ServerSideEncryptionCustomerKeysSSEUsingRESTAPI.html. The question boils down to what subset of these options would we like to support when writing to external storage? The decision on whether to use URI params or stringy WITH options can be based on the degree of flexibility we need. |
55834: storage: support AWS KMS server side encryption r=pbardea,dt a=adityamaru S3 URIs now support two additional params to provide server side encryption: AWS_SERVER_ENC_MODE: can be set to either AES256 or aws:kms AWS_SERVER_KMS_ID: if the above is set to aws:kms then this is the ID of the CMK to be used Closes: #31612 Release note (api change): `AWS_SERVER_ENC_MODE` and `AWS_SERVER_KMS_ID` can now be specified as a param in all s3 URIs which write to the store. (egs: Changefeed, BACKUP) Co-authored-by: Aditya Maru <adityamaru@gmail.com>
S3 offers transparent server-side encryption, which is enabled by setting a combination of
x-amz-server-side-encryption
headers on the PutObject request. We should allow the user to pass string values for these options, likely as S3 URL parameters. For full support of server-side encryption, we'd need to allow the user to specify values for the following headers (all of which appear to be available in thePutObjectInput
type in the S3 Go SDK):x-amz-server-side-encryption
x-amz-server-side-encryption-aws-kms-key-id
x-amz-server-side-encryption-context
x-amz-server-side-encryption-customer-algorithm
x-amz-server-side-encryption-customer-key
x-amz-server-side-encryption-customer-key-MD5
The text was updated successfully, but these errors were encountered: