S3 cse encryption and compression #160
Conversation
and support gzip compression on data stored in S3
| catch (Exception x) { | ||
| throw new RuntimeException("Unable to initialize EncryptionMaterialsProvider class " + empClassName + ": " + x, x); | ||
| } | ||
| } |
There was a problem hiding this comment.
I think we want to move the EncryptionMaterialsProvider into its own provider method, having this method depend on it.
|
Sorry for taking a while to get to this PR, but I left a few comments. If you can address these comments I'd be more than happy to merge your pr in |
|
Cool — thanks. I'll try and get to this this week, but all the comments look good. Thanks -nate Nate Sammons From: Andy Kramolisch <notifications@github.commailto:notifications@github.com> Sorry for taking a while to get to this PR, but I left a few comments. If you can address these comments I'd be more than happy to merge your pr in — CONFIDENTIALITY NOTICE: This e-mail and any attachments are for the exclusive and confidential use of the intended recipient and may constitute non-public information. If you received this e-mail in error, disclosing, copying, distributing or taking any action in reliance of this e-mail is strictly prohibited and may be unlawful. Instead, please notify us immediately by return e-mail and promptly delete this message and its attachments from your computer system. We do not waive any work product or other applicable legal privilege(s) by the transmission of this message. |
Simplified code around switching between compressed and uncompressed input/output, added provider method for the encryption materials provider
|
OK, just pushed changes – decided this was more interesting than my conference call this morning :) -nate Nate Sammons From: Andy Kramolisch <notifications@github.commailto:notifications@github.com> Sorry for taking a while to get to this PR, but I left a few comments. If you can address these comments I'd be more than happy to merge your pr in — CONFIDENTIALITY NOTICE: This e-mail and any attachments are for the exclusive and confidential use of the intended recipient and may constitute non-public information. If you received this e-mail in error, disclosing, copying, distributing or taking any action in reliance of this e-mail is strictly prohibited and may be unlawful. Instead, please notify us immediately by return e-mail and promptly delete this message and its attachments from your computer system. We do not waive any work product or other applicable legal privilege(s) by the transmission of this message. |
|
Looks great, thanks for your contribution @natesammons-nasdaq |
S3 cse encryption and compression
|
Thanks! This is great -- thanks for open sourcing this tool, it's extremely handy. Nate Sent from my iPhone On Feb 4, 2016, at 12:29 PM, Andy Kramolisch <notifications@github.commailto:notifications@github.com> wrote: Looks great, thanks for your contribution @natesammons-nasdaqhttps://github.com/natesammons-nasdaq — CONFIDENTIALITY NOTICE: This e-mail and any attachments are for the exclusive and confidential use of the intended recipient and may constitute non-public information. If you received this e-mail in error, disclosing, copying, distributing or taking any action in reliance of this e-mail is strictly prohibited and may be unlawful. Instead, please notify us immediately by return e-mail and promptly delete this message and its attachments from your computer system. We do not waive any work product or other applicable legal privilege(s) by the transmission of this message. |
This change adds support for gzip compression on data stored in S3 to reduce storage size.
This change also adds support for S3 encryption materials provider implementations when dealing with data in S3. This allows for client-managed encryption keys which are still transparently used when viewing/downloading output files stored in S3.