S3 Encrypted Client

[tracend]

This request is a direct follow-up to the issue #800 , raised a few days ago.

As previously mentioned, I used the Ruby lib as a basis to reconstruct the logic for the JavaScript lib. Certain liberties were taken, some related with the JavaScript syntax and some with the existing conventions of the AWS SDK; the linting script was responsible for a few tweaks as well.

Please note, the new code relates with the ability to automatically encrypt/decrypt S3 objects from the client-side. It is not currently addressing SSE (Server-side Encryption), which I'm not sure if it's under the AWS.S3 domain.

At the current state my main objective, to read KMS encrypted SES messages is fulfilled. This can be easily done like this:

var params = {
  Bucket: '{{BUCKET_NAME}}',
  Key: '{{BUCKET_ADDRESS}}',
};

var store = new AWS.S3.Encryption.Client({
  accessKeyId: '...',
  secretAccessKey: '...',
  region: '...', 
  sslEnabled: true,
  signatureVersion: 'v4',
});

var downloader = store.getObject(params, function(err, response ){
  // output...
  res.end( response.Body );
});

I'm also satisfied that the logic is laid out so it can grow. Additional abilities are in place, but not fully tested:

putObject

• Encrypt and upload content using a KMSKeyId and EncryptionKey.
• Encrypt and upload content using only a KMSKeyId, with the ``EncryptionKey` automatically generated.
• Encrypt content using a custom kms_client, passed with the options of AWS.S3.Encryption.Client.

getObject

• Decrypt and download content using a KMSKeyId and EncryptionKey.
• Decrypt and download content using only a KMSKeyId, with the ``EncryptionKey` resourced from the file headers (as 'x-amz-key-v2').
• Decrypt content using a custom kms_client, passed with the options of AWS.S3.Encryption.Client.

There is also scaffolding for allowing an instruction file, which will contain the encryption context of an encrypted file, but that functionality is at present disabled.

Some of these features I didn't want to complete by introducing my own conventions. For example, arbitrary including the x-amz-key-v2 key in the putObject response (to allow the user to save that info).

In addition, I tried to incorporate the unit tests from the Ruby lib, although they are failing, mostly because the stub requests for the KMS and S3 services aren't setup properly. I'm sure this can be easily amended by someone more familiar with the inner workings of the API/SDK.

I understand that further adjustments may be required before this code can be integrated to the main lib. That's why I'm committing it as a separate branch. Frankly, I feel it is a significant component that shouldn't be addressed by just one contributor anyway.

On a positive note, everything is contained under the AWS.S3.Encryption namespace and should not interfere with any of the existing functionality. I can use my branch to decrypt SES messages and there's an open invitation for anyone that wants to work on the other features.

Let me know if you have any questions.

[leore]

Hey Guys, Any update on this?

Is this related to http://docs.aws.amazon.com/AmazonS3/latest/dev/UsingClientSideEncryption.html

I see that the javascript library does not yet support Client Side Encryption

cc @robinjoseph08

[jeskew]

Hi @leore,

This PR isn't ready to be merged in at the moment, both because the last commit had some failing tests and because it doesn't seem possible to use it in a browser. I see that files over one megabyte are decrypted into a temp file, and I'm not sure what the performance characteristics of doing that through filesystem-browserify would be.

[leore]

What is the best way to do client side encryption through node?

[mallocator]

+1 Very much need this to decrypt data uploaded from a java client using symmetric client side encryption in my nodejs server.

Only option I see right now is trying to reverse engineer whatever was done in Java.

[mallocator]

would be great to support buffers as key

[albertovasquez]

+1 This is a critical feature needed. If Amazon wanted to allow easy adoption they should have a technical solution, either blog on how to accomplish decrypting or include in sdk.

I'm using this PR and it is working perfectly. Thanks for making this available.

[tracend]

Thanks @andylibrian for pointing out that the decryption may fail sometimes on the finalize method.

According to the documentation:

Once the cipher.final() method has been called, the Cipher object can no longer be used to encrypt data. Attempts to call cipher.final() more than once will result in an error being thrown.

As suggested, I've wrapped it with a try catch to silent the error (temporarily). If anyone has a more proper solution, feel free to contribute.

[guerrerocarlos]

Alternatively, if you have 'being able to do ranged GET' among your requirements, stream-cipher can be used for client side encription (node.js).

[chrisradek]

Thanks for getting the conversation started. We have been investigating a separate encryption client where this functionality would better live.

[tracend]

Oh, I think I did more than just start the conversation, if you were willing to give me credit ;)

[tracend]

The code contributed as a pull request in 2015 is now available as a separate module:

https://github.com/makesites/aws-s3-encryption-client

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs and link to relevant comments in this thread.