Moves from Signature V2 to Signature V4 for S3 #199

Closed
wants to merge 16 commits into
from

Conversation

Projects
None yet
2 participants
@angerman
Contributor

angerman commented Apr 9, 2016

This PR so far adds support for:

  • S3.GetBucket (not yet tested)
  • S3.GetBucketLocation (not yet tested)
  • S3.PutBucket (not yet tested)
  • S3.DeleteBucket (not yet tested)
  • S3.GetService (not yet tested)
  • S3.HeadObject (not yet tested)
  • S3.GetObject (REST)
  • S3.GetObject (URL)
  • S3.PutObject (without Content Sha256 by default)
  • S3.Multipart (not yet tested)
  • S3.CopyObject (not yet tested)
  • S3.DeleteObject (not yet tested)
  • S3.DeleteObjects (not yet tested)
  • IAM Token (not yet tested)

It additionally does:

  • Reuse authorizationV4
  • Adds a pure version of authorizationV4
  • Extracted the Signature (signatureV4) and Credential (credentialV4) generation from the authorizationV4 as these were needed separately to support the URL generation for GetObject.

It also introduces a breaking API change, as Endpoints are replaced by Regions. I tried to keep these as similar to those provided in DynamoDb.

It should fix #167

angerman added some commits Apr 8, 2016

Adds Region (DynamoDb) Datatype to S3
This adds the `Region` datatype, as found in DynamoDb to S3, and
adds all regions as of today with a similar naming scheme, as used
in DynamoDb.

This change lays the basis for Signature V4 signing; it *does* break
backwards compatibility!
Sign using V4 instead of V2
Changes the signature method to version 4 to support the buckets
in China (Beijing) or EU (Frankfurt).
Fixes #167
Extract credentialV4 from authorizationV4
credentialV4 is useful outside of authorizationV4 as well. It is used to build the signed S3 URI link, whe the credentialV4 string is embedded in the canonical request.
@angerman

This comment has been minimized.

Show comment
Hide comment
@angerman

angerman Apr 9, 2016

Contributor

Data.Bifunctor, was introduced in base-4.8.

Contributor

angerman commented Apr 9, 2016

Data.Bifunctor, was introduced in base-4.8.

@angerman

This comment has been minimized.

Show comment
Hide comment
@angerman

angerman Apr 9, 2016

Contributor

Looks like I broke the google storage nearline example.

Contributor

angerman commented Apr 9, 2016

Looks like I broke the google storage nearline example.

@angerman

This comment has been minimized.

Show comment
Hide comment
@angerman

angerman Apr 9, 2016

Contributor

This might eventually break google storage support? I'm not certain about that, and I don't have a gcs account nor did I find detailed documentation on the S3 compatibility.

Contributor

angerman commented Apr 9, 2016

This might eventually break google storage support? I'm not certain about that, and I don't have a gcs account nor did I find detailed documentation on the S3 compatibility.

@aristidb

This comment has been minimized.

Show comment
Hide comment
@aristidb

aristidb Apr 10, 2016

Owner

To keep compatibility with Google, maybe put a SignatureVersion field into the S3Config? Then the google config can just say "V2 signing please".

Owner

aristidb commented Apr 10, 2016

To keep compatibility with Google, maybe put a SignatureVersion field into the S3Config? Then the google config can just say "V2 signing please".

@aristidb

This comment has been minimized.

Show comment
Hide comment
@aristidb

aristidb Apr 10, 2016

Owner

And do I understand it correctly that you consider your pull request complete and ready for merging when you have ticked off all the checkboxes, @angerman ?

Owner

aristidb commented Apr 10, 2016

And do I understand it correctly that you consider your pull request complete and ready for merging when you have ticked off all the checkboxes, @angerman ?

@angerman

This comment has been minimized.

Show comment
Hide comment
@angerman

angerman Apr 10, 2016

Contributor

I guess that would be optimal. Though I must admit, I don't see this getting finished soon. Especially multi-part seems to be quite tricky to get right.

Contributor

angerman commented Apr 10, 2016

I guess that would be optimal. Though I must admit, I don't see this getting finished soon. Especially multi-part seems to be quite tricky to get right.

@mrkkrp mrkkrp referenced this pull request Mar 14, 2017

Closed

S3: invalid request #227

@aristidb

This comment has been minimized.

Show comment
Hide comment
@aristidb

aristidb Dec 10, 2017

Owner

Superseded by #241

Owner

aristidb commented Dec 10, 2017

Superseded by #241

@aristidb aristidb closed this Dec 10, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment