-
Notifications
You must be signed in to change notification settings - Fork 924
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
RFC for AWS SignatureVersion 4 for EC2 #444
Conversation
74f5b2f
to
e4b2a5c
Compare
Great, thanks - I will look asap. |
# For self.method == GET | ||
request_params = '&'.join(["%s=%s" % (urlquote(k, safe=''), urlquote(v, safe='-_~')) | ||
for k, v in sorted(params.items())]) | ||
payload_hash = hashlib.sha256('').hexdigest() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hm, is this correct? It looks like it's just calculating SHA256 hash of an empty string.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, one of the reasons this patch only works for GET-requests. This should be a hash of the payload of the request, which for GET requests is empty. See http://docs.aws.amazon.com/general/latest/gr/sigv4-create-canonical-request.html, step 6.
@Kami what do you think of this? I'm happy to improve this work, but would like some feedback first. |
@gertjanol So far it looks fine, but I didn't have time to test it end to end yet. |
return params | ||
|
||
def pre_connect_hook(self, params, headers): | ||
now = datetime.utcnow() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
datetime.utcnow
returns a naive datetime. You should use datetime.now(tzinfo=libcloud.utils.uso8601.UTC)
.
https://github.com/apache/libcloud/blob/trunk/libcloud/utils/iso8601.py#L65
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
But as this currently works, I deduce that AWS is not that strict about this header 😉
I'm loving it, even though we're only field-tested for EC2. But, that's exactly why there is only the EC2Connection class and no others. 👍 |
@allardhoeve You tested it with the new Frankfurt region? |
@Kami we're using this right now for both Frankfurt and Ireland region (@allardhoeve and me == co-workers :)) |
Cool - I will also look into it (and test it) myself and try to get it merged ASAP. |
@@ -5545,6 +5586,15 @@ class EC2EUNodeDriver(EC2NodeDriver): | |||
_region = 'eu-west-1' | |||
|
|||
|
|||
class EC2EUCentralNodeDriver(EC2NodeDriver): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I just noticed an issue while testing this change - this code will only use signature v4 if you are using old and deprecated class based approach. It won't use it if you use the region
argument.
EC2 = get_driver(Provider.EC2)
driver = EC2('...', '...', region='eu-central-1')
-------- begin 140518062231056 request ----------
curl -i -X GET -H 'Host: ec2.eu-central-1.amazonaws.com' -H 'X-LC-Request-ID: 140518062231056' -H 'Accept-Encoding: gzip,deflate' -H 'User-Agent: libcloud/0.17.1-dev (Amazon EC2) ' --compress 'https://ec2.eu-central-1.amazonaws.com:443/?SignatureVersion=2&AWS...'
I will work on a fix.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I will probably do something along those lines:
- Modify
SignedAWSConnection
class to takesignature_version
argument in the constructor and default it to2
. - Modify
_ex_connection_class_kwargs
method to pass the correct version to the connection class constructor in case the region iseu-central-1
- Remove the new `EC2EUCentralNodeDriver``class since the class based approach has been deprecated quite a while ago now (see https://libcloud.readthedocs.org/en/latest/upgrade_notes.html#amazon-ec2-compute-driver-changes).
@gertjanol Will change nr. 3 affect you? Do you use a class per region approach?
As shortly discussed with @Kami in #407, this patch adds support for Signature Version 4 to the
SignedAWSConnection
class. v4 will be used when a Driver-class selectsEC2V4Connection
as itsconnectionCls
. In this patch this is only enabled for the the new Frankfurt region, represented by a newEC2EUCentralNodeDriver
class.Haven't created any tests yet, since this is just the PoC. If this has any chance of getting merged in, I'll do the tests.
I would like some feedback and thoughts on this work.
Some questions:
pre_connect_hook
to do the work, because we need both the headers and the parameters to calculate the signature. Hope that's ok?SignedAWSConnection
, theELBConnection
, but I’m guessing that that also supports V4. It would reduce complexity if we could drop support for V2 in the SignedAWSConnection altogether.EC2Connection
. I don’t like this, so maybe there is a better way?