GoogleStorageDriver can now use either our S3 authentication or other… #633
Conversation
|
Thanks. Since this is a bigger change I would please you to submit an iCLA - https://libcloud.readthedocs.org/en/latest/development.html#contributing-bigger-changes |
|
Just submitted an ICLA. I am contributing to this as a Google employee, but the ICLA instructions told me to provide my personal email (crunk1@gmail.com), not my work email (crunkleton@google.com). |
|
ICLA has been received and filed. |
|
@erjohnso PTAL |
| @@ -111,6 +116,15 @@ def _is_gce(): | |||
| return False | |||
|
|
|||
|
|
|||
| def _is_gcs_s3(user_id): | |||
| """Checks S3 key format: 20 alphanumeric chars starting with GOOG.""" | |||
| return bool(re.match(r'GOOG[A-Z0-9]{16}', user_id)) | |||
There was a problem hiding this comment.
nit: probably good enough to just use len() and user_id.startswith vs pulling in the re module
|
@crunk1 Thanks. I didn't know you work for Google, I would imagine Google already has CCLA on file. In any case, thanks. |
| API_VERSION = '2006-03-01' | ||
| NAMESPACE = 'http://doc.s3.amazonaws.com/%s' % (API_VERSION) | ||
|
|
||
|
|
||
| class GoogleStorageConnection(ConnectionUserAndKey): | ||
| def _get_aws_auth_param(method, headers, params, expires, secret_key, |
There was a problem hiding this comment.
Might be a good idea to move this to libcloud/common/google.py.
There was a problem hiding this comment.
Some unit tests for this function would also be good.
I believe we have some for AWS so maybe we can borrow or compare with the tests from there.
There was a problem hiding this comment.
Done. Refactored this to use the S3 HMAC Auth method. Also, there were no tests for S3's method, so I added one.
|
@Kami, I believe the Travis failure is not my fault. Please take a look at it/retrigger a build. |
|
@Kami, ping. Sorry if that's annoying, this is blocking a Google-internal dependency of mine. |
|
Changes look good to me, but I will wait for @erjohnso to do a final review and merge it. |
|
@crunk1 - can you squash your commits and rebase to trunk? I'm having a heck of a time applying the patch. That should also trigger a new travis run too, so we can get the assurance that the last failure was a flake. |
… Google Cloud Platform OAuth2 authentication methods. GoogleBaseConnection allows for a GCS_S3 auth type now, but does not handle creating the S3 HMAC header. GoogleBaseConnection still handles OAuth2 for GCE, IA, and SA auth types. Also did some minor cleanup to GoogleBaseConnection. Refactored S3 HMAC auth signature creation method a bit. Refactored GoogleStorageConnection to use S3's signature method instead of reinventing it. Created a test for S3's HMAC signature method. Changed an InvalidContainerNameError to ContainerError in S3 create_container. The exception was being raised on ANY 400 error, which can be returned for things other than an invalid name. In other words, the exception was a misnomer. Added tests for new logic. Did other minor cleanup. Tests run: Some manual tests putting, getting, and deleting objects/buckets. I used a Service Account, Installed App creds, and S3 interoperability creds. python setup.py test tox -e lint
|
Thanks @tonybaloney! |
|
Thanks, everyone! |
… Cloud OAuth2 authentication methods.
GoogleBaseConnection allows for a GCS_S3 auth type now, but does not handle creating the S3 HMAC header. GoogleBaseConnection still handles OAuth2 for GCE, IA, and SA auth types.
GoogleStorageConnection contains the logic for creating an S3 HMAC auth header and the logic for switching between the oauth2 auth or the S3 HMAC auth.
Changed an InvalidContainerNameError to ContainerError in S3 create_container. The exception was being raised on ANY 400 error, which can be returned for things other than an invalid name. In other words, the exception was a misnomer.
Added tests for new logic.
Did other minor cleanup.
Tests run:
Some manual tests putting, getting, and deleting objects/buckets. I used a Service Account, Installed App creds, and S3 interoperability creds.
libcloud.test.common.test_google
libcloud.test.compute.test_gce (to make sure changes to the GoogleBaseConnection didn't break it)
libcloud.test.storage.test_google_storage
libcloud.test.storage.test_s3