Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix various bugs and add "floating IP handler" to the OpenStack provider #301

Closed
wants to merge 11 commits into from

Conversation

@hcs42
Copy link
Contributor

hcs42 commented May 27, 2014

This commit contains:

  • Fixes of typos in docstrings and comments
  • 7 various small bugfixes
  • a small feature (add "floating IP handler" to the OpenStack provider)
  • a small addition to the DUMMY provider

If I should break up this pull request into several pull requests and/or I should write Jira tickets, just let me know.

hcs42 added 5 commits Apr 10, 2014
"id" is a built-in function in Python, which caused this error to slip through
the unit tests.
The original code was faulty, because the
ex_find_or_import_keypair_by_key_material function returned a key pair object,
and that object was placed in params['KeyName'], which was later expected to
contain only the key name, not the key pair object.

This fix places only the key name in params['KeyName'].
In Python 3, the original code didn't work, because base64key was a byte
string, and it became a value in the `params` dictionary, but that needs
strings as values.
@@ -16,6 +16,10 @@
import base64
import hashlib

from Crypto.Util.py3compat import bchr

This comment has been minimized.

@Kami

Kami May 27, 2014 Member

paramiko (and pycrypto) is an optional dependency, but get_pubkey_openssh_fingerprint should also work without those libraries so we can't rely on bchr function from pycrypto.

This comment has been minimized.

@hcs42

hcs42 May 27, 2014 Author Contributor

Is it OK if I add a bchr function to our py3 module?

This comment has been minimized.

@Kami

Kami May 27, 2014 Member

Yep

@@ -93,6 +101,11 @@ def dictvalues(d):

def tostring(node):
return ET.tostring(node, encoding='unicode')

def hexadigits(s):

This comment has been minimized.

@Kami

Kami May 27, 2014 Member

It would be nice to add a small test case for this.

This comment has been minimized.

@hcs42

hcs42 May 27, 2014 Author Contributor

Will do.

@@ -2168,6 +2168,14 @@ def ex_list_floating_ip_pools(self):
return self._to_floating_ip_pools(
self.connection.request('/os-floating-ip-pools').object)

def ex_create_floating_ip_handler(self):

This comment has been minimized.

@Kami

Kami May 27, 2014 Member

For consistency, please avoid handler approach and add method directly to the driver. I'm not totally sure about the name though since ex_create_floating_ip might be confusing since we already have OpenStack_1_1_FloatingIpPool.create_floating_ip.

This comment has been minimized.

@hcs42

hcs42 May 27, 2014 Author Contributor

I did it this way so that I can easier reuse the code from OpenStack_1_1_FloatingIpHandler.

But that's not a lot of code, and I agree that it would be more convenient to use this functionality without having to create a handler. I will rewrite it and we will see how that looks like.

hcs42 added 6 commits May 27, 2014
In Python 3, the original code didn't work, because there is no `encode`
function for binaries:

    (Python3.3)>>> [x.encode("hex") for x in b"abc"]
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "<stdin>", line 1, in <listcomp>
    AttributeError: 'int' object has no attribute 'encode'

String do have an `encode` method, but it is a different function from the
`encode` in Python 2:

    (Python3.3)>>> [x.encode("hex") for x in "abc"]
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "<stdin>", line 1, in <listcomp>
    LookupError: unknown encoding: hex

So I added a "hexadigits" function to our compatibility module (py3) and used
that in _to_md5_fingerprint.
In Python 3, the original code didn't work, because there we cannot concatenate
a string and a byte string.

The new code produces two binaries in Python 3 and two string in Python 2, so
they can be concatenated in both versions.
If the cidr_ips parameter in this function is None, and therefore the
IpPermissions.1.IpRanges.1.CidrIp is not present in the request to the EC2 API,
the request is not performed. (The response does not indicate an error, but the
security rule is not created).

This behaviour is not documented in
http://docs.aws.amazon.com/AWSEC2/latest/APIReference/ApiReference-query-AuthorizeSecurityGroupIngress.html,
but it is reproducable.

The following request didn't work:

    Action=AuthorizeSecurityGroupIngress
    GroupId=sg-a4b44ece
    IpPermissions.1.FromPort=22
    IpPermissions.1.IpProtocol=tcp
    IpPermissions.1.ToPort=22
    AWSAccessKeyId=...
    Signature=...
    SignatureMethod=HmacSHA256
    SignatureVersion=2
    Timestamp=2014-04-16T07%3A43%3A21Z
    Version=2013-10-15

While this one did (the only difference is adding CidrIp):

    Action=AuthorizeSecurityGroupIngress
    GroupId=sg-a4b44ece
    IpPermissions.1.FromPort=22
    IpPermissions.1.IpProtocol=tcp
    IpPermissions.1.ToPort=22
    IpPermissions.1.IpRanges.1.CidrIp=0.0.0.0/0
    AWSAccessKeyId=...
    Signature=...
    SignatureMethod=HmacSHA256
    SignatureVersion=2
    Timestamp=2014-04-16T07%3A43%3A21Z
    Version=2013-10-15
According to https://wiki.openstack.org/wiki/Os-security-groups#Delete_Security_Group
and my own experiments with the HP cloud, the provider's answer upon success is
not NO_CONTENT (HTTP status code 204) but ACCEPTED (HTTP status code 202).

So I modified to code to accept ACCEPTED. I still kept NO_CONTENT (I would
think it was put there because some provider did return NO_CONTENT).
The Compute API of the HP Cloud used by Libcloud [1] does not support floating IP
pools, but it does support floating IPs. (To be more precise, one needs to use
floating IPs if he wants to access the instance from outside the cloud.)

So I added floating ip handling methods to the OpenStack_1_1_NodeDriver class.

[1] HP Helion Public Cloud v13.5 Compute Service:
https://docs.hpcloud.com/api/v13/compute/#3.10Extensions
@hcs42
Copy link
Contributor Author

hcs42 commented May 28, 2014

I updated three commits:

  • "Fix publickey._to_md5_fingerprint for Python 3": added a unit test.

  • "Fix publickey.get_pubkey_ssh2_fingerprint for Python 3": use our own py3.bchr function.

  • "Add "floating IP" functions to the OpenStack provider": instead of using a handler, I added IP address handler methods directory to the driver class. I used the method name you mentioned, I don't find it's confusing.

    I had to add a "driver" parameter to the OpenStack_1_1_FloatingIpAddress class so that the object can use it to delete itself. I didn't want to use the "pool" object for this, I felt that that would be more confusing. Suggestions about better solutions are welcome.

@asfgit asfgit closed this in 21a0d06 May 28, 2014
@Kami
Copy link
Member

Kami commented May 28, 2014

I've squashed the commits, fixed some lint issues and merged your changes into trunk.

Thanks!

@Kami
Copy link
Member

Kami commented May 29, 2014

@hcs42
Copy link
Contributor Author

hcs42 commented May 29, 2014

Thanks! I posted my opinion in the JIRA ticket.

hcs42 added a commit to esl/elibcloud that referenced this pull request May 29, 2014
The following PR shows that after I proposed my solution for the problem of
using floating IP addressed with ex_create_floating_ip_handler, in the end we
went for a different solution (ex_create_floating_ip):
apache/libcloud#301

This commit changes elibcloud accordingly.
dahlia pushed a commit to spoqa/geofront that referenced this pull request Jul 4, 2014
apache-libcloud 0.15.0 fixed some Python 3 compatibility bugs which
might critical:

apache/libcloud#301
https://libcloud.readthedocs.org/en/latest/changelog.html#changes-with-apache-libcloud-0-15-0
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

2 participants
You can’t perform that action at this time.