Driver Implementation for the cloudscale.ch API #951

davidhalter · 2016-11-22T15:29:53Z

A Driver Implementation for the cloudscale.ch API

Description

It's pretty simple, I have added a new driver for the cloudscale.ch API (https://www.cloudscale.ch/en/api/v1). The cloudscale.ch API is very simple and doesn't offer a lot more than what libcloud's compute module by default offers. I have also tried to communicate this in the documentation.

Status

The code feels mostly like it's done and ready for review. It passes 2.7, 3.3, 3.4, 3.5, lint and pylint. I have looked at other drivers in libcloud and tried to copy how they document/test. I haven't added tons of tests, but comparably to others the driver is also smaller and therefore the amount of tests seem to align.

Just let me know what you think. Happy to improve my code!

Checklist (tick everything that applies)

Code linting (required, can be done after the PR checks)
Documentation
Tests
ICLA (required for bigger changes)

Feuermurmel · 2016-11-23T09:50:01Z

docs/compute/drivers/cloudscale.rst

+---------------------
+
+Simply visit `<https://control.cloudscale.ch/user/api-tokens>`_ and start
+having fun!


Maybe you want to reword this. ;)

Feuermurmel · 2016-11-23T09:52:51Z

docs/examples/compute/cloudscale/cloudscale_compute_simple.py

+    libcloud.DriverType.COMPUTE.CLOUDSCALE
+)
+
+driver = cls('3pjzjh3h3rfynqa4iemvtvc33pyfzss2')


I'd store the token string in a variable token to clarify what that string is. Or maybe add a comment.

Feuermurmel · 2016-11-23T09:56:13Z

libcloud/compute/drivers/cloudscale.py

+        '''
+        return self._list_resources('/v1/images', self._to_image)
+
+    def create_node(self, name, size, image, location=None, ex_create_attr={}):


What do you think about accepting the arguments in ex_create_attr directly separate parameters instead? Compare with CloudStackNodeDriver.create_node(): https://github.com/apache/libcloud/blob/trunk/libcloud/compute/drivers/cloudstack.py#L1528

I would argue that some other drivers (e.g. digitalocean) are doing it the same way. Prefixing everything with ex_ feels clunky and having a dictionary makes it easy to be "future-compatible".

I'm ok with this, as long as the attributes are documented somewhere. You could also name the main ones and just merge them into the dictionary.

I have documented in docs/compute/drivers/cloudscale.rst. Should I add it to the docstring?

Feuermurmel · 2016-11-23T10:02:28Z

docs/compute/drivers/cloudscale.rst

+- ``use_private_network``: ``boolean`` Attaching/Detaching the private network interface.
+- ``use_ipv6``: ``boolean`` Enabling/Disabling IPv6.
+- ``anti_affinity_with``: ``string``; Pass the UUID of another server.
+- ``user_data``: ``string``; Cloud-init configuration (cloud-config). Provide YAML.


I'd use Python's standard names for the types here (i.e. bool, str, …).

Also, is there a standard convention for how to format such lists with type information, e.g. here the format <id> (<type>) – <description> is used.

I have adapted that. I think it's nicer.

Feuermurmel · 2016-11-23T10:05:55Z

libcloud/compute/drivers/cloudscale.py

+
+class CloudscaleConnection(ConnectionKey):
+    """
+    Connection class for the Vultr driver.


You're not covering your tracks very well. :P

Feuermurmel · 2016-11-23T10:08:30Z

libcloud/test/compute/fixtures/cloudscale/list_images.json

@@ -0,0 +1 @@
+[{"slug":"ubuntu-16.10","name":"Ubuntu 16.10 (2016-10-17)","operating_system": "Ubuntu"}]


Why no pretty printing here? :(

If you want to keep it on a single line, if suggest adding spaces after : and ,.

I have actually removed the pretty printing because other drivers also don't use it (and this one is pretty short).

Then at least remove that space after the last :. :P

davidhalter · 2016-11-23T14:58:35Z

@Feuermurmel I have improved pretty much all the documentation that you complained about, except for the two places where I've commented.

tonybaloney

Overall this is a great contribution, I had a concern about the way errors are handled if there is a cleaner way of achieving this?

tonybaloney · 2016-11-28T05:14:27Z

libcloud/compute/drivers/cloudscale.py

+
+
+class CloudscaleResponse(JsonResponse):
+    valid_response_codes = [httplib.OK, httplib.ACCEPTED, httplib.CREATED,


this is overridden just because of the addition of httplib.NO_CONTENT?

Hmm, I don't see us overriding anything here, valid_response_codes does not exist in JsonResponse or Response.

correct, never mind https://github.com/apache/libcloud/blob/trunk/libcloud/common/base.py#L216

tonybaloney · 2016-11-28T05:15:18Z

libcloud/compute/drivers/cloudscale.py

+            # but that doesn't really matter. It's nicer if the error is just
+            # one error (because it's a Python API and there's only one
+            # exception.
+            return next(iter(body.values()))


wah, ok what's this about? This is unusual!

It means "take the first/any element of the collection". There's no nicer way to express it if your collection is not a list.

If it means that we change the base method then I'd be happier with that, you should have a bit more control over the meta in the exception. Assuming that the other errors in the response could provide context to the user about what they did wrong they shouldn't have to use a packet sniffer to get that data https://github.com/apache/libcloud/blob/trunk/libcloud/common/base.py#L178-L180
Change that to an instance method (self.exception_from_message), which calls the top level function.
Override that method in your own class to raise a new exception class, inheriting from LibcloudException, adding a field for the list of errors. update __str__ with the first error, or a list joined with commas?

@davidhalter

I'll do this another time. 👍 merging

tonybaloney · 2016-11-28T05:17:30Z

libcloud/compute/drivers/cloudscale.py

+        '''
+        return self._list_resources('/v1/images', self._to_image)
+
+    def create_node(self, name, size, image, location=None, ex_create_attr={}):


I'm ok with this, as long as the attributes are documented somewhere. You could also name the main ones and just merge them into the dictionary.

tonybaloney · 2016-11-28T05:19:01Z

libcloud/compute/drivers/cloudscale.py

+        )
+        return response.status == httplib.OK
+
+    def _list_resources(self, url, tranform_func):


Nice pattern 👍

href · 2016-11-28T08:24:55Z

docs/compute/drivers/cloudscale.rst

+
+- ``driver.list_sizes()``
+- ``driver.list_images()``
+- ``driver.list_sizes()``


It seems like this is a duplicate.

Hmm, how is it a duplicate? Do you mean the autoclass listing?

list_sizes is twice

Oh, how did I not see that after you pointed it out. Thank you!

It should be list_nodes. I have changed it.

davidhalter · 2016-11-28T13:13:45Z

@tonybaloney How would you like me to modify error handling? Or what do you think is not clean enough?

tonybaloney · 2016-11-28T20:52:02Z

@davidhalter if you haven't already, I need you to sign the ASF ICLA (an agreement for individual contributors) https://libcloud.readthedocs.io/en/latest/development.html#contributing-bigger-changes

davidhalter · 2016-11-29T09:51:03Z

@tonybaloney

I have signed it and I have also gotten confirmation:

This message acknowledges receipt of your ICLA, which has been filed in the Apache Software Foundation records.
With this message, the Libcloud PMC has been notified that your ICLA has been filed.

closes #951

tonybaloney · 2016-12-02T10:33:12Z

merged. thanks @davidhalter @pquentin @Feuermurmel and @href for getting this over the line and thanks for contributing to the project. New drivers can take a little bit of time but it's important to get some consistency.

davidhalter · 2016-12-02T14:57:56Z

@tonybaloney Thanks for reacting so quickly. Wish I could do that on my open source projects as well :) Keep up the good work!

davidhalter · 2017-01-03T13:54:06Z

@tonybaloney The documentation doesn't seem like it's up-to-date. Did we break the builds with this PR?

(AFAIK I was able to build the docs with this commit).

Kami · 2017-01-03T15:04:34Z

@davidhalter It looks like that for some reasons docs builds haven't been automatically triggered for a while now.

I manually triggered it for latest for now and I will look what is going on with automatically triggered builds.

Kami · 2017-01-03T15:24:50Z

The manually triggered build has finished and the provider is listed there now - https://libcloud.readthedocs.io/en/latest/compute/supported_providers.html

I believe there were two issues.

Automatic build triggering has been broken for a while now (:sadface:) - c06497f
Table with link to the provider documentation has been broken. Just a heads up - this table is automatically generated using ./contrib/generate_provider_feature_matrix_table.py script - e5fc049

I believe both issues should be fixed now, but need to wait for the Travis and RTD build to finish :)

Kami · 2017-01-03T15:42:29Z

It looks like automatic docs build triggering is working again - https://travis-ci.org/apache/libcloud/jobs/188560819#L855 and provider matrix is also back again https://libcloud.readthedocs.io/en/latest/supported_providers.html#provider-matrix :)

/cc @tonybaloney

davidhalter · 2017-01-04T11:03:30Z

@Kami Thanks! Is there any way to PR for the website as well? I wasn't able to find a repository of it.

davidhalter added 3 commits November 22, 2016 13:50

An initial implementation for the cloudscale.ch API.

c6ad701

Fix a pep8 linter issue.

588cb0c

Create documentation for the cloudscale driver.

fd9fefd

Feuermurmel reviewed Nov 23, 2016

View reviewed changes

Improved the documentation in a few places.

d1b7d78

Fix a small typo.

8f11620

tonybaloney reviewed Nov 28, 2016

View reviewed changes

href reviewed Nov 28, 2016

View reviewed changes

Remove a space to please @Feuermurmel.

5bf298f

Change the duplicate of list_sizes and rename it to list_nodes.

75300c5

asfgit merged commit 75300c5 into apache:trunk Dec 2, 2016

asfgit pushed a commit that referenced this pull request Dec 2, 2016

Merge branch 'github-951' into trunk

28dc1c3

closes #951

asfgit pushed a commit that referenced this pull request Dec 2, 2016

change for #951

b2662d5

		@@ -0,0 +1 @@
		[{"slug":"ubuntu-16.10","name":"Ubuntu 16.10 (2016-10-17)","operating_system": "Ubuntu"}]



		class CloudscaleResponse(JsonResponse):
		valid_response_codes = [httplib.OK, httplib.ACCEPTED, httplib.CREATED,

Driver Implementation for the cloudscale.ch API #951

Driver Implementation for the cloudscale.ch API #951

Conversation

davidhalter commented Nov 22, 2016 • edited Loading

A Driver Implementation for the cloudscale.ch API

Description

Status

Checklist (tick everything that applies)

Choose a reason for hiding this comment

Feuermurmel Nov 23, 2016 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

davidhalter Nov 23, 2016 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

davidhalter commented Nov 23, 2016

tonybaloney left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

davidhalter commented Nov 28, 2016

tonybaloney commented Nov 28, 2016

davidhalter commented Nov 29, 2016

tonybaloney commented Dec 2, 2016

davidhalter commented Dec 2, 2016 • edited Loading

davidhalter commented Jan 3, 2017

Kami commented Jan 3, 2017

Kami commented Jan 3, 2017 • edited Loading

Kami commented Jan 3, 2017

davidhalter commented Jan 4, 2017

davidhalter commented Nov 22, 2016 •

edited

Loading

Feuermurmel Nov 23, 2016 •

edited

Loading

davidhalter Nov 23, 2016 •

edited

Loading

davidhalter commented Dec 2, 2016 •

edited

Loading

Kami commented Jan 3, 2017 •

edited

Loading