New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Libcloud-589] ProfitBricks Compute Driver Support #352

Closed
wants to merge 38 commits into
base: trunk
from

Conversation

Projects
None yet
3 participants
@baldwinSPC
Contributor

baldwinSPC commented Aug 26, 2014

I have completed support for the ProfitBricks compute driver. This adds support for the core compute driver functions against ProfitBricks. The JIRA ticket can be found here:

https://issues.apache.org/jira/browse/LIBCLOUD-589

Let me know if there are any issues or if it can be accepted or not.

Thanks.
-matt

@Kami

This comment has been minimized.

Show comment
Hide comment
@Kami

Kami Aug 26, 2014

Member

Sweet, thanks!

I will try to review it asap.

Member

Kami commented Aug 26, 2014

Sweet, thanks!

I will try to review it asap.

@Kami

This comment has been minimized.

Show comment
Hide comment
@Kami

Kami Aug 26, 2014

Member

Travis build appears to be failing - https://travis-ci.org/apache/libcloud/builds/33635636

Member

Kami commented Aug 26, 2014

Travis build appears to be failing - https://travis-ci.org/apache/libcloud/builds/33635636

@baldwinSPC

This comment has been minimized.

Show comment
Hide comment
@baldwinSPC

baldwinSPC Aug 26, 2014

Contributor

Thanks. Going through and correcting them now.

On Tue, Aug 26, 2014 at 2:28 PM, Tomaz Muraus notifications@github.com
wrote:

Travis build appears to be failing -
https://travis-ci.org/apache/libcloud/builds/33635636


Reply to this email directly or view it on GitHub
#352 (comment).

Contributor

baldwinSPC commented Aug 26, 2014

Thanks. Going through and correcting them now.

On Tue, Aug 26, 2014 at 2:28 PM, Tomaz Muraus notifications@github.com
wrote:

Travis build appears to be failing -
https://travis-ci.org/apache/libcloud/builds/33635636


Reply to this email directly or view it on GitHub
#352 (comment).

raw=raw)
class Datacenter(UuidMixin):

This comment has been minimized.

@Kami

Kami Aug 30, 2014

Member

Hm, it seems like you could use existing libcloud.compute.baseNodeLocation base class instead of defining a new Profitbricks specific class?

Or is this not possible?

@Kami

Kami Aug 30, 2014

Member

Hm, it seems like you could use existing libcloud.compute.baseNodeLocation base class instead of defining a new Profitbricks specific class?

Or is this not possible?

This comment has been minimized.

@baldwinSPC

baldwinSPC Sep 1, 2014

Contributor

@Kami I have fixed most of what you have raised. I'm now beginning to look at the Datacenter class to see if I can re-use the NodeLocation class. At the moment, I'm not sure if this would be the right approach. The ProfitBricks platform requires you to create a virtual datacenter (VDC) within which you place your instances. This is similar to the Cloud Service concept on Azure. The docs for NodeLocation indicate that this is the physical location of the nodes. A VDC is a virtual location.

I agree that my Datacenter class looks basically like the NodeLocation class. My concern is with usability and users of ProfitBricks understanding that location equates to a virtual datacenter. Additionally, the VDC has a region versus country associated to it, e.g. Las Vegas vs United States. NodeLocation uses 'country'.

As I expand this driver I will expand the Datacenter class to incorporate more features they support on the VDCs, e.g. doing a GET on a VDC will show you all objects attached to that VDC. This seems to all go beyond the scope of NodeLocation.

Thoughts?

@baldwinSPC

baldwinSPC Sep 1, 2014

Contributor

@Kami I have fixed most of what you have raised. I'm now beginning to look at the Datacenter class to see if I can re-use the NodeLocation class. At the moment, I'm not sure if this would be the right approach. The ProfitBricks platform requires you to create a virtual datacenter (VDC) within which you place your instances. This is similar to the Cloud Service concept on Azure. The docs for NodeLocation indicate that this is the physical location of the nodes. A VDC is a virtual location.

I agree that my Datacenter class looks basically like the NodeLocation class. My concern is with usability and users of ProfitBricks understanding that location equates to a virtual datacenter. Additionally, the VDC has a region versus country associated to it, e.g. Las Vegas vs United States. NodeLocation uses 'country'.

As I expand this driver I will expand the Datacenter class to incorporate more features they support on the VDCs, e.g. doing a GET on a VDC will show you all objects attached to that VDC. This seems to all go beyond the scope of NodeLocation.

Thoughts?

@baldwinSPC

This comment has been minimized.

Show comment
Hide comment
@baldwinSPC

baldwinSPC Sep 2, 2014

Contributor

@Kami I've corrected all items you called out in your review. The only pending question is the Datacenter class. Let me know what you're thoughts are now. Travis CI passes all tests. 👍

Contributor

baldwinSPC commented Sep 2, 2014

@Kami I've corrected all items you called out in your review. The only pending question is the Datacenter class. Let me know what you're thoughts are now. Travis CI passes all tests. 👍

@Kami

This comment has been minimized.

Show comment
Hide comment
@Kami

Kami Sep 2, 2014

Member

Added some more comments.

When those are addressed, please squash all the commits.

Member

Kami commented Sep 2, 2014

Added some more comments.

When those are addressed, please squash all the commits.

@Kami

This comment has been minimized.

Show comment
Hide comment
@Kami

Kami Sep 6, 2014

Member

@baldwinSPC How are things looking for those last couple of issues?

I will probably be preparing a new release this weekend and it would be great if those things could get addressed so I can also include this driver.

If you can't do it this weekend, let me know and I will do it myself (address those issues + squash the commits) while merging the patch.

Member

Kami commented Sep 6, 2014

@baldwinSPC How are things looking for those last couple of issues?

I will probably be preparing a new release this weekend and it would be great if those things could get addressed so I can also include this driver.

If you can't do it this weekend, let me know and I will do it myself (address those issues + squash the commits) while merging the patch.

@baldwinSPC

This comment has been minimized.

Show comment
Hide comment
@baldwinSPC

baldwinSPC Sep 6, 2014

Contributor

Hey Tomaz

I've corrected about half of these. I'll commit and push those this
afternoon. I don't think I can correct the others, but what I'll do is
comment on the ones I've finished up.
On Sep 6, 2014 6:30 AM, "Tomaz Muraus" notifications@github.com wrote:

@baldwinSPC https://github.com/baldwinSPC How are things looking for
those last couple of issues?

I will probably be preparing a new release this weekend and it would be
great if those things could get addressed so I can also include this driver.

If you can't do it this weekend, let me know and I will do it myself
(address those issues + squash the commits) while merging the patch.


Reply to this email directly or view it on GitHub
#352 (comment).

Contributor

baldwinSPC commented Sep 6, 2014

Hey Tomaz

I've corrected about half of these. I'll commit and push those this
afternoon. I don't think I can correct the others, but what I'll do is
comment on the ones I've finished up.
On Sep 6, 2014 6:30 AM, "Tomaz Muraus" notifications@github.com wrote:

@baldwinSPC https://github.com/baldwinSPC How are things looking for
those last couple of issues?

I will probably be preparing a new release this weekend and it would be
great if those things could get addressed so I can also include this driver.

If you can't do it this weekend, let me know and I will do it myself
(address those issues + squash the commits) while merging the patch.


Reply to this email directly or view it on GitHub
#352 (comment).

@baldwinSPC

This comment has been minimized.

Show comment
Hide comment
@baldwinSPC

baldwinSPC Sep 7, 2014

Contributor

@Kami I just pushed some updates that address most of the items you called out -- there are only two remaining ones: waiting for state and re-working the ET code.

Contributor

baldwinSPC commented Sep 7, 2014

@Kami I just pushed some updates that address most of the items you called out -- there are only two remaining ones: waiting for state and re-working the ET code.

@Kami

This comment has been minimized.

Show comment
Hide comment
@Kami

Kami Sep 7, 2014

Member

@baldwinmathew Great, thanks.

I'll work on rest of the issues and try to get it merged tonight.

Member

Kami commented Sep 7, 2014

@baldwinmathew Great, thanks.

I'll work on rest of the issues and try to get it merged tonight.

@Kami

This comment has been minimized.

Show comment
Hide comment
@Kami

Kami Sep 7, 2014

Member

Another thing which got me confused while reviewing the changes:

PROVISIONING_STATE = {
        'INACTIVE': NodeState.PENDING,
        'INPROCESS': NodeState.PENDING,
        'AVAILABLE': NodeState.RUNNING,
        'DELETED': NodeState.TERMINATED,
    }

....

def _wait_for_datacenter_state(
            self,
            datacenter,
            state=PROVISIONING_STATE.get(NodeState.RUNNING),
            timeout=300,
            interval=5):

NodeState.RUNNING is mapped to int 0 (libcloud.compute.types.NodeState), but PROVISIONING_STATE dictionary contains strings as keys. Unless I am missing something, this means .get will always return None and _wait function will always time out.

Same goes with while condition - self.PROVISIONING_STATE.get(NodeState.PENDING).

I believe the state should default to state=PROVISIONING_STATE['available']. I will push those fixes to my branch and wait for you to review them before merging them into trunk.

Member

Kami commented Sep 7, 2014

Another thing which got me confused while reviewing the changes:

PROVISIONING_STATE = {
        'INACTIVE': NodeState.PENDING,
        'INPROCESS': NodeState.PENDING,
        'AVAILABLE': NodeState.RUNNING,
        'DELETED': NodeState.TERMINATED,
    }

....

def _wait_for_datacenter_state(
            self,
            datacenter,
            state=PROVISIONING_STATE.get(NodeState.RUNNING),
            timeout=300,
            interval=5):

NodeState.RUNNING is mapped to int 0 (libcloud.compute.types.NodeState), but PROVISIONING_STATE dictionary contains strings as keys. Unless I am missing something, this means .get will always return None and _wait function will always time out.

Same goes with while condition - self.PROVISIONING_STATE.get(NodeState.PENDING).

I believe the state should default to state=PROVISIONING_STATE['available']. I will push those fixes to my branch and wait for you to review them before merging them into trunk.

Various fixes in the Profitbricks driver:
1. Fix "ex_describe_datacenter" to return a single object instead of a list
2. Fix "ex_describe_datacenter" method signature to take an id instead of an object
3. Fix default value for "state" argument in the "_wait_for_datacenter_state" method
4. Fix while condition in the "_wait_for_datacenter_state" method
5. Modify "_wait_for_datacenter_state" to throw an Exception on timeout
6. Do the same fixes in the "_wait_for_storage_volume_state" method
7. Fix "ex_create_datacenter" to return single object instead of a list
8. Fix "create_volume" to return a single object instead of a list
9. Fix "ex_describe_node" to return a single object instead of a list
10. Fix "create_node" to retirn a single object instead of a list
11. Fix "attach_volume" to return volume on success
12. Fix "ex_create_network_interface" to return a single object instead of a list
@Kami

This comment has been minimized.

Show comment
Hide comment
@Kami

Kami Sep 7, 2014

Member

@baldwinSPC I performed another review. During the review I still spotted many issues (mostly with function claiming it returns a single object but it actually returned a list) which I have fixed in my branch - Kami@28e9774

I would appreciate if you can go over my fixes, double check them and test them with the actual Profitbricks API (I don't have an account).

Thanks!

Member

Kami commented Sep 7, 2014

@baldwinSPC I performed another review. During the review I still spotted many issues (mostly with function claiming it returns a single object but it actually returned a list) which I have fixed in my branch - Kami@28e9774

I would appreciate if you can go over my fixes, double check them and test them with the actual Profitbricks API (I don't have an account).

Thanks!

@baldwinSPC

This comment has been minimized.

Show comment
Hide comment
@baldwinSPC

baldwinSPC Sep 9, 2014

Contributor

@Kami Great. Thanks Tomaz. I will be testing through this tomorrow morning and will update this thread.

Contributor

baldwinSPC commented Sep 9, 2014

@Kami Great. Thanks Tomaz. I will be testing through this tomorrow morning and will update this thread.

@baldwinSPC

This comment has been minimized.

Show comment
Hide comment
@baldwinSPC

baldwinSPC Sep 9, 2014

Contributor

@Kami I just merged your branch into mine after testing all the methods live against the API. Everything looks good, however, in my testing I caught two bugs due to empty data coming back from their API. Those are two minor changes you should see on my next commit.

Contributor

baldwinSPC commented Sep 9, 2014

@Kami I just merged your branch into mine after testing all the methods live against the API. Everything looks good, however, in my testing I caught two bugs due to empty data coming back from their API. Those are two minor changes you should see on my next commit.

@baldwinSPC

This comment has been minimized.

Show comment
Hide comment
@baldwinSPC

baldwinSPC Sep 9, 2014

Contributor

@Kami BTW, thank you for all your support. :)

Contributor

baldwinSPC commented Sep 9, 2014

@Kami BTW, thank you for all your support. :)

@Kami

This comment has been minimized.

Show comment
Hide comment
@Kami

Kami Sep 12, 2014

Member

@baldwinmathew Great and you are welcome.

I will look into your changes and and look into merging this pull request into trunk this weekend :)

Member

Kami commented Sep 12, 2014

@baldwinmathew Great and you are welcome.

I will look into your changes and and look into merging this pull request into trunk this weekend :)

@asfgit asfgit closed this in 34cff30 Sep 14, 2014

@Kami

This comment has been minimized.

Show comment
Hide comment
@Kami

Kami Sep 14, 2014

Member

Looks good.

I've made "a bunch of ifs" to "for loop" change and merged patch into trunk. Thanks!

Member

Kami commented Sep 14, 2014

Looks good.

I've made "a bunch of ifs" to "for loop" change and merged patch into trunk. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment