Skip to content
Permalink
Browse files
Merge pull request #3 from NOPping/gce_ga_support
Finish basic ga support addition, close #2
  • Loading branch information
brogand93 committed May 29, 2014
2 parents f1c5504 + 8fa3a58 commit dd47b10abb3bf22acc390116fc253a0519cf4919
Show file tree
Hide file tree
Showing 15 changed files with 1,011 additions and 732 deletions.
@@ -1,4 +1,4 @@
gstack/data/app.db
*.db

*.py[cod]
*~
308 README.md

This file was deleted.

@@ -0,0 +1,147 @@
========
GSTACK
========

A Google Compute Engine Interface For Cloudstack
################################################

.. image:: https://badge.fury.io/py/gstack.png
.. image:: https://api.travis-ci.org/NOPping/gstack.png

Proposal
_________

This project aims to provide a new compute API interface for Apache Cloudstack that is compatible with Google Compute Engine `GCE <https://cloud.google.com/products/compute-engine>`_ GCE is Google's Infrastructure-as-a-Service (IaaS) compute service that competes with Amazon Web Services EC2. In short, this is a mapping of the GCE `API <https://developers.google.com/compute/docs/reference/latest/>`_ and the CloudStack `API <http://cloudstack.apache.org/docs/api/index.html>`_

Implementation
______________

The GCE APIs for Cloudstack is supplied as a layer over the current Cloudstack API. The given application will take in a GCE based API request, pass it over to the Cloudstack API, get the required data and return it in a suitable format.

It is written in Python, using `Flask <http://flask.pocoo.org/>`_ to expose a GCE compliant REST API. Requests, Pycrypto, Pyopenssl and Flask-sqlachemy are dependencies. A custom version of pyoauth2 is included in the source.

- Disks
- listdisks
- getdisk
- Firewalls
- addfirewall
- deletefirewall
- listfirewalls
- getfirewall
- Images
- listimages
- getimage
- Instances
- addinstance
- deleteinstance
- listinstances
- getinstance
- Machinetypes
- listmachinetypes
- getmachinetype
- Project
- getproject
- Zones
- listzones
- getzone


Installation
#############

Developers
___________

Clone the repository

git clone https://github.com/NOPping/gstack.git

Install the package

python ./setup.py install

Users
_____

Users can grab the package from Pypi

pip install gstack

Configuration
#############

Before running `gstack` you must configure it. To do so run

gstack-configure


And enter your configuration information as prompted.

Install the stand alone `gcutil <https://developers.google.com/compute/docs/gcutil/#gcutilupgrade/>`_

gstack comes with a self-signed certificate for the local endpoint ``gstack/data/server.crt``, copy the certificate to the gcutil certificates file ``gcutil/lib/httplib2/httplib2/cacerts.txt``

At this stage your CloudStack apikey and secretkey need to be entered in the gcutil auth_helper.py file at ``gcutil/lib/google_compute_engine/gcutil/auth_helper.py``

This is far from ideal and we opened a feature request with google to pass the ``client_id`` and ``client_secret`` as options to gcutil, hopefully future release of gcutil will allow us to do so.

Start gstack:

gstack


Create a cached parameters file for gcutil:

- Make a flagile, something like ``~/.gcutil_params``
- Insert required flags to ease usage. Something like:


`--auth_local_webserver`

`--auth_host_port=9999`

`--dump_request_response`

`--authorization_uri_base=https://localhost:5000/oauth2`

`--ssh_user=root`

`--fetch_discovery`

`--auth_host_name=localhost`

`--api_host=https://localhost:5000/`

`--nocheck_for_new_version`


gcutil will issue auth requests to the local Flask application, get an OAuth token and then issue requests to the CloudStack endpoint you specified when cofiguring gstack.

Usage
######

You can start issuing standard gcutil commands.

$ ./gcutil --flag_file=~/.gcutil_params --project=brogand93@darrenbrogan.ie listzones


================== ======== ====================
name status next-maintenance
================== ======== ====================
Sandbox-simulator UP None scheduled
================== ======== ====================


Trouble shooting
#################

If you encounter authentication/authorization issues, clean up your gcutil authentication information ``rm -rf ~/.gcutil_auth``, make sure that you set your ``client_id`` and ``client_secret`` in ``gcutil/lib/google_compute_engine/gcutil/auth_helper.py``

Apache CloudStack
##################

For more information about CloudStack check the official `<website http://cloudstack.apache.org>`_

Copyright © 2013 The Apache Software Foundation, Licensed under the Apache License, Version 2.0.
"Apache", "CloudStack", "Apache CloudStack", and the Apache feather logos are registered trademarks or trademarks of The Apache Software Foundation.
g
@@ -36,7 +36,7 @@ def _create_config_folder():
def _create_config_file(config_folder):
config_file = open(config_folder + '/gstack.conf', 'w+')

config_file.write('PATH = \'%s\'\n' % 'compute/v1beta15/projects/')
config_file.write('PATH = \'%s\'\n' % 'compute/v1/projects/')

gstack_address = raw_input('gstack bind address [0.0.0.0]: ')
if gstack_address == '':
@@ -81,18 +81,37 @@ def _cloudstack_volume_to_gce(cloudstack_response, projectid, zone):
return response


@app.route(
'/' + app.config['PATH'] + '<projectid>/aggregated/disks', methods=['GET'])
@app.route('/' + app.config['PATH'] + '<projectid>/aggregated/disks', methods=['GET'])
@authentication.required
def aggregatedlistdisks(projectid, authorization):
disk_list = _get_disks(authorization=authorization)
zone_list = zones.get_zone_names(authorization=authorization)

disk = None
filter = helper.get_filter(request.args)

if 'name' in filter:
disk = filter['name']

items = {}

for zone in zone_list:
zone_disks = []
if disk_list['listvolumesresponse']:
if disk:
disk = get_disk_by_name(
authorization=authorization,
disk=disk
)
if disk:
zone_disks.append(
_cloudstack_volume_to_gce(
cloudstack_response=disk,
projectid=projectid,
zone=zone,
)
)

elif disk_list['listvolumesresponse']:
for disk in disk_list['listvolumesresponse']['volume']:
zone_disks.append(
_cloudstack_volume_to_gce(
@@ -102,7 +121,6 @@ def aggregatedlistdisks(projectid, authorization):
)
)
items['zone/' + zone] = {}
items['zone/' + zone]['zone'] = zone
items['zone/' + zone]['disks'] = zone_disks

populated_response = {
@@ -115,8 +133,7 @@ def aggregatedlistdisks(projectid, authorization):
return helper.create_response(data=populated_response)


@app.route('/' + app.config['PATH'] +
'<projectid>/zones/<zone>/disks', methods=['GET'])
@app.route('/' + app.config['PATH'] + '<projectid>/zones/<zone>/disks', methods=['GET'])
@authentication.required
def listdisks(projectid, authorization, zone):
disk = None
@@ -167,11 +184,7 @@ def listdisks(projectid, authorization, zone):
return helper.create_response(data=populated_response)


@app.route(
'/' +
app.config['PATH'] +
'<projectid>/zones/<zone>/disks/<disk>',
methods=['GET'])
@app.route('/' + app.config['PATH'] + '<projectid>/zones/<zone>/disks/<disk>', methods=['GET'])
@authentication.required
def getdisk(projectid, authorization, zone, disk):
response = get_disk_by_name(
@@ -48,4 +48,4 @@ def filter_by_name(data, name):

def get_root_url():
return 'https://' + \
app.config['LISTEN_ADDRESS'] + ':' + app.config['LISTEN_PORT']
app.config['GSTACK_BIND_ADDRESS'] + ':' + app.config['GSTACK_PORT']
@@ -23,9 +23,9 @@
import json


@app.route('/discovery/v1/apis/compute/v1beta15/rest', methods=['GET'])
@app.route('/discovery/v1/apis/compute/v1/rest', methods=['GET'])
def discovery():
with open(app.config['DATA'] + '/v1beta15.json') as template:
with open(app.config['DATA'] + '/v1.json') as template:
discovery_template = json.loads(template.read())

discovery_template[
@@ -118,18 +118,19 @@ def _cloudstack_virtual_machine_to_gce(cloudstack_response, zone, projectid):
response['disks'] = []

networking = {}
networking['network'] = cloudstack_response['securitygroup'][0]['name']
networking['networkIP'] = cloudstack_response['nic'][0]['ipaddress']
networking['name'] = cloudstack_response['nic'][0]['id']
networking['accessConfigs'] = []
if cloudstack_response['securitygroup']:
networking['network'] = cloudstack_response['securitygroup'][0]['name']
networking['networkIP'] = cloudstack_response['nic'][0]['ipaddress']
networking['name'] = cloudstack_response['nic'][0]['id']
networking['accessConfigs'] = []

accessconfig = {}
accessconfig['kind'] = 'compute#accessConfig'
accessconfig['type'] = 'ONE_TO_ONE_NAT'
accessconfig['name'] = 'External NAT'
accessconfig['natIP'] = cloudstack_response['nic'][0]['ipaddress']

networking['accessConfigs'].append(accessconfig)
networking['accessConfigs'] = accessconfig

response['networkInterfaces'].append(networking)

@@ -163,32 +164,47 @@ def _get_virtual_machine_by_name(authorization, instance):
return None


@app.route(
'/' +
app.config['PATH'] +
'<projectid>/aggregated/instances',
methods=['GET'])
@app.route('/' + app.config['PATH'] + '<projectid>/aggregated/instances', methods=['GET'])
@authentication.required
def aggregatedlistinstances(authorization, projectid):
zone_list = zones.get_zone_names(authorization=authorization)
virtual_machine_list = _get_virtual_machines(authorization=authorization)

instance = None
filter = helper.get_filter(request.args)

if 'name' in filter:
instance = filter['name']

items = {}

for zone in zone_list:
zones_instances = []
if virtual_machine_list['listvirtualmachinesresponse']:
zone_instances = []
if instance:
virtual_machine = _get_virtual_machine_by_name(
authorization=authorization,
instance=instance
)
if virtual_machine:
zone_instances.append(
_cloudstack_virtual_machine_to_gce(
cloudstack_response=virtual_machine,
projectid=projectid,
zone=zone
)
)

elif virtual_machine_list['listvirtualmachinesresponse']:
for instance in virtual_machine_list['listvirtualmachinesresponse']['virtualmachine']:
zones_instances.append(
zone_instances.append(
_cloudstack_virtual_machine_to_gce(
cloudstack_response=instance,
projectid=projectid,
zone=zone
)
)
items['zone/' + zone] = {}
items['zone/' + zone]['zone'] = zone
items['zone/' + zone]['instances'] = zones_instances
items['zone/' + zone]['instances'] = zone_instances

populated_response = {
'kind': 'compute#instanceAggregatedList',
@@ -199,11 +215,7 @@ def aggregatedlistinstances(authorization, projectid):
return helper.create_response(data=populated_response)


@app.route(
'/' +
app.config['PATH'] +
'<projectid>/zones/<zone>/instances',
methods=['GET'])
@app.route('/' + app.config['PATH'] + '<projectid>/zones/<zone>/instances', methods=['GET'])
@authentication.required
def listinstances(authorization, projectid, zone):
instance = None
@@ -250,11 +262,7 @@ def listinstances(authorization, projectid, zone):
return helper.create_response(data=populated_response)


@app.route(
'/' +
app.config['PATH'] +
'<projectid>/zones/<zone>/instances/<instance>',
methods=['GET'])
@app.route('/' + app.config['PATH'] + '<projectid>/zones/<zone>/instances/<instance>', methods=['GET'])
@authentication.required
def getinstance(projectid, authorization, zone, instance):
response = _get_virtual_machine_by_name(
@@ -278,18 +286,15 @@ def getinstance(projectid, authorization, zone, instance):
return errors.resource_not_found(function_route)


@app.route(
'/' +
app.config['PATH'] +
'<projectid>/zones/<zone>/instances',
methods=['POST'])
@app.route('/' + app.config['PATH'] + '<projectid>/zones/<zone>/instances', methods=['POST'])
@authentication.required
def addinstance(authorization, projectid, zone):
data = json.loads(request.data)
print data
args = {}
args['name'] = data['name']
args['serviceoffering'] = data['machineType'].rsplit('/', 1)[1]
args['template'] = data['image'].rsplit('/', 1)[1]
args['template'] = data['disks'][0]['initializeParams']['sourceImage'].rsplit('/', 1)[1]
args['zone'] = zone

network = data['networkInterfaces'][0]['network'].rsplit('/', 1)[1]
@@ -324,11 +329,7 @@ def addinstance(authorization, projectid, zone):
return helper.create_response(data=populated_response)


@app.route(
'/' +
app.config['PATH'] +
'<projectid>/zones/<zone>/instances/<instance>',
methods=['DELETE'])
@app.route('/' + app.config['PATH'] + '<projectid>/zones/<zone>/instances/<instance>', methods=['DELETE'])
@authentication.required
def deleteinstance(projectid, authorization, zone, instance):
deletion_result = _destroy_virtual_machine(authorization, instance)

0 comments on commit dd47b10

Please sign in to comment.