Initial support for a Cloudstack builder #922

mindjiver · 2014-02-28T23:09:29Z

Add support for Apache Cloudstack (https://cloudstack.apache.org) as a
builder. Supports creating new templates of existing templates as well
as booting from ISO to PXE boot a full installation to an empty block
device.

To be able to bootstrap a template from an empty block device we need
to be able to netboot and kickstart a OS installation. To do this we
spin up a HTTP web server to send files to the virtual machine. We
assume the virtual machine will be booting from a iPXE ISO which will
chain boot off the userdata we provide when deploying the virtual
machine.

This also supports passing in templated userdata to be downloaded from
the booted virtual machine.

mindjiver · 2014-02-28T23:10:13Z

Comments are highly appreciated.

mindjiver · 2014-02-28T23:13:04Z

builder/cloudstack/step_http_server.go

+	}
+}
+
+func ipAddressToListenOn() *net.IPNet {


Not 100% pleased with this part, will always return the last unicast IP. Needed to template out the userdata which will contain the URL to a kickstart file hosted by this web server.

Add support for Apache Cloudstack (https://cloudstack.apache.org) as a builder. Supports creating new templates of existing templates as well as booting from ISO to PXE boot a full installation to an empty block device. To be able to bootstrap a template from an empty block device we need to be able to netboot and kickstart a OS installation. To do this we spin up a HTTP web server to send files to the virtual machine. We assume the virtual machine will be booting from a iPXE ISO which will chain boot off the userdata we provide when deploying the virtual machine. This also supports passing in templated userdata to be downloaded from the booted virtual machine.

CloudStack should be Cloudstack, also correct some minor spelling nits.

Newest version of gopherstack adds option to skip certificate validation. Should only be used for testing.

Correct bug when trying to template out userdata variables which had not been set. We solve this by setting userdata to an empty string. Also save dummy values for both HTTP IP and port. Also align printout of 'SSH'.

Ensure we have the neccessary values to perform a proper deployVirtualMachine API call.

If a cloudstack build is cancelled/halted we display a message back to the user.

Corrected naming of Cloudstack in underlaying library. Updating usage accordingly.

* Only detach ISOs from virtual machine if it's actually attached * Add configuration option to specify time to wait until we detach ISO

To make debugging of HTTP server problems easier we print both the IP adress and the port where the HTTP server has been exposed.

mitchellh · 2014-04-22T03:58:47Z

This looks decent. Unfortunately, I don't want to merge this without having access to a CloudStack installation that I can use for testing. Do you know any vendors that would be willing to provide a free account for basic testing of this builder?

Until then, please make this a plugin, it would work great!

I'll work on vendors as well. Let me know if you can get me access. I'm going to close for now, but will reopen when/if I can get access. Waah!

vogxn · 2014-04-22T05:30:21Z

Exoscale (exoscale.ch) has trial accounts with a free tier. Also the Apache CloudStack project might have installation endpoints with latest master / development snapshots located widely. I've reached out to the project to help with this. Hopefully, someone will come back here with the needed info.

mindjiver · 2014-04-22T09:08:08Z

@mitchellh Thanks for the feedback. I have some contact with Exoscale and I'll try reaching out to them.

fifthecho · 2014-04-22T17:09:01Z

@mindjiver If you have any issues with Exoscale, we at Datapipe might be able to provide access to a CloudStack cloud.

mitchellh · 2014-04-22T17:52:09Z

Exoscale sounds good. Please let me know if anything comes through. The Packer project only needs basic access (spin up and down a few machines very rarely) to test this sort of thing. Thanks!

retrack · 2014-04-22T19:32:30Z

Hello all, this sounds great. Always happy to help. @mitchellh you can register on https://portal.exoscale.ch/register with the code PACKERROCKS

We will meet at DotScale next month. Happy testing.

ddoc · 2014-04-24T17:17:48Z

I can also provide access to a cloudstack instance for you.

jwthomp · 2014-05-05T23:16:16Z

Checking to see if this is going to open back up due to offers of cloudstack access?

mindjiver · 2014-05-20T09:30:19Z

@retrack @mitchellh Did you guys manage to meet up and get the ball rolling?

lndbrg · 2014-05-22T21:44:34Z

How is this going? I'm interested in cloudstack support for packer as well?

C-Duv · 2014-06-22T22:04:54Z

Couldn't we use apache-libcloud to make Packer support multiple cloud providers instead of having multiple PR to add single provider support?

rickard-von-essen · 2014-06-23T05:02:04Z

Using apache-libcloud would add a dependency on python and libcloud. Currently packer ships as a self contained binary in a zip file. Starting to depend on something on as complicated as python would require "real" packages, this would add lots of reoccurring work for the maintainers.

mindjiver · 2014-06-23T09:51:29Z

What @rickard-von-essen said. However a switch to gocs as the underlaying cloudstack API library would be something to look at. But after this has been merged, if it ever gets merged 😄

C-Duv · 2014-06-23T09:55:42Z

@rickard-von-essen: right, forgot this convenient binary-shipping thing :(

One could do a plugin that depends on this library (or CLI tools that uses this library)?

mindjiver · 2014-06-23T13:45:40Z

@C-Duv to extract this builder into a stand-alone plugin for packer you mean? Sure, that would be possible if this won't be merged. To use the underlaying Golang-library in another project, that would also be possible but gopherstack was pretty tailormade for the things needed in packer.

C-Duv · 2014-06-23T14:13:47Z

@mindjiver yes, but even if this PR gets merged. An apache-libcloud dependent Packer builder plugin (say "LibCloudBuilder") may provide building feature for all the LibCloud-compatible cloud providers.
This dependency could even be that the plugin internally calls a 1:1 lib-CLI tool.

But I'm realizing this PR is not the place for that.

abayer · 2014-07-18T18:22:15Z

Any update on this?

mindjiver · 2014-07-20T17:59:01Z

@abayer Nope.. I have merged some PRs on my fork which now has been some more action on more Cloudstack installations, versions and hypervisors. But I still think the main issue is to get @mitchellh a good reliable account at a provider which uses Cloudstack so he can test towards it.

dduportal · 2014-08-06T16:16:03Z

@mindjiver I (and my company) are also interested with this cloudstack support. I'm trying to learn and build packer from your repo but do you have precompiled binaries for Windows/Unix ?

dduportal · 2014-08-22T14:11:27Z

ping @mindjiver @retrack Any news ?

mindjiver · 2014-09-04T16:59:29Z

@dduportal nope, no news. I might work on making a separate builder which I will build and package for all the necessary platforms. Internally at Klarna we have been using this for over 9 months now.

abayer · 2014-09-24T17:14:57Z

@mindjiver Do you have docs for the CloudStack builder by any chance?

chobbs- · 2014-09-25T18:05:50Z

@mindjiver I'm looking for a bit of help here as well. I was able to get packer compiled with your pull request using instructions from a mail archive but I'm wondering if you have any guidance for spinning up from ISO and PXE?

For others looking, here's what I had to do to get Packer compiled with this pull request:

Prereqs: golang, git, hg and bzr installed (hg is part of mercurial)

go get github.com/mitchellh/gox

export GOPATH=$HOME/go
export PATH=$GOPATH/bin:$PATH
mkdir -p $GOPATH
go get github.com/mitchellh/packer # build download, build and install the
master version of packer into $GOPATH/bin
cd $GOPATH/src/github.com/mitchellh/packer/
git fetch origin refs/pull/922/head && git checkout FETCH_HEAD # will download
the HEAD of the latest version of pull request and check it out
make # will compile packer with cloudstack support and install into $GOPATH/bin

After running make for the first time, it will fail. There are two errors that need to be fixed:

Fix first error:

 1. https://code.google.com/p/google-api-go-client/source/browse/compute/v1beta16?r=2f3a11c875227040c6420113d8d1a634a489031e#v1beta16%253Fstate%253Dclosed
2. Download zip
3. unzip
4. cd go/src/code.google.com/p/google-api-go-client/compute
5. rsync -r ~/Downloads/google-api-go-client-2f3a11c87522/compute/v1beta16 ./

Fix ssh issue:

cd src/code.google.com/p
hg clone https://code.google.com/p/gosshold/
Get patch file here: https://gist.github.com/rickard-von-essen/10444950
cd $GOPATH/src/github.com/mithcellh/packer
git apply < gosshold_packer.patch
Manually fix builder/cloudstack/ssh.go:
1. line 4: change ‘go.crypto’ to ‘gosshold’

Now run make again in $GOPATH/src/github.com/mitchellh/packer/ and it should successfully build and place packer under $GOPATH/bin/

Here's an example template.json:

{
"builders": [
{
"type": "cloudstack",

  "api_url": "https://server/client/api";,
  "api_key": "***", 
  "secret": "***",

  "ssh_timeout": "60s",
  "state_timeout": "100s",

  "ssh_username": "root",
  "ssh_port": 22,
  "ssh_key_path": "id_rsa",
  "http_directory": "http",

  "service_offering_id": "46446411-f9ab-4856-a046-087dc115ee30",
  "template_id": "83ca136a-f8c8-4063-812b-378b6ac9246d",
  "zone_id": "912e5efc-cc9a-4386-9b55-d438274b9b30",
  "network_ids": [ "2c5b96ce-2666-467c-88c4-d0a0575d20ca" ],
  "disk_offering_id": "eca03965-d10e-4b36-b6c4-e697ede7d136",
  "user_data": "testingsadfasd",

  "template_name": "test",
  "template_display_text": "test",
  "template_os_id": 1
}

]
}

Add "insecure_skip_verify": "true", if you don't have a valid cert on your CloudStack instance.

Hope this helps.

mindjiver · 2014-09-25T18:57:05Z

Hi @chobbs- @abayer : I'm planning on working on releasing this a separate builder with some further instructions on how to build templates for Cloudstack using Packer. Including on how to go from an empty block device via iPXE to a auto-installed operating system. I just need some time to setup the proper infrastructure at work to get this going :)

dduportal · 2014-09-26T16:43:15Z

Thanks for the instructions !

I could not build packer from my compatny, because we got a corporate HTTP / HTTPS-only proxy, and the go get seems to be stuck (while git, docker, curl, wget are working perfectly. What the What ?) :

bash-4.1# make
\033[32;01m==> Installing dependencies\033[0m
code.google.com/p/go.crypto (download)
github.com/mitchellh/go-fs (download)
github.com/mitchellh/goamz (download)
github.com/vaughan0/go-ini (download)
github.com/mitchellh/multistep (download)
github.com/mindjiver/gopherstack (download)
github.com/ActiveState/tail (download)
github.com/howeyc/fsnotify (download)
Fetching https://gopkg.in/tomb.v1?go-get=1
Parsing meta tags from https://gopkg.in/tomb.v1?go-get=1 (status code 200)
get "gopkg.in/tomb.v1": found meta tag main.metaImport{Prefix:"gopkg.in/tomb.v1", VCS:"git", RepoRoot:"https://gopkg.in/tomb.v1"} at https://gopkg.in/tomb.v1?go-get=1
gopkg.in/tomb.v1 (download)
^Cmake: *** [deps] Error 2

But guess what, i'm going to publish a container on the Dockerhub like boot2docker iso, which will embed a pre-compiled version, and you will use it to rebuild if wanted :)

Wait & see some days, i'll publish the link here.

dduportal · 2014-09-28T15:37:39Z

OK guys, First build try (Linux 64 only at this time, i need to cross compile for Mac OS / Windows) :
https://registry.hub.docker.com/u/dduportal/packerbuild/dockerfile/

So you just have to :

docker run --rm dduportal/packerbuild:cloudstack-latest > packer
chmod a+x
./packer --help

Native build on Windows confirmed work (except the post-build script, which role is to rename packer_packer to packer).

Base testing are running

benjimcc · 2014-10-09T11:04:06Z

@mindjiver Hello great work on the Cloudstack builder, just wondering if you have any existing template examples which take a Centos base iso all the way through Packer to Cloudstack using your PR.

mindjiver · 2014-10-10T14:39:31Z

@benjimcc See the following diagram

The special chain boot iPXE ISO was built from latest HEAD of master for iPXE with the following embedded script, see http://ipxe.org/embed for more details on how to built the ISO:

#!pxe
dhcp
chain http://${dhcp-server}/latest/userdata

The JSON payload for packer-cloudstack looks like this example:

{
  "provisioners": [
    {
      "type": "shell",
      "scripts": [
        "scripts/base.sh",
        "scripts/motd.sh",
        "scripts/version.sh",
        "scripts/chef-client11.sh",
        "scripts/setupvm.sh",
        "scripts/tuned.sh",
        "scripts/tuneio.sh",
        "scripts/xs-tools.sh",
        "scripts/vmtweaks.sh",
        "scripts/cleanup.sh",
        "scripts/zerodisk.sh"
      ]
    }
  ],
  "builders": [
    {
      "type": "cloudstack",
      "hypervisor": "xenserver",
      "service_offering_id" : "4ccec2a3-0b53-4db0-aebc-6735019581b2",
      "template_id" : "b34f2d7b-2bec-497e-a18e-06d0de94526e",
      "zone_id" : "489e5147-85ba-4f28-a78d-226bf03db47c",
      "disk_offering_id" :"ef781d7f-f8e8-4f73-985c-e0b0a8ef8d48",
      "network_ids" : ["9ab9719e-1f03-40d1-bfbe-b5dbf598e27f"],
      "ssh_username": "root",
      "ssh_key_path": "data/vagrant_insecure_private_key",
      "ssh_timeout": "15m",
      "state_timeout": "30m",
      "template_name": "centos-6.5-20gb-chef11",
      "template_display_text": "CentOS 6.5 20GB chef11",
      "template_os_id": "144",
      "http_directory": "web",
      "user_data": "#!ipxe\nkernel http://{{.HTTPIP}}:{{.HTTPPort}}/vmlinuz ks=http://{{.HTTPIP}}:{{.HTTPPort}}/ks.cfg\ninitrd http://{{.HTTPIP}}:{{.HTTPPort}}/initrd.img\nboot"
    }
  ]
}

I'm serving the vmlinuz, initrd and kickstart files form the webserver packer spins up on the local workstation that is also performing the API calls to Cloudstack.

mindjiver · 2014-10-11T17:49:51Z

@vogxn @fifthecho @retrack @ddoc @jwthomp @C-Duv @ericlathrop @abayer @dduportal @chobbs- @benjimcc Lets move all discussion over to my initial builder release here:

https://github.com/mindjiver/packer-cloudstack

mindjiver reviewed Feb 28, 2014
View reviewed changes

mindjiver mentioned this pull request Apr 7, 2014

Support for CloudStack #76

Closed

Peter Jönsson and others added 10 commits April 7, 2014 22:57

Make error messages specify what you actually need

0ad347c

Correct minor spelling nits

34be3ad

CloudStack should be Cloudstack, also correct some minor spelling nits.

Add option to skip SSL certificate validation

dd8e731

Newest version of gopherstack adds option to skip certificate validation. Should only be used for testing.

Correct variable templating bugs

980836b

Correct bug when trying to template out userdata variables which had not been set. We solve this by setting userdata to an empty string. Also save dummy values for both HTTP IP and port. Also align printout of 'SSH'.

builder/cloudstack: Check all required configuration values

6fefa06

Ensure we have the neccessary values to perform a proper deployVirtualMachine API call.

builder/cloudstack: Display error if build was cancelled

2046ab5

If a cloudstack build is cancelled/halted we display a message back to the user.

build/cloudstack: Correct name of Cloudstack

6e339e3

Corrected naming of Cloudstack in underlaying library. Updating usage accordingly.

builder/cloudstack: More ISO features added

36e1944

* Only detach ISOs from virtual machine if it's actually attached * Add configuration option to specify time to wait until we detach ISO

build/cloudstack: Show IP and port where HTTP server is started

f65bc60

To make debugging of HTTP server problems easier we print both the IP adress and the port where the HTTP server has been exposed.

mitchellh closed this Apr 22, 2014

ghost locked and limited conversation to collaborators Apr 10, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Initial support for a Cloudstack builder #922

Initial support for a Cloudstack builder #922

mindjiver commented Feb 28, 2014

mindjiver commented Feb 28, 2014

mindjiver Feb 28, 2014

mitchellh commented Apr 22, 2014

vogxn commented Apr 22, 2014

mindjiver commented Apr 22, 2014

fifthecho commented Apr 22, 2014

mitchellh commented Apr 22, 2014

retrack commented Apr 22, 2014

ddoc commented Apr 24, 2014

jwthomp commented May 5, 2014

mindjiver commented May 20, 2014

lndbrg commented May 22, 2014

C-Duv commented Jun 22, 2014

rickard-von-essen commented Jun 23, 2014

mindjiver commented Jun 23, 2014

C-Duv commented Jun 23, 2014

mindjiver commented Jun 23, 2014

C-Duv commented Jun 23, 2014

abayer commented Jul 18, 2014

mindjiver commented Jul 20, 2014

dduportal commented Aug 6, 2014

dduportal commented Aug 22, 2014

mindjiver commented Sep 4, 2014

abayer commented Sep 24, 2014

chobbs- commented Sep 25, 2014

mindjiver commented Sep 25, 2014

dduportal commented Sep 26, 2014

dduportal commented Sep 28, 2014

benjimcc commented Oct 9, 2014

mindjiver commented Oct 10, 2014

mindjiver commented Oct 11, 2014

Initial support for a Cloudstack builder #922

Initial support for a Cloudstack builder #922

Conversation

mindjiver commented Feb 28, 2014

mindjiver commented Feb 28, 2014

mindjiver Feb 28, 2014

Choose a reason for hiding this comment

mitchellh commented Apr 22, 2014

vogxn commented Apr 22, 2014

mindjiver commented Apr 22, 2014

fifthecho commented Apr 22, 2014

mitchellh commented Apr 22, 2014

retrack commented Apr 22, 2014

ddoc commented Apr 24, 2014

jwthomp commented May 5, 2014

mindjiver commented May 20, 2014

lndbrg commented May 22, 2014

C-Duv commented Jun 22, 2014

rickard-von-essen commented Jun 23, 2014

mindjiver commented Jun 23, 2014

C-Duv commented Jun 23, 2014

mindjiver commented Jun 23, 2014

C-Duv commented Jun 23, 2014

abayer commented Jul 18, 2014

mindjiver commented Jul 20, 2014

dduportal commented Aug 6, 2014

dduportal commented Aug 22, 2014

mindjiver commented Sep 4, 2014

abayer commented Sep 24, 2014

chobbs- commented Sep 25, 2014

mindjiver commented Sep 25, 2014

dduportal commented Sep 26, 2014

dduportal commented Sep 28, 2014

benjimcc commented Oct 9, 2014

mindjiver commented Oct 10, 2014

mindjiver commented Oct 11, 2014