Permalink
Browse files

Initial commit.

Not yet in a particularly interesting state, but the basic
building blocks of with-config and lift-one-node-and-phase
is there.
  • Loading branch information...
sundbp committed Jan 7, 2013
0 parents commit 47b9565de596ac202a8d118ffca59d53c8329b86
Showing with 345 additions and 0 deletions.
  1. +11 −0 .gitignore
  2. +98 −0 README.md
  3. +3 −0 doc/intro.md
  4. +10 −0 download-custom-debs.sh
  5. +23 −0 project.clj
  6. +109 −0 src/kvm_crate/api.clj
  7. +53 −0 src/kvm_crate/crate.clj
  8. +31 −0 src/kvm_crate/specs.clj
  9. +7 −0 test/kvm_crate/api_test.clj
@@ -0,0 +1,11 @@
+/target
+/lib
+/classes
+/checkouts
+/resources/custom-debs
+pom.xml
+*.jar
+*.class
+.lein-deps-sum
+.lein-failures
+.lein-plugins
@@ -0,0 +1,98 @@
+# kvm-crate
+
+A Pallet crate to work with KVM servers and KVM guests.
+
+There is no KVM backend for jclouds/pallet and this is a first attempt
+at "manually" supporting KVM. By manually I mean that all operations
+are done via the node-list and lift (converge is never used).
+
+Hence you define your servers and guests in a config map (the format is
+described further down), then perform specific phases on the KVM server(s) to:
+* Setup an already existing host as a KVM server
+* Create a KVM guest VM on a given KVM server
+* Create one big VLAN spanning several KVM servers
+
+After that you can of course perform operations on the KVM guest VMs
+directly without going via the KVM server.
+
+For the moment kvm-crate assumes KVM servers are Ubuntu 12.04 LTS hosts. This
+restriction can loosened in the future if others provide variations that
+works on other distributions and versions.
+
+kvm-crate utilizes the following for setting up VMs and networking:
+* python-vm-builder to create VM images from scratch (if not using base images)
+* libvirt for managing guests
+* openvswitch for networking (and GRE+IPSec for connecting OVS on several KVM servers)
+
+As it stands the libvirt shipped with Ubuntu 12.04 does not come with
+support for openvswitch. Hence I've prepared some custom libvirt 1.0
+packages that are used instead (prepared from the Ubuntu raring sources).
+
+## Configuring a KVM server
+
+First all make sure the following things hold true:
+1. Your intended KVM server host exists and runs Ubuntu 12.04 LTS
+2. The host is included in your hosts config map
+3. You have downloaded the custom libvirt deps to "custom-debs/" somewhere on the classpath
+
+Step 2 and 3 will be described in more detail shortly, let's first see how
+the call looks (assuming you are use'ing the kvm-create.api namespace):
+ (with-config [hosts-config {}]
+ (configure-kvm-server "host.to.configure")
+
+The second argument to *with-config* is a map that will be passed as the :environment
+argument to any subsequent lift operation that takes place under the covers (and is
+hence available to any of your own pallet plan functions.
+
+The format of the hosts-config argument that kvm-crate looks for is this (it's ok to
+add additional content):
+ {"host.to.configure" {:host-type :kvm-server
+ :group-spec kvm-create.specs/kvm-server-g
+ :ip "1.2.3.4"
+ :admin-user {:username "root"
+ :ssh-public-key-path (utils/resource-path "ssh-keys/kvm-keys/kvm-id_rsa.pub")
+ :ssh-private-key-path (utils/resource-path "ssh-keys/kvm-keys/kvm-id_rsa")
+ :passphrase "foobar"}}}
+
+(The function *utils/resource-path* is from the namespace pallet.utils and
+is handy for referring to paths on the local machine)
+
+As for step 3 the custom debs are too big to distribute with kvm-create
+and are instead downloaded onto the machine pallet is run from, then
+transferred over the KVM server target for install. An example script
+of how and where to place the files can be found in the provided
+*download-custom-debs.sh* script.
+
+You can use *configure-kvm-server* in your own functions to create
+more complete functions that do more than just the KVM server
+configuration. For example this is how we setup newly ordered Hetzner
+servers as KVM servers:
+ (defn configure-hetzner-kvm-server
+ "Setup a KVM server in Hetzner data-center"
+ [hostname]
+ (kvm-api/with-config [hosts-config/config {}]
+ (hetzner-api/hetzner-initial-setup hostname)
+ (println (format "Sleep 3mins while we wait for KVM server %s to reboot.." hostname))
+ (Thread/sleep (* 3 60 1000)) ;; wait for the host to reboot
+ (kvm-api/configure-kvm-server hostname)))
+
+(the *hetzner-initial-setup* function performs actions needed to
+for a fresh Hetzner machine)
+
+## Creating a KVM guest VM
+
+FIXME
+
+## Connecting the openvswitches of several KVM servers
+
+To create one big VLAN where KVM guests on seveal KVM servers can
+communicate with each other we use GRE+IPSec to connect the
+openvswitches of the KVM servers.
+
+FIXME
+
+## License
+
+Copyright © 2013 Board Intelligence
+
+Distributed under the MIT License, see http://boardintelligence.mit-license.org for details.
@@ -0,0 +1,3 @@
+# Introduction to kvm-crate
+
+TODO: write [great documentation](http://jacobian.org/writing/great-documentation/what-to-write/)
@@ -0,0 +1,10 @@
+#!/bin/bash
+
+# get custom debs we need
+mkdir -p resources/custom-debs
+cd resources/custom-debs
+wget http://vagrant.vannavolga.info/boxes/libvirt0_1.0.0-0ubuntu4_amd64.deb
+wget http://vagrant.vannavolga.info/boxes/libvirt0-dbg_1.0.0-0ubuntu4_amd64.deb
+wget http://vagrant.vannavolga.info/boxes/libvirt-bin_1.0.0-0ubuntu4_amd64.deb
+wget http://vagrant.vannavolga.info/boxes/libvirt-dev_1.0.0-0ubuntu4_amd64.deb
+wget http://vagrant.vannavolga.info/boxes/libvirt-doc_1.0.0-0ubuntu4_all.deb
@@ -0,0 +1,23 @@
+(defproject boardintelligence/kvm-crate "0.1.0-SNAPSHOT"
+ :description "Pallet crate for working with KVM servers and guests"
+ :url "https://github.com/boardintelligence/kvm-crate"
+ :license {:name "MIT"
+ :url "http://boardintelligence.mit-license.org"}
+
+ :dependencies [[org.clojure/clojure "1.4.0"]
+ [org.cloudhoist/pallet "0.8.0-SNAPSHOT" :exclusions [useful]]
+ [ch.qos.logback/logback-classic "1.0.7"]
+ [clj-time "0.4.4"]]
+
+ :dev-dependencies [[org.cloudhoist/pallet "0.8.0-SNAPSHOT" :type "test-jar"]
+ [org.cloudhoist/pallet-lein "0.5.2"]]
+
+ :profiles {:dev
+ {:dependencies [[org.cloudhoist/pallet "0.8.0-SNAPSHOT" :classifier "tests"]]
+ :plugins [[org.cloudhoist/pallet-lein "0.5.2"]]}}
+
+ :local-repo-classpath true
+
+ :repositories
+ {"sonatype-snapshots" "https://oss.sonatype.org/content/repositories/snapshots"
+ "sonatype" "https://oss.sonatype.org/content/repositories/releases/"})
@@ -0,0 +1,109 @@
+(ns kvm-crate.api
+ (:require
+ [pallet.algo.fsmop :as fsmop]
+ [pallet.node :as node]
+ [pallet.configure :as configure]
+ [pallet.compute :as compute]
+ [pallet.api :as api]))
+
+(def ^{:dynamic true
+ :doc "Dynamic var epxected to be bound to a map of KVM hosts config.
+ The format of the map is described in the README file."}
+ *kvm-hosts-config* nil)
+
+(def ^{:dynamic true
+ :doc "Dynamic var epxected to be bound to a map to use as pallet env."}
+ *pallet-environment* nil)
+
+(def ^{:dynamic true
+ :doc "Dynamic var epxected to be bound to a computeservice."}
+ *compute-service* nil)
+
+(defn- node-list-info-for-host
+ "Returns a single host info vector"
+ [[hostname host-config]]
+ [hostname (:group-name (:group-spec host-config)) (:ip host-config) :ubuntu])
+
+(defn- generate-node-list
+ "Transform the host config hash into the format expected by pallet"
+ [config]
+ (vec (map node-list-info-for-host config)))
+
+(defn node-list-compute-service
+ "Create a node-list compute service based on KVM servers and guests."
+ [config]
+ (configure/compute-service "node-list" :node-list (generate-node-list config)))
+
+(defmacro with-config
+ "Utility function to wrap operations to use a particular KVM config."
+ [[hosts-config pallet-environment] & body]
+ `(binding [*kvm-hosts-config* ~hosts-config
+ *compute-service* (node-list-compute-service ~hosts-config)
+ *pallet-environment* ~pallet-environment]
+ ~@body))
+
+(defn ensure-bindings []
+ "Ensure the relevant bindings for using KVM api is in place."
+ (when-not (instance? clojure.lang.IPersistentMap *kvm-hosts-config*)
+ (throw (IllegalArgumentException. "*kvm-hosts-config* is not a map")))
+ (when-not (instance? clojure.lang.IPersistentMap *pallet-environment*)
+ (throw (IllegalArgumentException. "*pallet-environment* is not a map")))
+ (when (nil? *compute-service*)
+ (throw (IllegalArgumentException. "*compute-service* is nil"))))
+
+(defn- get-group-spec
+ [hostname]
+ (get-in *kvm-hosts-config* [hostname :group-spec]))
+
+(defn- node-for-hostname
+ [hostname]
+ (first (filter #(= (:name %) hostname) (compute/nodes *compute-service*))))
+
+(defn- get-admin-user
+ [hostname & {:keys [sudo-user]}]
+ (let [admin-username (get-in *kvm-hosts-config* [hostname :admin-user :username])
+ ssh-public-key-path (get-in *kvm-hosts-config* [hostname :admin-user :ssh-public-key-path])
+ ssh-private-key-path (get-in *kvm-hosts-config* [hostname :admin-user :ssh-private-key-path])
+ passphrase (.getBytes (get-in *kvm-hosts-config* [hostname :admin-user :passphrase]))]
+ (api/make-user admin-username
+ :public-key-path ssh-public-key-path
+ :private-key-path ssh-private-key-path
+ :passphrase passphrase
+ :sudo-user sudo-user)))
+
+(defn lift-one-node-and-phase
+ "Lift a given host, applying only one specified phase"
+ ([hostname phase] (lift-one-node-and-phase hostname (get-admin-user hostname) phase {}))
+ ([hostname phase env-options] (lift-one-node-and-phase hostname (get-admin-user hostname) phase env-options))
+ ([hostname user phase env-options]
+ (let [spec (get-group-spec hostname)
+ node (node-for-hostname hostname)
+ result (api/lift
+ {spec node}
+ :environment (merge *pallet-environment* env-options {:host-config *kvm-hosts-config*})
+ :phase phase
+ :user user
+ :compute *compute-service*)]
+ (fsmop/wait-for result)
+ (when (fsmop/failed? result)
+ (do
+ ;; TODO: use logger here
+ (println "Errors encountered:")
+ (fsmop/report-operation result)))
+ result)))
+
+(defn host-is-kvm-server?
+ "Check if a host is a KVM server (as understood by the kvm-create)"
+ [hostname]
+ (contains? (:phases (get-group-spec hostname)) :configure-kvm-server))
+
+(defn configure-kvm-server
+ "Set up a machine to act as a KVM server"
+ [hostname]
+ (println (format "Configuring KVM server for %s.." hostname))
+ (ensure-bindings)
+ (when-not (host-is-kvm-server? hostname)
+ (throw (IllegalArgumentException. (format "%s is not a kvm-server!" hostname))))
+ (when (fsmop/failed? (lift-one-node-and-phase hostname
+ :configure-kvm-server))
+ (throw (IllegalStateException. "Failed to configure KVM server!"))))
@@ -0,0 +1,53 @@
+(ns kvm-crate.crate
+ "Crate with functions for setting up and configuring KVM servers and guests"
+ (:require
+ [pallet.actions :as actions]
+ [pallet.crate :as crate]
+ [pallet.utils :as utils]
+ [pallet.crate.sudoers :as sudoers]
+ [pallet.crate.ssh-key :as ssh-key]
+ [pallet.environment :as env])
+ (:use [pallet.crate :only [def-plan-fn]]))
+
+(def-plan-fn install-deb
+ "Install a single deb by transferring local file and installing via dpkg"
+ [deb-name]
+ [tmp-path (m-result (str "/tmp/" deb-name))
+ local-file (m-result (utils/resource-path (str "custom-debs/" deb-name)))]
+ ;; transfer file and install it via dpkg
+ (actions/remote-file tmp-path :local-file local-file)
+ (actions/exec-checked-script
+ "install my custom debs (these will fail, but are triggered later)"
+ (dpkg -i ~tmp-path)
+ (exit 0)))
+
+(def-plan-fn install-custom-debs
+ "Install all needed custom debs for KVM server using openvswitch"
+ []
+ [custom-debs (m-result ["libvirt0_1.0.0-0ubuntu4_amd64.deb"
+ "libvirt0-dbg_1.0.0-0ubuntu4_amd64.deb"
+ "libvirt-bin_1.0.0-0ubuntu4_amd64.deb"
+ "libvirt-dev_1.0.0-0ubuntu4_amd64.deb"
+ "libvirt-doc_1.0.0-0ubuntu4_all.deb"])]
+ (m-result (map install-deb custom-debs)))
+
+(def-plan-fn configure-openvswitch
+ "Configure openvswtich for KVM server."
+ []
+ [])
+
+(def-plan-fn configure-server
+ "Install packages for KVM server and configure networking"
+ []
+ [node-hostname crate/target-name
+ host-config (env/get-environment [:host-config node-hostname])
+ admin-username (m-result (get-in host-config [:admin-user :username]))]
+
+ ;; install packages
+ (install-custom-debs)
+ (actions/packages :aptitude ["kvm" "ubuntu-virt-server" "python-vm-builder"
+ "openvswitch-controller" "openvswitch-brcompat"
+ "openvswitch-switch" "openvswitch-datapath-source"])
+
+ ;; setup openvswitch
+ (configure-openvswitch))
@@ -0,0 +1,31 @@
+(ns kvm-crate.specs
+ "Server and group specs for working with KVM servers and guests"
+ (:require
+ [pallet.api :as api]
+ [kvm-crate.crate :as kvm]))
+
+(def
+ ^{:doc "Server spec for a KVM server (host)"}
+ kvm-server
+ (api/server-spec
+ :phases
+ {:configure-kvm-server (api/plan-fn (kvm/configure-server))
+ ;;:create-guest-vm-user (api/plan-fn (kvm/create-guest-vm-user))
+ ;;:create-guest-vm (api/plan-fn (kvm/create-guest-vm))
+ ;;:create-guest-vm-upstart (api/plan-fn (kvm/create-guest-vm-upstart))
+ }))
+
+(def
+ ^{:doc "Group spec for a KVM server (host)"}
+ kvm-server-g
+ (api/group-spec
+ "kvm-server-g"
+ :extends [kvm-server]))
+
+(def
+ ^{:doc "Spec for a KVM guest server"}
+ kvm-guest-vm
+ (api/server-spec
+ :phases
+ {;;:firstboot (api/plan-fn (kvm/setup-guest-vm-firstboot))
+ }))
@@ -0,0 +1,7 @@
+(ns kvm-crate.api-test
+ (:use clojure.test
+ kvm-crate.api))
+
+(deftest a-test
+ (testing "FIXME, I fail."
+ (is (= 0 1))))

0 comments on commit 47b9565

Please sign in to comment.