D03c_KVM - Go! VM.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# About: VM - Go! with prepared VM image\n",
    "\n",
    "---\n",
    "\n",
    "Start VM instance with prepared VM image using livbirt.\n",
    "\n",
    "libvirtがインストールされている仮想化基盤上で、VMを起動するためのNotebook。\n",
    "\n",
    "すでに**VMイメージ作成Notebook**により、イメージが作成されているものとする。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## *Operation Note*\n",
    "\n",
    "*This is a cell for your own recording.  ここに経緯を記述*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Notebookと環境のBinding\n",
    "\n",
    "Inventory中のgroup名でBind対象を指示する。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2016-04-26T08:07:17.170145",
     "start_time": "2016-04-26T08:07:17.162322"
    },
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "target_group = 'test-hypervisor'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Bind対象への疎通状態を確認する。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2016-04-26T08:07:19.352910",
     "start_time": "2016-04-26T08:07:18.027874"
    },
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[0;32mXXX.XXX.XXX.105 | SUCCESS => {\r\n",
      "    \"changed\": false, \r\n",
      "    \"ping\": \"pong\"\r\n",
      "}\u001b[0m\r\n"
     ]
    }
   ],
   "source": [
    "!ansible -m ping {target_group}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "対象マシンにlibvirtがインストールされているかを確認する。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[0;32mXXX.XXX.XXX.105 | SUCCESS | rc=0 >>\r\n",
      "Compiled against library: libvirt 0.10.2\r\n",
      "Using library: libvirt 0.10.2\r\n",
      "Using API: QEMU 0.10.2\r\n",
      "Running hypervisor: QEMU 0.12.1\r\n",
      "\u001b[0m\r\n"
     ]
    }
   ],
   "source": [
    "!ansible -b -a 'virsh version' {target_group}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# VMイメージの指定\n",
    "\n",
    "作成対象のVMのあるディレクトリを指定する。**VMイメージ作成Notebook**により生成されたイメージが格納されているディレクトリを指定すること。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "image_base_dir = '/mnt/ubuntu14.04-base-vm'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "以下の2つのファイルが存在している必要がある。\n",
    "\n",
    "- base.img\n",
    "- libvirt-base.xml"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[0;32mXXX.XXX.XXX.105 | SUCCESS | rc=0 >>\r\n",
      "total 3853356\r\n",
      "drwxr-xr-x 2 root root         4096 Jun 17 19:36 .\r\n",
      "drwxr-xr-x 6 root root         4096 Jun 17 19:16 ..\r\n",
      "-rwxr-xr-x 1 root root 107374182400 Jun 17 19:36 base.img\r\n",
      "-rw-r--r-- 1 root root         2461 Jun 17 19:36 libvirt-base.xml\r\n",
      "\u001b[0m\r\n"
     ]
    }
   ],
   "source": [
    "!ansible -b -a \"ls -la {image_base_dir}\" {target_group}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "作成するVM名のリストを指定する。お手本では例として2つのVMを指定している。\n",
    "\n",
    "起動したいVM名をlistで指定すること。**既存のVMと重複してはならない。**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['testvm-001']"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "vm_names = ['testvm-001']\n",
    "vm_names"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# VMの作成\n",
    "\n",
    "VM用のファイルは以下のように作成される。\n",
    "\n",
    "- /mnt\n",
    "  - (VM名).xml ... libvirtに与えるXML定義\n",
    "  - (VM名).img ... VM用の仮想ディスク"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## XML定義の生成\n",
    "\n",
    "基本となるXML定義を得る。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'/tmp/tmp1c6xt5'"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import tempfile\n",
    "work_dir = tempfile.mkdtemp()\n",
    "work_dir"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[0;33mXXX.XXX.XXX.105 | SUCCESS => {\r\n",
      "    \"changed\": true, \r\n",
      "    \"checksum\": \"544dfe399f5626ed3b316b9d3b0d3041fbc0b922\", \r\n",
      "    \"dest\": \"/tmp/tmp1c6xt5/libvirt-base.xml\", \r\n",
      "    \"md5sum\": \"6d84ff7f9786c472caa07f0ecdd3d74e\", \r\n",
      "    \"remote_checksum\": \"544dfe399f5626ed3b316b9d3b0d3041fbc0b922\", \r\n",
      "    \"remote_md5sum\": null\r\n",
      "}\u001b[0m\r\n"
     ]
    }
   ],
   "source": [
    "!ansible -b -m fetch -a 'src={image_base_dir}/libvirt-base.xml dest={work_dir}/libvirt-base.xml flat=yes' {target_group}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "基本のXML定義に基づいて、VM用定義を生成する。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "-rw-r--r-- 1 root root 2461 Jun 17 22:36 /tmp/tmp1c6xt5/libvirt-base.xml\r\n",
      "-rw-r--r-- 1 root root 2512 Jun 17 22:36 /tmp/tmp1c6xt5/testvm-001.xml\r\n"
     ]
    }
   ],
   "source": [
    "import xml.etree.ElementTree as ET\n",
    "import virtinst.util\n",
    "import os\n",
    "\n",
    "for n in vm_names:\n",
    "    vmxml = ET.parse(os.path.join(work_dir, 'libvirt-base.xml')).getroot()\n",
    "    vmxml.find('name').text = n\n",
    "    vmxml.find('devices').find('disk').find('source').attrib['file'] = os.path.join('/mnt', n + '.img')\n",
    "    vmxml.find('devices').find('interface').find('mac').attrib['address'] = virtinst.util.randomMAC()\n",
    "    ET.ElementTree(vmxml).write(os.path.join(work_dir, n + '.xml'))\n",
    "!ls -la {work_dir}/*.xml"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "ホストに定義ファイルをコピーする。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[0;33mXXX.XXX.XXX.105 | SUCCESS => {\r\n",
      "    \"changed\": true, \r\n",
      "    \"checksum\": \"aff0775772be017583b5155fe38993ea0a46cd9f\", \r\n",
      "    \"dest\": \"/mnt/testvm-001.xml\", \r\n",
      "    \"gid\": 0, \r\n",
      "    \"group\": \"root\", \r\n",
      "    \"md5sum\": \"01081a332cce845696c84fd1d12fde21\", \r\n",
      "    \"mode\": \"0644\", \r\n",
      "    \"owner\": \"root\", \r\n",
      "    \"size\": 2512, \r\n",
      "    \"src\": \"/home/ansible/.ansible/tmp/ansible-tmp-1466170607.85-174997417957451/source\", \r\n",
      "    \"state\": \"file\", \r\n",
      "    \"uid\": 0\r\n",
      "}\u001b[0m\r\n"
     ]
    }
   ],
   "source": [
    "for n in vm_names:\n",
    "    !ansible -b -m copy -a 'src={work_dir}/{n}.xml dest=/mnt/{n}.xml' {target_group}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## イメージファイルのコピー\n",
    "\n",
    "イメージファイルをVM用に複製する。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[0;32mXXX.XXX.XXX.105 | SUCCESS | rc=0 >>\r\n",
      "\r\n",
      "\u001b[0m\r\n"
     ]
    }
   ],
   "source": [
    "for n in vm_names:\n",
    "    !ansible -b -a 'cp {image_base_dir}/base.img /mnt/{n}.img' {target_group}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## VMの起動\n",
    "\n",
    "XMLファイル、仮想ディスクファイルがあるかどうかを確認する。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[0;32mXXX.XXX.XXX.105 | SUCCESS | rc=0 >>\r\n",
      "-rwxr-xr-x 1 root root 107374182400 Jun 17 22:38 /mnt/testvm-001.img\r\n",
      "-rw-r--r-- 1 root root         2512 Jun 17 22:36 /mnt/testvm-001.xml\r\n",
      "\u001b[0m\r\n"
     ]
    }
   ],
   "source": [
    "for n in vm_names:\n",
    "    !ansible -a 'ls -la /mnt/{n}.img /mnt/{n}.xml' {target_group}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "VMを起動する。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[0;32mXXX.XXX.XXX.105 | SUCCESS | rc=0 >>\r\n",
      "Domain testvm-001 created from /mnt/testvm-001.xml\r\n",
      "\u001b[0m\r\n"
     ]
    }
   ],
   "source": [
    "import time\n",
    "for n in vm_names:\n",
    "    !ansible -b -a 'virsh create /mnt/{n}.xml' {target_group}\n",
    "    time.sleep(60)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "VMに設定されたIPアドレスを確認する。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import re\n",
    "def get_mac_address(vmname):\n",
    "    domiflist_stdio = !ansible -b -a \"virsh domiflist {vmname}\" {target_group}\n",
    "    mac_pattern = re.compile(r'.*bridge.*\\s([0-9a-f\\:]+)\\s*')\n",
    "    vmmac = [mac_pattern.match(line).group(1) for line in domiflist_stdio if mac_pattern.match(line)][0]\n",
    "    return vmmac\n",
    "\n",
    "def get_ip_address(vmmac):\n",
    "    leases_stdio = !ansible -b -a \"grep {vmmac} /var/lib/dnsmasq/dnsmasq.leases\" {target_group}\n",
    "    ip_pattern = re.compile(r'.*\\s([0-9a-f\\:]+)\\s+([0-9\\.]+)\\s.*')\n",
    "    ipaddr = [ip_pattern.match(line).group(2) for line in leases_stdio if ip_pattern.match(line)][0]\n",
    "    return ipaddr"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[('testvm-001', 'XXX.XXX.XXX.66')]"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "vmdescs = zip(vm_names, map(lambda mac: get_ip_address(mac), map(lambda n: get_mac_address(n), vm_names)))\n",
    "vmdescs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Inventoryの更新\n",
    "\n",
    "Inventoryに、作成したマシンのIPアドレスを追加する。変更する前に、現在の内容をコピーしておく。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "!cp inventory {work_dir}/inventory-old"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[Inventory](../edit/inventory) を修正する。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--- /tmp/tmp1c6xt5/inventory-old\t2016-06-17 22:39:22.008979291 +0900\r\n",
      "+++ inventory\t2016-06-17 22:40:16.505428369 +0900\r\n",
      "@@ -1,2 +1,5 @@\r\n",
      " [test-hypervisor]\r\n",
      " XXX.XXX.XXX.105\r\n",
      "+\r\n",
      "+[test-vm]\r\n",
      "+XXX.XXX.XXX.66\r\n"
     ]
    }
   ],
   "source": [
    "!diff -ur {work_dir}/inventory-old inventory"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "追加したグループ名でpingが通じるかどうかを確認する。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "target_vmgroup = 'test-vm'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[0;32mXXX.XXX.XXX.66 | SUCCESS => {\r\n",
      "    \"changed\": false, \r\n",
      "    \"ping\": \"pong\"\r\n",
      "}\u001b[0m\r\n"
     ]
    }
   ],
   "source": [
    "!ansible -m ping {target_vmgroup}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "# 後始末\n",
    "\n",
    "一時ディレクトリを削除する。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2016-04-26T08:14:39.273093",
     "start_time": "2016-04-26T08:14:39.103511"
    },
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "!rm -fr {work_dir}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.12"
  },
  "toc": {
   "toc_cell": false,
   "toc_number_sections": true,
   "toc_threshold": 6,
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}