MonascaBootcamp.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Monasca Bootcamp\n",
    "## Hands on Lab\n",
    "\n",
    "https://github.com/witekest/monasca-bootcamp\n",
    "\n",
    "---\n",
    "### Witek Bedyk\n",
    "\n",
    "company: Fujitsu EST\n",
    "\n",
    "email: <witold.bedyk@est.fujitsu.com>\n",
    "\n",
    "### Stefano Canepa\n",
    "\n",
    "company: Hewlett Packard Enterprise\n",
    "\n",
    "email: <stefano.canepa@hpe.com>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Introduction\n",
    "\n",
    "In this workshop we'll take you on a journey through Monasca features. We'll use a Jupyter notebook as a guide to not get lost in all the microservices that build Monasca solution. A Jupyter notebook is runnig on each VM and you can check it out from github and try on your own in conjunction with monasca-docker.\n",
    "\n",
    "The required softwares to replay this workshop on you computer are:\n",
    "* Jupyter (http://jupyter.org/)\n",
    "\n",
    "* python-monascaclient (https://github.com/openstack/python-monascaclient)\n",
    "\n",
    "* monasca-docker (https://github.com/monasca/monasca-docker)\n",
    "\n",
    "* monasca-agent (https://github.com/openstack/monasca-agent)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Agenda\n",
    "---\n",
    "* Architecture and Overview\n",
    "* API, CLI (python-monascaclient) and client hands-on\n",
    "* Agent hands-on\n",
    "* Horizon, Grafana, Kibana hands-on"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Architecture\n",
    "---\n",
    "![Monasca Architecture](architecture.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "# Logging\n",
    "---\n",
    "![Logging Architecture](Logging_Architecture.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "# Download and deploy Monasca server\n",
    "---\n",
    "Use *monasca-docker* repository to download and install Monasca environment."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "%cd\n",
    "!git clone https://github.com/monasca/monasca-docker.git\n",
    "%pushd monasca-docker\n",
    "!docker-compose up -d\n",
    "%popd"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "# Check services"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "%pushd monasca-docker\n",
    "!docker-compose ps\n",
    "%popd"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Import libraries and initialization\n",
    "---\n",
    "Let's first import some libraries used in the rest of the notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [],
   "source": [
    "import datetime\n",
    "import json\n",
    "import os\n",
    "import time\n",
    "\n",
    "# Import the Monasca client\n",
    "from monascaclient import client"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "# Initialize the Keystone and Monasca Client\n",
    "---\n",
    "We'll be connecting to the DevStack VM and using the `mini-mon` project and username."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [],
   "source": [
    "KEYSTONE_URL = 'http://keystone:5000/v3'\n",
    "PROJECT_NAME = 'mini-mon'\n",
    "USERNAME = 'mini-mon'\n",
    "PASSWORD = 'password'\n",
    "MON_API_URL = 'http://monasca:8070/v2.0'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [],
   "source": [
    "# Initialize environment variables to use the Monasca CLI\n",
    "%env OS_PROJECT_NAME=$PROJECT_NAME\n",
    "%env OS_PASSWORD=$PASSWORD\n",
    "%env OS_AUTH_URL=$KEYSTONE_URL\n",
    "%env OS_USERNAME=$USERNAME\n",
    "%env OS_IDENTITY_API_VERSION=3\n",
    "%env MONASCA_API_URL=$MON_API_URL\n",
    "\n",
    "# Create the Monasca client\n",
    "monasca_client = client.Client(api_version='2_0',\n",
    "                      username=os.environ.get('OS_USERNAME'),\n",
    "                      password=os.environ.get('OS_PASSWORD'),\n",
    "                      auth_url=os.environ.get('OS_AUTH_URL'),\n",
    "                      project_name=os.environ.get('OS_PROJECT_NAME'),\n",
    "                      user_domain_name=\"Default\",\n",
    "                      project_domain_name=\"Default\",\n",
    "                      auth_version=os.environ.get('OS_IDENTITY_API_VERSION'),\n",
    "                      endpoint='http://monasca:8070/v2.0')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Using the API\n",
    "---\n",
    "\n",
    "The [Monasca API](https://github.com/openstack/monasca-api/blob/master/docs/monasca-api-spec.md) has the following resources:\n",
    "\n",
    "* Versions\n",
    "\n",
    "* Metrics\n",
    "\n",
    "* Metrics Measurements\n",
    "\n",
    "* Metrics Statistics\n",
    "\n",
    "* Metrics Names\n",
    "\n",
    "* Notification Methods\n",
    "\n",
    "* Alarm Definitions\n",
    "\n",
    "* Alarms\n",
    "\n",
    "* Alarms Count\n",
    "\n",
    "* Alarms State History\n",
    "\n",
    "Please consult the [spec](https://github.com/openstack/monasca-api/blob/master/docs/monasca-api-spec.md) for the full details."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Metrics\n",
    "---\n",
    "* POST /v2.0/metrics\n",
    "\n",
    "* name (string(255), required) - The name of the metric. Naming conventions for metric names:\n",
    "\n",
    "    * lowercase, `.` to delimit groups, `_` (snake case) to delimit words, with a unit of measurement often as the suffix.\n",
    "    \n",
    "    * e.g. cpu.user_perc, kafka.consumer_lag\n",
    "\n",
    "* dimensions ({string(255): string(255)}, optional) - A dictionary consisting of (key, value) pairs used to uniquely identify a metric and slice and dice on.\n",
    "\n",
    "* timestamp (string, required) - The timestamp in milliseconds from the Epoch.\n",
    "\n",
    "* value (float, required) - Value of the metric.\n",
    "\n",
    "* value_meta ({string(255): string(2048)}, optional) - A dictionary consisting of (key, value) pairs used to describe the metric.\n",
    "\n",
    "    * Examples: status_code, msg\n",
    "\n",
    "* tenant_id: Tenant ID to create metrics on behalf of.\n",
    "\n",
    "    * This parameter can be used to submit metrics from one tenant, to another.\n",
    "    * Requires the delegate role."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "# Dimensions\n",
    "---\n",
    "\n",
    "* A dictionary of (key, value) pairs that are used to uniquely identify a metric.\n",
    "\n",
    "* Used to slice and dice metrics when querying.\n",
    "\n",
    "* Examples: hostname, service, component, region zone, resource_id, ...\n",
    "\n",
    "* Dimensions can be anything you want, but naming conventions should be adopted for consistency.\n",
    "\n",
    "* Examples of dimension keys are the following: hostname, region, zone, service, component, process, ..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "# Example: Metrics request body\n",
    "---\n",
    "In this example, we are reporting the status of a HTTP check as a binary value, 0 or 1, for a specific host. The API is down, and the status code and msg returned are returned as meta data.\n",
    "```\n",
    "{\n",
    "\tname: http_status,\n",
    "\tdimensions: {\n",
    "\t\turl: http://service.domain.com:80,\n",
    "\t\tregion: uswest,\n",
    "\t\tzone: 1,\n",
    "\t\tservice: compute\n",
    "\t}\n",
    "\ttimestamp: 1461600900000, /* milliseconds */\n",
    "\tvalue: 1.0,\n",
    "\tvalue_meta: {\n",
    "\t\tstatus_code: 500,\n",
    "\t\tmsg: Internal server error\n",
    "\t}\n",
    "}\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "# Help: Create metrics\n",
    "---"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "scrolled": false,
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [],
   "source": [
    "!monasca help metric-create"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "# Example: Create a metric\n",
    "Let's create a metric with the metric `openstack.handson_status` and some dimensions "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "!monasca metric-create --dimensions region=NSW,county=Cumberland,city=Sydney,session=monasca openstack.handson_status 1.0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "# List metrics\n",
    "Lists all the unique metrics in the system\n",
    "\n",
    "* GET /v2.0/metrics\n",
    "\n",
    "* A unique metric is identitifed by it's name and dimensions.\n",
    "\n",
    "* Starttime, endtime, offset and limit paramaters are available.\n",
    "\n",
    "* Note, if you are monitoring OpenStack resources, such as VMs, based on the amount of churn (VMs being created/destroyed) in the system and your retention policy, it is recommended to specify the starttime, to limit the amount of metrics returned, such that queries return in a reasonable amount of time."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "# Help: List Metrics\n",
    "---\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "!monasca help metric-list"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "# Example: List metrics"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [],
   "source": [
    "!monasca metric-list --limit 10"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "# Exercise 1: List metrics and filter on name, dimensions\n",
    "---\n",
    "Extend the following command to filter the list and find the previously generated metric."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "!monasca metric-list # put your options here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "# Example: Create a function to get metrics using the Monasca client\n",
    "---\n",
    "Now let's try to create a function which returns the list of metrics using the Monasca Python client.\n",
    "We will use the previously instantiated 'monasca_client' object and its 'metrics.list' method.\n",
    "All the CLI options can be used as named arguments.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import json\n",
    "\n",
    "def get_metrics(names=[None], dimensions={}, limit=10):\n",
    "    metrics = []\n",
    "    for name in names:        \n",
    "        # Invoke the Monasca client\n",
    "        metrics = metrics + monasca_client.metrics.list(name=name, dimensions=dimensions, limit=limit)\n",
    "    return metrics\n",
    "\n",
    "metrics = get_metrics(['openstack.handson_status'])\n",
    "print json.dumps(metrics, indent=4)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Metric names\n",
    "* List all the metric names stored in the service\n",
    "\n",
    "* GET /v2.0/metrics/names\n",
    "\n",
    "* Query parameters: tenant_id, offset, limit and dimensions\n",
    "\n",
    "* Very useful for user-interfaces that need to display the names of all the metrics in the system.\n",
    "\n",
    "    * Querying all metrics and determining all the distinct metric names is too costly."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Help: List metric names"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "!monasca help metric-name-list"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Example: List metric names"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "!monasca metric-name-list --limit 20"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Metrics Measurements\n",
    "---\n",
    "Operations for returning measurements.\n",
    "\n",
    "* GET /v2.0/metrics/measurements\n",
    "\n",
    "* Name and dimensions supplied as query parameters\n",
    "\n",
    "* Startime, endtime, offset, limit and group_by supplied.\n",
    "\n",
    "* Note:\n",
    "\n",
    "    * If group_by is not specified, metrics must be fully qualified with name and dimensions."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "# Help: List Measurements\n",
    "---"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "!monasca help measurement-list"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "# Excercise 2: List measurement\n",
    "---\n",
    "Get the list of 5 last measurements with the metric name 'monasca.thread_count' for service 'monitoring' within the last 5 minutes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [],
   "source": [
    "!monasca # put your command here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "# Exercise 3: List measurements by dimension\n",
    "---\n",
    "Get the list of measurements with the metric name 'fs.usage_bytes' within last 2 minutes. Please compare with the metric-list and try out using 'group_by' option.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# put your command here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "# Exercise 4: Create a function to get measurements\n",
    "---\n",
    "Let's try to use client's method 'metrics.list_measurements' to get the list of measurements.\n",
    "Please use the output of the get_metrics() function defined in the previous example as the first argument of the new function. Remember that the metric is defined by its name and the unique set of dimensions (these are attributes of the metric).\n",
    "You can filter the measurements by name, dimensions and time range."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def get_measurements(metrics, start_time = None, end_time = None, limit=100):\n",
    "    measurements = []\n",
    "    \n",
    "    if start_time == None:\n",
    "        start_date = datetime.datetime.utcnow() - datetime.timedelta(seconds=3600)\n",
    "        start_time = start_date.strftime(\"%Y-%m-%dT%H:%M:%SZ\")\n",
    "\n",
    "    if end_time == None:\n",
    "        end_date = datetime.datetime.utcnow() - datetime.timedelta(seconds=0)\n",
    "        end_time = end_date.strftime(\"%Y-%m-%dT%H:%M:%SZ\")\n",
    "        \n",
    "    for metric in metrics:        \n",
    "        # Invoke the Monasca client\n",
    "        \n",
    "        measurements += # put your code here\n",
    "        \n",
    "    return measurements\n",
    "\n",
    "# TEST\n",
    "metrics = get_metrics(['cpu.user_time','cpu.system_time'])\n",
    "measurements = get_measurements(metrics, limit=2)\n",
    "assert measurements[0]['name'] == 'cpu.user_time'\n",
    "assert len(measurements[0]['measurements']) == 2\n",
    "assert measurements[1]['name'] == 'cpu.system_time'\n",
    "\n",
    "print json.dumps(measurements, indent=4)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Notification Methods\n",
    "---\n",
    "Specify the name, type and address to send a notification to.\n",
    "\n",
    "* GET, POST, PUT, DELETE, PATCH /v2.0/notification-methods\n",
    "\n",
    "* Notification methods are associated with actions in alarms and are invoked when alarm state transitions occur.\n",
    "\n",
    "* Supported notification methods are:\n",
    "\n",
    "  * Email\n",
    "  \n",
    "  * PagerDuty\n",
    "  \n",
    "  * Webhooks\n",
    "  \n",
    "* Plugins available for:\n",
    "\n",
    "  * Jira\n",
    "  \n",
    "  * HipChat\n",
    "  \n",
    "  * Slack\n",
    "  \n",
    "* Alarm grouping, silencing and inhibition [under development](https://review.openstack.org/434537)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "source": [
    "# Delete all existing notifications\n",
    "Let's first delete all existing notifications to prepare the service for creating new ones."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "notifications = monasca_client.notifications.list()\n",
    "\n",
    "for notification in notifications:\n",
    "    monasca_client.notifications.delete(\n",
    "        notification_id = notification['id']\n",
    "    )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "# Help: Create Notification Method\n",
    "---"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [],
   "source": [
    "!monasca help notification-create"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "# Example: Create an email notification method"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [],
   "source": [
    "!monasca notification-create \"Email Notification\" EMAIL john.doe@domain.com"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Example: Create HipChat notification method"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "!monasca notification-create \"HipChat Notification\" HIPCHAT \"https://monasca.hipchat.com/v2/room/3235013/notification?auth_token=8PtY96mOobVpGIuSucQZCJYLkRgC89OqECkRJprr\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "HipChat plugin is not enabled per default.\n",
    "Please add it to `NF_PLUGINS` environment variable in `docker-compose.yml` and recreate the container.\n",
    "Then go back to the previous cell and try to create the HipChat notification again."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "%pushd monasca-docker\n",
    "!docker-compose up -d monasca-notification\n",
    "%popd"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "# Help: List Notification Methods\n",
    "---"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [],
   "source": [
    "!monasca help notification-list"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "# Example: List Notification Methods"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [],
   "source": [
    "!monasca notification-list -j"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Alarm Definitions\n",
    "---\n",
    "Operations for creating, reading updating and deleting alarm definitions.\n",
    "\n",
    "* GET, POST /v2.0/alarm-definitions\n",
    "\n",
    "* GET, PUT, PATCH, DELETE /v2.0/alarm-definitions/{alarm-definition-id}\n",
    "\n",
    "* Alarm definitions are templates that are used to automatically create alarms based on matching metric names and dimensions\n",
    "\n",
    "    * The `match-by` paramater is used to match/group metrics together by dimension\n",
    "    \n",
    "    * e.g. `--match-by hostname` will create an alarm per unique hostname.\n",
    "\n",
    "* One alarm definition can result in many alarms\n",
    "\n",
    "* Simple grammar for creating compound alarm expressions:\n",
    "\n",
    "   avg(cpu.user_perc{}) > 85 or avg(disk.read_ops{device=vda}, 120) > 1000\n",
    "\n",
    "\n",
    "* Alarm state (ALARM, OK and UNDETERMINED)\n",
    "\n",
    "* Actions (notification methods) associated with alarms for state transitions\n",
    "\n",
    "* User assigned severity (LOW, MEDIUM, HIGH, CRITICAL)\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "source": [
    "# Delete existing alarm definitions\n",
    "Let's first delete all existing alarm definitions to prepare the service for creating new ones."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "alarm_definitions = monasca_client.alarm_definitions.list()\n",
    "\n",
    "for definition in alarm_definitions:\n",
    "    monasca_client.alarm_definitions.delete(alarm_id=definition['id'])    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "# Help: Create alarm definition"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "!monasca help alarm-definition-create"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "# Example: Create alarm definition\n",
    "Create an alarm definition that triggers if the maximum of CPU user percent is greater than 80% over 3 periods of 60 seconds."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "!monasca alarm-definition-create simple-alarm \"max(cpu.user_time_sec{}, 60) > 0.8 times 3\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "# Example: Create a compound alarm definition\n",
    "An alarm definition that triggers if either the CPU user or system percent is greater than 80% over 3 periods of 60 seconds."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "!monasca alarm-definition-create compound-alarm \"avg(cpu.user_time_sec{}, 60) > 0.8 times 3 or avg(cpu.system_time_sec{}, 60) > 0.8 times 3\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "# Alarm definition match_by\n",
    "\n",
    "* As metrics are consumed by the Threshold Engine they are filtered and checked if they match the definitions that have been specified, based on the metric name and dimensions.\n",
    "\n",
    "* If a match is found and it is the first one, a new alarm is created.\n",
    "\n",
    "* If the match_by parameter is specified, metrics are grouped into alarms that match the specified dimensions."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "# Example: Two alarm definitions\n",
    "1. One with no match_by parameter\n",
    "2. One with match_by set on hostname"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "!monasca alarm-definition-create \"match name only\" \"max(test-metric{}) > 80\"\n",
    "!monasca alarm-definition-create --match-by hostname \"match by hostname\" \"max(test-metric{}) > 80\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "# Example: Send two metrics\n",
    "1. One with a name of test-metric and a hostname set to foo\n",
    "2. One with a name of test-metric and a hostname set to bar"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "!monasca metric-create --dimensions hostname=foo test-metric 0.0\n",
    "!monasca metric-create --dimensions hostname=bar test-metric 0.0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "# Quiz: How many alarms were created?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "!monasca alarm-list --metric-name \"test-metric\" -j"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Three alarms were created:\n",
    "\n",
    "1. There is one alarm with name \"match name only\". It has two metrics that are associated with it.\n",
    "\n",
    "2. There are two alarms with name \"match by hostname\". Each alarm has one metric that is associated with it, based on the hostnames of `foo` and `bar`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "# UNDETERMINED alarm state\n",
    "* There are three states of an alarm, `OK`, `ALARM` and `UNDETERMINED`.\n",
    "\n",
    "* The `UNDETERMINED` alarm state occurs when metrics are no longer being received by an alarm.\n",
    "\n",
    "* For example, a service is disabled, a system goes down, the network experiences a disruption.\n",
    "\n",
    "* You can define the alarm as `deterministic`. The only allowed states are then: `OK` and `ALARM`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "# Example: Create an alarm definition with an alarm action."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Get the ID of the HipChat notification method\n",
    "notifications = monasca_client.notifications.list(sort_by='type')\n",
    "for notification in notifications:\n",
    "    if notification['type'] == 'HIPCHAT':\n",
    "        hipchat_notification = notification   \n",
    "        hipchat_notification_id = hipchat_notification['id']\n",
    "\n",
    "# Create the alarm\n",
    "monasca_client.alarm_definitions.create(name='bootcamp-test',\n",
    "                                        expression='max(cpu.user_time_sec) > 1',\n",
    "                                        alarm_actions=[hipchat_notification_id])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Stress the system and wait for notification\n",
    "---\n",
    "* Open [HipChat room](https://www.hipchat.com/gxUiEZgyf)\n",
    "* Stress the system\n",
    "* Wait for your notification"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "!stress-ng -c 2 -i 1 -m 1 --vm-bytes 128M -t 60s"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Alarms\n",
    "---\n",
    "Alarms are created when incoming metrics match alarm definitions\n",
    "\n",
    "\n",
    "* GET /v2.0/alarms\n",
    "* GET, PUT, PATCH, DELETE /v2.0/alarms/{alarm-id}\n",
    "\n",
    "* Query Parameters\n",
    "\n",
    "    * alarm_definition_id (string, optional) - Alarm definition ID to filter by.\n",
    "\n",
    "    * metric_name (string(255), optional) - Name of metric to filter by.\n",
    "\n",
    "    * metric_dimensions ({string(255): string(255)}, optional) - Dimensions of metrics to filter by specified as a comma separated array of (key, value) pairs as `key1:value1,key1:value1, ...`\n",
    "\n",
    "    * state (string, optional) - State of alarm to filter by, either `OK`, `ALARM` or `UNDETERMINED`.\n",
    "\n",
    "    * state_updated_start_time (string, optional) - The start time in ISO 8601 combined date and time format in UTC.\n",
    "    \n",
    "    * sort-by: Fields to sort by\n",
    "    \n",
    "    * offset, limit\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "# Help: Alarms list"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [],
   "source": [
    "!monasca help alarm-list"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "# Example: Alarm list"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "!monasca alarm-list --sort-by severity -j"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Alarm Counts\n",
    "---\n",
    "Provides the ability to query counts of alarms using a number of filter and group-by query parameters.\n",
    "\n",
    "* GET /v2.0/alarms/counts\n",
    "\n",
    "* Primarily used in summary/overview dashboards to show the number of alarms in the OK, ALARM and UNDETERMINED state.\n",
    "\n",
    "* Queries are processed in-database, not client-side.\n",
    "\n",
    "* A number of filter and group-by fields are supported."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "# Help: Alarm counts"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [],
   "source": [
    "!monasca help alarm-count"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "# Example: Alarm counts"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [],
   "source": [
    "!monasca alarm-count --metric-dimensions service=monitoring --group-by state,dimension_name"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Metrics Agent\n",
    "---\n",
    "* A Python monitoring agent\n",
    "* Push model\n",
    "* Agent is installed on the systems that we want to monitor\n",
    "* Collects metrics by running a set of collection plugins every X amount of seconds \n",
    "* Collection plugins are enabled by detection plugins\n",
    "* Detection plugins generate yaml config files that the collection plugins read from\n",
    "* The agent has a monasca-setup command line tool that helps configure the agent and run detection plugins"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "# Agent Metrics and Plugins\n",
    "\n",
    "* System metrics (cpu, memory, network, filesystem, …)\n",
    "\n",
    "* Service metrics\n",
    "\n",
    "* RabbitMQ, MySQL, Kafka, and many others\n",
    "\n",
    "* Application metrics\n",
    "\n",
    "    * Built-in statsd daemon\n",
    "\n",
    "    * Python monasca-statsd library: Adds support for dimensions\n",
    "    \n",
    "    * Prometheus metrics\n",
    "\n",
    "* VM system metrics\n",
    "\n",
    "* Kubernetes metrics\n",
    "\n",
    "* Active checks\n",
    "\n",
    "    * HTTP status checks and response times\n",
    "    \n",
    "    * System up/down checks (ping and ssh)\n",
    "    \n",
    "* Support for Nagios and checkmk\n",
    "\n",
    "* Extensible/Pluggable: Additional services can be easily added\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "# Monasca-setup options"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "!/opt/monasca-agent/bin/monasca-setup -h"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "!sudo /opt/monasca-agent/bin/monasca-setup \\\n",
    "    --username monasca-agent \\\n",
    "    --password $PASSWORD \\\n",
    "    --project_name $PROJECT_NAME \\\n",
    "    --keystone_url $KEYSTONE_URL \\\n",
    "    --project_domain_name Default \\\n",
    "    --user_domain_name Default \\\n",
    "    --system_only"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "# Monasca agent configuration file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "!sudo cat /etc/monasca/agent/agent.yaml"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "# Agent Detection Plugins\n",
    "\n",
    "* Run after initial configuration is run\n",
    "* [List of avaiable plugins](https://github.com/openstack/monasca-agent/blob/master/docs/Plugins.md)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "# Run httpcheck Detection Plugin"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "!sudo  /opt/monasca-agent/bin/monasca-setup -d httpcheck -a 'url=http://keystone:5000/'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "# Example: Detection yaml configuration"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [],
   "source": [
    "!sudo cat /etc/monasca/agent/conf.d/http_check.yaml"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "# Example: Query the HTTP response time metrics"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [],
   "source": [
    "!monasca metric-list --name http_response_time"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "# Example: Query the HTTP response time measurements"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [],
   "source": [
    "!monasca measurement-list http_response_time -10 --limit 10"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Enable logging pipeline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "%pushd monasca-docker\n",
    "!docker-compose -f docker-compose.yml -f log-pipeline.yml up -d\n",
    "%popd"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Logs Agent\n",
    "---\n",
    "* Based on Logstash (alternatively Beaver)\n",
    "\n",
    "* Uses plugins\n",
    "\n",
    "    * file for input\n",
    "    \n",
    "    * [monasca-log-api](https://github.com/logstash-plugins/logstash-output-monasca_log_api) for output"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "# Example: Examine the configuration of log agent"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [],
   "source": [
    "!cat /etc/monasca/log-agent.conf"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "# Example: Collect logs of metric agent\n",
    "---\n",
    "* Edit the configuration of log-agent\n",
    "\n",
    "    `sudo nano /etc/monasca/log-agent.conf`\n",
    "    \n",
    "* Add the following block to the input plugin configuration\n",
    "    ```\n",
    "    file {\n",
    "        add_field => { \"dimensions\" => { \"service\" => \"monasca\" }}\n",
    "        add_field => { \"dimensions\" => { \"component\" => \"monasca-agent\" }}\n",
    "        path => \"/var/log/monasca/agent/*\"\n",
    "    }\n",
    "    ```\n",
    "    \n",
    "* Save the new configuration    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "# Test the syntax of new configuration"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [],
   "source": [
    "!/opt/logstash/bin/logstash -f /etc/monasca/log-agent.conf --configtest"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "# Restart the service"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [],
   "source": [
    "!sudo systemctl start monasca-log-agent"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Horizon, Grafana and Kibana Demo\n",
    "---\n",
    "* Log in to the Horizon dashbord \n",
    "    \n",
    "    * URL: `http://<your_instance_ip>`\n",
    "    \n",
    "    * User: mini-mon\n",
    "    \n",
    "    * Password: password"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Monitoring Dashboard\n",
    "---\n",
    "* Explore the panels\n",
    "\n",
    "    * Overview\n",
    "    * Alarm Definitions\n",
    "    * Alarms\n",
    "    * Notifications"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Grafana 4\n",
    "---\n",
    "* Log in to Grafana\n",
    "\n",
    "    * URL: `https://<your_instance_ip>:3000`\n",
    "\n",
    "    * User: mini-mon\n",
    "    \n",
    "    * Password: password"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Kibana\n",
    "---\n",
    "Kibana multi-tenancy plugin is not enabled per default.\n",
    "Edit `log-pipeline.yml` file in `monasca-docker` directory and add the following environment variables for *kibana* container:\n",
    "\n",
    "```\n",
    "MONASCA_PLUGIN_ENABLED: \"True\"\n",
    "BASE_PATH: \"/dashboard/monitoring/logs_proxy\"\n",
    "```\n",
    "\n",
    "Then recreate the *kibana* container."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "%pushd monasca-docker\n",
    "!docker-compose -f docker-compose.yml -f log-pipeline.yml up -d kibana\n",
    "%popd"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Open Log Management from Monitoring Overview\n",
    "* Monasca Kibana plugin enhancements:\n",
    "    * OpenStack user role based authorization\n",
    "    * Set default index pattern based on the current project\n",
    "    * Filter queries to authorized project only\n",
    "* Explore the panels:\n",
    "    * Discover\n",
    "    * Visualize\n",
    "    * Dashboard"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Contact the Monasca Team\n",
    "---\n",
    "Here are a couple of ways of getting in-touch with us.\n",
    "\n",
    "* Monasca Weekly Meetings\n",
    "\n",
    "    * Weekly on Wednesday at 1400 UTC in #openstack-meeting\n",
    "    \n",
    "* Monasca in IRC\n",
    "\n",
    "    * #openstack-monasca"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Thank you"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "source": [
    "# Solution ex. 1:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "!monasca metric-list --name openstack.handson_status --dimensions 'city=Sydney'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Solution ex. 2:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "!monasca measurement-list --dimensions 'service=monitoring' --limit 5 monasca.thread_count -5   "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Solution ex. 3:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "!monasca metric-list --name fs.usage_bytes\n",
    "!monasca measurement-list --group_by device fs.usage_bytes -2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "source": [
    "# Solution ex. 4:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "measurements += monasca_client.metrics.list_measurements(\n",
    "                name=metric['name'],\n",
    "                dimensions=metric['dimensions'],\n",
    "                start_time=start_time,\n",
    "                end_time=end_time,\n",
    "                limit=limit)"
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Slideshow",
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}