api-examples/ambari-workshop-blueprints.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Deploying an HDP cluster with Ambari Blueprints\n",
    "\n",
    "Below we will make 3 HTTP requests using curl:\n",
    "\n",
    "1. Upload the Blueprint\n",
    "2. Create the Cluster\n",
    "3. Check Cluster creation status\n",
    "\n",
    "You can copy and paste to run directly from your terminal. Use this as an example for sending from your preferred tool or programming language.\n",
    "\n",
    "The ambari-api-examples ipython notebook _(in this same directory)_ has these examples and many others using Python.\n",
    "\n",
    "## Other related API URLs\n",
    "\n",
    "You can browse these directly in the browser as they are GET requests:\n",
    "\n",
    "  * List Blueprints: http://hostname:8080/api/v1/blueprints\n",
    "  * Lists Clusters: http://hostname:8080/api/v1/clusters\n",
    "  * Lists Hosts: http://hostname:8080/api/v1/hosts\n",
    "  * Export Cluster configuration as Blueprint: http://hostname:8080/api/v1/clusters/clustername?format=blueprint\n",
    "\n",
    "\n",
    "_Note: Please do not send passwords in the clear when working on a real cluster._"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1. Upload the Blueprint\n",
    "\n",
    "Explanation of the curl command:\n",
    "\n",
    "    -u admin:password\n",
    "      * sets the username and password.\n",
    "      * Alternatively, run `-u admin` to be prompted for the password.\n",
    "    -H X-Requested-By:seano\n",
    "      * Ambari requires the user to identify themselves.\n",
    "      & This can be set to anything you like (instead of 'seano')\n",
    "    -X POST -d @blueprint_hdp22_multinode-simple.blueprint.json\n",
    "      * Tells curl to submit the blueprint file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\n",
      "                                 Dload  Upload   Total   Spent    Left  Speed\n",
      "\r",
      "  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0\r",
      "  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0\r",
      "100  2429    0     0  100  2429      0   1520  0:00:01  0:00:01 --:--:--  1520\r",
      "100  2429    0     0  100  2429      0   1520  0:00:01  0:00:01 --:--:--  1520\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "\n",
    "## Upload the Blueprint\n",
    "\n",
    "ambari_server=pregion-sean01.cloud.hortonworks.com ## This is the hostname of the ambari-server.\n",
    "                       ##   - Set to localhost if running directly on the server.\n",
    "pass=admin        ## The Ambari admin password\n",
    "\n",
    "curl -u admin:${pass} \\\n",
    "  -H X-Requested-By:seano \\\n",
    "  -X POST -d @blueprints/hdp-201503_blueprint.json \\\n",
    "  http://${ambari_server}:8080/api/v1/blueprints/hdp-201503"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2. Create the Cluster"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      "  \"href\" : \"http://pregion-sean01.cloud.hortonworks.com:8080/api/v1/clusters/mycluster/requests/1\",\n",
      "  \"Requests\" : {\n",
      "    \"id\" : 1,\n",
      "    \"status\" : \"InProgress\"\n",
      "  }\n",
      "}"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\n",
      "                                 Dload  Upload   Total   Spent    Left  Speed\n",
      "\r",
      "  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0\r",
      "100   870    0     0  100   870      0    698  0:00:01  0:00:01 --:--:--   698\r",
      "100   870    0     0  100   870      0    386  0:00:02  0:00:02 --:--:--   387\r",
      "100   870    0     0  100   870      0    267  0:00:03  0:00:03 --:--:--   267\r",
      "100   870    0     0  100   870      0    204  0:00:04  0:00:04 --:--:--   204\r",
      "100   870    0     0  100   870      0    165  0:00:05  0:00:05 --:--:--   165\r",
      "100   870    0     0  100   870      0    139  0:00:06  0:00:06 --:--:--     0\r",
      "100   870    0     0  100   870      0    119  0:00:07  0:00:07 --:--:--     0\r",
      "100   870    0     0  100   870      0    105  0:00:08  0:00:08 --:--:--     0\r",
      "100   870    0     0  100   870      0     93  0:00:09  0:00:09 --:--:--     0\r",
      "100   870    0     0  100   870      0     84  0:00:10  0:00:10 --:--:--     0\r",
      "100   870    0     0  100   870      0     77  0:00:11  0:00:11 --:--:--     0\r",
      "100   870    0     0  100   870      0     70  0:00:12  0:00:12 --:--:--     0\r",
      "100   870    0     0  100   870      0     65  0:00:13  0:00:13 --:--:--     0\r",
      "100   870    0     0  100   870      0     60  0:00:14  0:00:14 --:--:--     0\r",
      "100   870    0     0  100   870      0     56  0:00:15  0:00:15 --:--:--     0\r",
      "100   870    0     0  100   870      0     53  0:00:16  0:00:16 --:--:--     0\r",
      "100   870    0     0  100   870      0     50  0:00:17  0:00:17 --:--:--     0\r",
      "100   870    0     0  100   870      0     47  0:00:18  0:00:18 --:--:--     0\r",
      "100   870    0     0  100   870      0     45  0:00:19  0:00:19 --:--:--     0\r",
      "100   870    0     0  100   870      0     42  0:00:20  0:00:20 --:--:--     0\r",
      "100  1036  100   166  100   870      7     41  0:00:23  0:00:20  0:00:03     0\r",
      "100  1036  100   166  100   870      7     41  0:00:23  0:00:20  0:00:03     0\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "\n",
    "## Create the cluster\n",
    "\n",
    "ambari_server=pregion-sean01.cloud.hortonworks.com\n",
    "pass=admin\n",
    "\n",
    "curl -u admin:${pass} \\\n",
    "  -H X-Requested-By:seano \\\n",
    "  -X POST \\\n",
    "  -d @blueprints/hdp-201503_cluster.json \\\n",
    "  http://${ambari_server}:8080/api/v1/clusters/mycluster"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3. Check Cluster creation status\n",
    "\n",
    "The command above will give you URL to check the Cluster creation status.\n",
    "\n",
    "Or you can check the status from the Ambari Web UI."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "\n",
    "ambari_server=hostname\n",
    "pass=mypassword\n",
    "\n",
    "curl -u admin:${pass} http://${ambari_server}:8080/api/v1/clusters/mycluster/requests/1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Extra: Deploy with ambari-bootstrap\n",
    "\n",
    "Below uses `ambari-bootstrap` to deploy Ambari Agents & Server. I used this to prepare my cluster for the deployment above.\n",
    "\n",
    "This can be used on clean RHEL/CentOS 6 systems."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "\n",
    "# change this to fit your configuration \n",
    "ambari_server=privateIPorHostnameOfTheServer\n",
    "\n",
    "## the URL to the installer script\n",
    "bootstrap_url=https://raw.githubusercontent.com/seanorama/ambari-bootstrap/master/ambari-bootstrap.sh\n",
    " \n",
    "## install the ambari-server\n",
    "pdsh -w ${ambari_server} \"curl -sSL ${bootstrap_url} | install_ambari_server=true sh\"\n",
    "\n",
    "## install to all other nodes. See ‘man pdsh’ for the various ways you can specify hosts.\n",
    "pdsh -w ${stack}0[2-3].cloud.hortonworks.com \"curl -sSL ${bootstrap_url} | ambari_server=${ambari_server} sh\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.4.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}