Global-Terrorism-Database.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Shmuel Ruppo - \n",
    "## Analyzing Global Terrorist Attacks"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<pre>In the GTD database, the groups who carried out some of the terror attacks have not been \n",
    "identified. The task is to build a model that predicts the responsible group of those \n",
    "unidentified attacks</pre>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "import time\n",
    "import h2o\n",
    "from h2o.estimators import H2ORandomForestEstimator\n",
    "import random"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<pre>\n",
    "Importing a large file in CSV format is 10-20 times faster than importing an Excel file.\n",
    "Using MS Excel, I have converted the Excel file provided to CSV. </pre>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(181691, 135)"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# read from file\n",
    "df = pd.read_csv('data/gtd/ALL_DATASET.csv', header=0, low_memory=False )\n",
    "df.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "   ## Data cleaning:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<pre>Throughout all the columns, the values -99, 9 stand for the 'Unknown' category.\n",
    "Therefore, they all can be replaced with nan </pre>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = df.replace(to_replace=[-99, -9], value=np.nan)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<pre>\n",
    "Using the codebook provided by GTD (https://www.start.umd.edu/gtd/downloads/Codebook.pdf),\n",
    "and double-checking through the excel file, I have learned that every textual categorical\n",
    "variable in the dataset is already encoded as a numerical categorical variable.\n",
    "For example, the list of countries : 'country_txt' is encoded as a numeric list of countries\n",
    "'country'. Thus, I can work only with numeric variables. \n",
    "But, an important exception is the name of the terror group 'gname', which is not encoded.\n",
    "So, I leave it as is. </pre>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = df._get_numeric_data().join(df['gname'])\n",
    "\n",
    "#Also, 'eventid' uniquely identifies each terror act. But it does not carry any meaning within it,\n",
    "#and therefore can be dropped\n",
    "\n",
    "df = df.drop('eventid', axis=1)\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<pre>The name of the terror group is the response variable.\n",
    "Terror groups that have been identified a small amount of times will likely will add more\n",
    "noise than information. Thus, I remove terror groups that have carried out less (or equal) to 5\n",
    "terror acts. </pre>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(177093, 77)"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "temp = ( pd.value_counts(df.loc[:,'gname']))\n",
    "df = df.loc[df['gname'].isin(temp.index[temp > 5]),:]\n",
    "\n",
    "\n",
    "# Display the dimensions of the data\n",
    "df.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<pre>I distinguish between numeric and categorical variables, making a list of each\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "#List of the numeric variables:\n",
    "numericList = [\n",
    " 'nkill', 'nkillus', 'nkillter', 'nwound', 'nwoundus', \n",
    "    'nwoundte', 'nhostkid', 'iyear', 'imonth', 'iday','nhostkidus',  'ransomamt', 'ransomamtus',\n",
    "'ransompaid', 'ransompaidus', 'nreleased', 'propvalue', 'latitude', 'longitude']\n",
    "\n",
    "\n",
    "# Get the location of numeric\n",
    "locNumeric = [df.columns.get_loc(c) for c in numericList]\n",
    "\n",
    "#Whatever is not numeric, is categoric\n",
    "categNum = [i for i in range(len(df.columns)) if i not in locNumeric]\n",
    "categList = [i for i in df.columns if i not in numericList]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<pre>I separate, and put  aside the terror acts that have not been identified as belonging to a \n",
    "specific terror group. I will use them for the final prediction</pre>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "mask_index = df.loc[(df['gname'] == 'Unknown'), 'gname'].index\n",
    "\n",
    "prd_group = df.loc[mask_index,:]   # Will be used for the final prediction\n",
    "df = df.drop(mask_index)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# The model : Random Forest"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<pre>I will use the Random Forest method for classification of the data.\n",
    "The Random Forest method is possible to use with a large amount of features.\n",
    "It is highly economical in terms of preprocessing. It handles both categorical \n",
    "and numerical variables. (The numerical data does not even need scaling). \n",
    "\n",
    "Rather than guessing \\ deciding using an expert opinion (which is unavailable to me)\n",
    "what are the most important features, it is possible to analyze the features that contribute\n",
    "the most to the Random Forest technique.\n",
    "\n",
    "My strategy is:\n",
    "First, use a Random Forest model on a small subset of the database.\n",
    "See what are the features that contribute most to the random forest.\n",
    "Then use those features on a larger subset of the database.\n",
    "\n",
    "I will use the H2O library. \n",
    "A benefit of using H2O library is its efficient use of categorical variables.\n",
    "It deals with categorical variables without needing one-hot encoding. </pre>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Checking whether there is an H2O instance running at http://localhost:54321..... not found.\n",
      "Attempting to start a local H2O server...\n",
      "  Java Version: openjdk version \"10.0.2\" 2018-07-17; OpenJDK Runtime Environment (build 10.0.2+13-Ubuntu-1ubuntu0.18.04.4); OpenJDK 64-Bit Server VM (build 10.0.2+13-Ubuntu-1ubuntu0.18.04.4, mixed mode)\n",
      "  Starting server from /usr/local/lib/python2.7/dist-packages/h2o/backend/bin/h2o.jar\n",
      "  Ice root: /tmp/tmpVsfDwC\n",
      "  JVM stdout: /tmp/tmpVsfDwC/h2o_unknownUser_started_from_python.out\n",
      "  JVM stderr: /tmp/tmpVsfDwC/h2o_unknownUser_started_from_python.err\n",
      "  Server is running at http://127.0.0.1:54321\n",
      "Connecting to H2O server at http://127.0.0.1:54321... successful.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div style=\"overflow:auto\"><table style=\"width:50%\"><tr><td>H2O cluster uptime:</td>\n",
       "<td>02 secs</td></tr>\n",
       "<tr><td>H2O cluster timezone:</td>\n",
       "<td>Etc/UTC</td></tr>\n",
       "<tr><td>H2O data parsing timezone:</td>\n",
       "<td>UTC</td></tr>\n",
       "<tr><td>H2O cluster version:</td>\n",
       "<td>3.22.0.3</td></tr>\n",
       "<tr><td>H2O cluster version age:</td>\n",
       "<td>4 days </td></tr>\n",
       "<tr><td>H2O cluster name:</td>\n",
       "<td>H2O_from_python_unknownUser_gpt3g2</td></tr>\n",
       "<tr><td>H2O cluster total nodes:</td>\n",
       "<td>1</td></tr>\n",
       "<tr><td>H2O cluster free memory:</td>\n",
       "<td>2.938 Gb</td></tr>\n",
       "<tr><td>H2O cluster total cores:</td>\n",
       "<td>2</td></tr>\n",
       "<tr><td>H2O cluster allowed cores:</td>\n",
       "<td>2</td></tr>\n",
       "<tr><td>H2O cluster status:</td>\n",
       "<td>accepting new members, healthy</td></tr>\n",
       "<tr><td>H2O connection url:</td>\n",
       "<td>http://127.0.0.1:54321</td></tr>\n",
       "<tr><td>H2O connection proxy:</td>\n",
       "<td>None</td></tr>\n",
       "<tr><td>H2O internal security:</td>\n",
       "<td>False</td></tr>\n",
       "<tr><td>H2O API Extensions:</td>\n",
       "<td>XGBoost, Algos, AutoML, Core V3, Core V4</td></tr>\n",
       "<tr><td>Python version:</td>\n",
       "<td>2.7.15 candidate</td></tr></table></div>"
      ],
      "text/plain": [
       "--------------------------  ----------------------------------------\n",
       "H2O cluster uptime:         02 secs\n",
       "H2O cluster timezone:       Etc/UTC\n",
       "H2O data parsing timezone:  UTC\n",
       "H2O cluster version:        3.22.0.3\n",
       "H2O cluster version age:    4 days\n",
       "H2O cluster name:           H2O_from_python_unknownUser_gpt3g2\n",
       "H2O cluster total nodes:    1\n",
       "H2O cluster free memory:    2.938 Gb\n",
       "H2O cluster total cores:    2\n",
       "H2O cluster allowed cores:  2\n",
       "H2O cluster status:         accepting new members, healthy\n",
       "H2O connection url:         http://127.0.0.1:54321\n",
       "H2O connection proxy:\n",
       "H2O internal security:      False\n",
       "H2O API Extensions:         XGBoost, Algos, AutoML, Core V3, Core V4\n",
       "Python version:             2.7.15 candidate\n",
       "--------------------------  ----------------------------------------"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python2.7/dist-packages/ipykernel_launcher.py:16: FutureWarning: \n",
      "Passing list-likes to .loc or [] with any missing label will raise\n",
      "KeyError in the future, you can use .reindex() as an alternative.\n",
      "\n",
      "See the documentation here:\n",
      "http://pandas.pydata.org/pandas-docs/stable/indexing.html#deprecate-loc-reindex-listlike\n",
      "  app.launch_new_instance()\n",
      "/usr/local/lib/python2.7/dist-packages/pandas/core/indexing.py:1367: FutureWarning: \n",
      "Passing list-likes to .loc or [] with any missing label will raise\n",
      "KeyError in the future, you can use .reindex() as an alternative.\n",
      "\n",
      "See the documentation here:\n",
      "http://pandas.pydata.org/pandas-docs/stable/indexing.html#deprecate-loc-reindex-listlike\n",
      "  return self._getitem_tuple(key)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Parse progress: |█████████████████████████████████████████████████████████| 100%\n",
      "drf Model Build progress: |███████████████████████████████████████████████| 100%\n",
      "seconds it took 2.154032\n",
      "Top-10 Hit Ratios: \n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div style=\"overflow:auto\"><table style=\"width:50%\"><tr><td><b>k</b></td>\n",
       "<td><b>hit_ratio</b></td></tr>\n",
       "<tr><td>1</td>\n",
       "<td>0.7895002</td></tr>\n",
       "<tr><td>2</td>\n",
       "<td>0.8399798</td></tr>\n",
       "<tr><td>3</td>\n",
       "<td>0.861686</td></tr>\n",
       "<tr><td>4</td>\n",
       "<td>0.8768299</td></tr>\n",
       "<tr><td>5</td>\n",
       "<td>0.8828874</td></tr>\n",
       "<tr><td>6</td>\n",
       "<td>0.8864210</td></tr>\n",
       "<tr><td>7</td>\n",
       "<td>0.8944977</td></tr>\n",
       "<tr><td>8</td>\n",
       "<td>0.8985361</td></tr>\n",
       "<tr><td>9</td>\n",
       "<td>0.9010600</td></tr>\n",
       "<tr><td>10</td>\n",
       "<td>0.9030792</td></tr></table></div>"
      ],
      "text/plain": [
       "k    hit_ratio\n",
       "---  -----------\n",
       "1    0.7895\n",
       "2    0.83998\n",
       "3    0.861686\n",
       "4    0.87683\n",
       "5    0.882887\n",
       "6    0.886421\n",
       "7    0.894498\n",
       "8    0.898536\n",
       "9    0.90106\n",
       "10   0.903079"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    }
   ],
   "source": [
    "t=time.clock()\n",
    "\n",
    "# Differentiate between training and response variables\n",
    "training_columns=list(df.columns.difference(['gname']))\n",
    "response_column = ['gname']\n",
    "\n",
    "# Create sample the data\n",
    "random.seed(123)\n",
    "mysample = random.sample(range(df.shape[0]), 10000)\n",
    "\n",
    "# init H2O\n",
    "h2o.init()\n",
    "h2o.remove_all()\n",
    "\n",
    "# convert the data into H2OFrame\n",
    "hf = h2o.H2OFrame(df.loc[mysample,:])\n",
    "\n",
    "# Distinguish between categoric and numeric features\n",
    "hf[:,numericList]=hf[:,numericList].asnumeric() \n",
    "hf[:,categList] = hf[:,categList].asfactor()\n",
    "\n",
    "## Split the data into a train and a test set.  Random Forest will be tested on the latter.\n",
    "train, test = hf.split_frame(ratios=[0.8], seed = 123)\n",
    "\n",
    "\n",
    "# Do not select a large number of trees grown, and too large a depth initially\n",
    "model = H2ORandomForestEstimator(  max_depth=5, ntrees = 5, seed= 123)\n",
    "\n",
    "# Train model\n",
    "\n",
    "model.train(x=training_columns, y=str(response_column[0]), training_frame=train)\n",
    "\n",
    "# Test the model\n",
    "performance = model.model_performance(test_data=test)\n",
    "print 'seconds it took', time.clock() - t\n",
    "print performance.hit_ratio_table()     "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<pre>I am taking only the most important features:\n",
    "I have selected the features whose percentage of contribution is more than 1%</pre>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "        variable  percentage\n",
      "0        country    0.129785\n",
      "1        natlty1    0.123128\n",
      "2           iday    0.106729\n",
      "3    specificity    0.099089\n",
      "4        success    0.097614\n",
      "5       extended    0.097484\n",
      "6     individual    0.096568\n",
      "7   targsubtype1    0.060893\n",
      "8       latitude    0.030478\n",
      "9          iyear    0.012483\n",
      "10  weapsubtype1    0.011800\n"
     ]
    }
   ],
   "source": [
    "importance = model.varimp(use_pandas=True)\n",
    "\n",
    "importance = importance[importance['percentage']>0.01]\n",
    "\n",
    "print importance[['variable', 'percentage']]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<pre>This is what the feature names stand for :\n",
    "\n",
    "country       The country where the incident occured\n",
    "natlty1       The nationality of the target that was attacked\n",
    "iday          Day of the month\n",
    "specificity   The precision with which the latitude and the longitude are known\n",
    "success       Whether there was a tangible effect of the attack \n",
    "extended      Whether the duration of the attack was more than 24 hours\n",
    "targsubtype1  The specific category to which the target belongs\n",
    "latitude      Latitude of the attack\n",
    "iyear         Year of the attack\n",
    "weapsubtype1  Specific category of the weapon used \n",
    "\n",
    "The selected features \"make sense\". For example, it is expected that the country the attack\n",
    "takes place in would be of high importance in the explanation.\n",
    "\n",
    "The next step is fine-tuning the model with the important features only. First, the set-up:</pre>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python2.7/dist-packages/ipykernel_launcher.py:11: FutureWarning: \n",
      "Passing list-likes to .loc or [] with any missing label will raise\n",
      "KeyError in the future, you can use .reindex() as an alternative.\n",
      "\n",
      "See the documentation here:\n",
      "http://pandas.pydata.org/pandas-docs/stable/indexing.html#deprecate-loc-reindex-listlike\n",
      "  # This is added back by InteractiveShellApp.init_path()\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Parse progress: |█████████████████████████████████████████████████████████| 100%\n"
     ]
    }
   ],
   "source": [
    "#Set-up the model with the important features\n",
    "\n",
    "training_columns=list(importance['variable'])\n",
    "response_column = ['gname']\n",
    "\n",
    "df2 = df[training_columns+ response_column]\n",
    "\n",
    "random.seed(123)\n",
    "mysample = random.sample(range(df2.shape[0]), 10000)\n",
    "\n",
    "df2 = df2.loc[mysample,:]\n",
    "\n",
    "hf2 = h2o.H2OFrame(df2)\n",
    "\n",
    "# For the sake of distinguishing between categorical and numeric variables\n",
    "# I distinguish between numerical variables selected as important:\n",
    "numericList_imp = list(np.intersect1d(numericList, importance['variable']))\n",
    "\n",
    "\n",
    "# and the  categoric variables selected as important\n",
    "categList_imp = list(np.intersect1d(categList, importance['variable']))\n",
    "#The name of the terror group is also categorical\n",
    "categList_imp.append('gname')\n",
    "\n",
    "# Make the distinction between numeric and categoric variables\n",
    "hf2[:,numericList_imp]=hf2[:,numericList_imp].asnumeric() \n",
    "hf2[:,categList_imp] = hf2[:,categList_imp].asfactor()\n",
    "\n",
    "\n",
    "# Again split into train and test\n",
    "\n",
    "train, test = hf2.split_frame(ratios=[0.8], seed = 123)\n",
    "\n",
    "\n",
    "# A function to iterate over the different parameters of the Random Forest model\n",
    "\n",
    "def EstimateModel( ntrees,  max_depth ):\n",
    "    t=time.clock()\n",
    "    global model2\n",
    "    \n",
    "    # Initialize the model\n",
    "    model2 = H2ORandomForestEstimator( ntrees = ntrees, max_depth = max_depth, seed= 123)\n",
    "    # Train the model\n",
    "    model2.train(x=training_columns, y=str(response_column[0]), training_frame=train)\n",
    "    # Predict performance\n",
    "    performance = model2.model_performance(test_data=test)    \n",
    "    \n",
    "    #Display parameters, performance, and the amount of time the analysis took\n",
    "    print 'ntrees = ', ntrees, 'max_depth=', max_depth, 'Hit ratio=', performance.hit_ratio_table()[1][0]\n",
    "    print 'time it took', time.clock() - t, 'seconds'\n",
    "  \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<pre>The next step I took is iterating over a combination of the parameters to find the best.\n",
    "I have created a vector of 5 different values for the amount of trees to be grown, \n",
    "and a vector of 5 values for the depth parameter. \n",
    "\n",
    "However, I do not check each of the 25 possible combinations.\n",
    "Rather, the initial step is to increment both parameters each iteration.\n",
    "\n",
    "I will use the hit-ratio as the metric to measure success.  It is the number of \n",
    "times that a correct prediciton has been made divided by  the total number of predictions. \n",
    "It is an easy-to-communicate parameter. </pre>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "drf Model Build progress: |███████████████████████████████████████████████| 100%\n",
      "ntrees =  8 max_depth= 3 Hit ratio= 0.79202425\n",
      "time it took 1.120228 seconds\n",
      "drf Model Build progress: |███████████████████████████████████████████████| 100%\n",
      "ntrees =  16 max_depth= 5 Hit ratio= 0.81120646\n",
      "time it took 1.10418 seconds\n",
      "drf Model Build progress: |███████████████████████████████████████████████| 100%\n",
      "ntrees =  20 max_depth= 7 Hit ratio= 0.81272084\n",
      "time it took 0.876742 seconds\n",
      "drf Model Build progress: |███████████████████████████████████████████████| 100%\n",
      "ntrees =  24 max_depth= 9 Hit ratio= 0.8192832\n",
      "time it took 1.000314 seconds\n",
      "drf Model Build progress: |███████████████████████████████████████████████| 100%\n",
      "ntrees =  28 max_depth= 10 Hit ratio= 0.819788\n",
      "time it took 1.07512 seconds\n"
     ]
    }
   ],
   "source": [
    "\n",
    "# Create a vector\n",
    "ntrees_list = (8,16,20, 24, 28) \n",
    "max_depth_list = (3,5,7,9,10)\n",
    "\n",
    "for i in range(len(ntrees_list)):\n",
    "    EstimateModel(ntrees_list[i], max_depth_list[i])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<pre>The changes in the hit ratio become rather small.\n",
    "\n",
    "When I decrease the number of trees, it has an additional small positive effect:</pre>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "drf Model Build progress: |███████████████████████████████████████████████| 100%\n",
      "ntrees =  24 max_depth= 10 Hit ratio= 0.8202928\n",
      "time it took 1.039864 seconds\n"
     ]
    }
   ],
   "source": [
    "EstimateModel(ntrees=24, max_depth=10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " #### The hit-ratio on the validation set  is 82% "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Final prediction:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<pre>Not all of the terror acts in the GTD database have been identified with a certain group.\n",
    "The task of the challenge is to predict the groups responsible for those un-identified terror\n",
    "acts. Though of course it's impossible to verify the truth of their predictions using the\n",
    "current database, a hit-ratio of 82% on the validation set does look good (especially if we \n",
    "take into account the large number of terror groups that are involved.) </pre>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<pre>I use the parameters that gave the highest hit-ratio so far (24 trees, 10 depth), and\n",
    "make the final prediction</pre>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Parse progress: |█████████████████████████████████████████████████████████| 100%\n",
      "Parse progress: |█████████████████████████████████████████████████████████| 100%\n",
      "drf Model Build progress: |███████████████████████████████████████████████| 100%\n",
      "drf prediction progress: |████████████████████████████████████████████████| 100%\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python2.7/dist-packages/h2o/job.py:69: UserWarning: Test/Validation dataset column 'country' has levels not trained on: [10, 17, 22, 23, 35, 63, 64, 72, 73, 80, 85, 91, 109, 115, 117, 120, 124, 129, 132, 143, 180, 181, 189, 219, 223, 226, 377, 428, 1001, 1002]\n",
      "  warnings.warn(w)\n",
      "/usr/local/lib/python2.7/dist-packages/h2o/job.py:69: UserWarning: Test/Validation dataset column 'natlty1' has levels not trained on: [10, 22, 23, 24, 31, 64, 70, 72, 73, 80, 85, 109, 117, 124, 125, 127, 129, 132, 134, 143, 176, 180, 189, 225, 377, 1002]\n",
      "  warnings.warn(w)\n"
     ]
    }
   ],
   "source": [
    "# Prepare the final dataset for H2O library\n",
    "\n",
    "\n",
    "hf_all = h2o.H2OFrame(df[training_columns+ response_column])\n",
    "\n",
    "#Preparing the final test-set\n",
    "prd_group = prd_group.drop('gname', axis=1)\n",
    "hf_pred = h2o.H2OFrame(prd_group)\n",
    "\n",
    "\n",
    "# Differentiate between numeric and categoric variables \n",
    "hf_pred[:,numericList_imp]=hf_pred[:,numericList_imp].asnumeric() \n",
    "hf_pred[:,categList_imp[:-1]] = hf_pred[:,categList_imp[:-1]].asfactor()\n",
    "\n",
    "hf_all[:,numericList_imp]=hf_all[:,numericList_imp].asnumeric() \n",
    "hf_all[:,categList_imp] = hf_all[:,categList_imp].asfactor()\n",
    "\n",
    "#initialize model\n",
    "model_all = H2ORandomForestEstimator( ntrees = 24, max_depth = 10, seed= 123)\n",
    "# Train the model\n",
    "model_all.train(x=training_columns, y=str(response_column[0]), training_frame=hf_all)\n",
    "\n",
    "# Make the prediction\n",
    "final_pred = model_all.predict(hf_pred)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<pre>Though using the current dataset only, it is not possible to verify the final results,\n",
    "a basic feasability-check can be performed.\n",
    "I show all the groups identified as being responsible for terror acts in Israel (my country),\n",
    "grouped by the number of identifications made for each group.</pre>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Hamas (Islamic Resistance Movement)                        280\n",
       "Palestinians                                               244\n",
       "Palestinian Islamic Jihad (PIJ)                             92\n",
       "Palestinian Extremists                                      51\n",
       "Hezbollah                                                   33\n",
       "Al-Aqsa Martyrs Brigade                                     31\n",
       "Popular Resistance Committees                               13\n",
       "Israeli extremists                                           9\n",
       "Popular Front for the Liberation of Palestine (PFLP)         8\n",
       "Ansar Bayt al-Maqdis (Ansar Jerusalem)                       4\n",
       "Democratic Front for the Liberation of Palestine (DFLP)      3\n",
       "Keshet                                                       2\n",
       "Palestine Liberation Organization (PLO)                      1\n",
       "Jewish Extremists                                            1\n",
       "Black Panther Group (Palestinian)                            1\n",
       "Tripoli Province of the Islamic State                        1\n",
       "Israeli settlers                                             1\n",
       "Liwa Ahrar al-Sunna                                          1\n",
       "dtype: int64"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "results = hf_pred.cbind(final_pred['predict']).as_data_frame()\n",
    "results[results['country'] == 97].loc[:,['predict']].stack().value_counts()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<pre>Many of those group names are familiar to Israeli citizens as groups that performed terror acts in   the past. \n",
    "One can also notice diversity in the results - though most of the predictions are of \n",
    "Palestinian and Islamic groups, not all are (Keshet,Jewish Extremists, Israeli Settlers). That helps to demonstrate to the non-technical reader the \"smartness\" of the algorithm. \n",
    "\n",
    "All in all, the results appear sensible. \n",
    "</pre>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "H2O session _sid_ad73 closed.\n"
     ]
    }
   ],
   "source": [
    "#Shutdown H2O\n",
    "h2o.cluster().shutdown(prompt = False)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.15"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}