11_model_rest_api_inference_filter_transforms.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Tutorial 11: Test inference + shapley values REST API Endpoint \n",
    "\n",
    "In this notebook, we will demonstrate how to perform inference using an ML model deployed as a REST API endpoint. REST API will also filter columns that are not needed for the prediction. Output of the inference will include  reason codes (shapley values) in addition to the prediction.\n",
    "\n",
    "## Steps\n",
    "- Import requests and pandas libraries. Requests is an elegant and simple HTTP library for Python, built for human beings. More about requests [here](https://docs.python-requests.org/en/latest/)\n",
    "- Create a pandas dataframe for the data on which we want to perform inference\n",
    "- Copy the REST API endpoint from _Models_ tab\n",
    "- Create a json from the pandas dataframe\n",
    "- Add additional columns to test column filter\n",
    "- Create a header for a post request\n",
    "- Perform a post request on the REST API endpoint"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "import requests\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import joblib"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Get test data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "input_df = pd.read_csv(\"credit_card_test_new_columns.csv\")\n",
    "y_test = input_df[\"Target\"]\n",
    "X_test = input_df.drop([\"Target\"],axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Add extra columns to reproduce real case"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_test[\"V100\"] = X_test[\"V22\"]\n",
    "X_test[\"V200\"] = X_test[\"V23\"]*0.9"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Test REST API Endpoint"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 11.3 ms, sys: 3.57 ms, total: 14.9 ms\n",
      "Wall time: 39.5 ms\n"
     ]
    }
   ],
   "source": [
    "%%time \n",
    "\n",
    "url = \"https://k1yltn5.rmlpoc.net/invocations\"\n",
    "input_data = X_test.head(10)\n",
    "data_json = input_data.to_json(orient=\"split\",index=False)\n",
    "headers = {\"Content-Type\":\"application/json; format=pandas-split\"}\n",
    "response = requests.post(url,data=data_json,headers=headers)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[0, 0, 1, 0, 1, 0, 0, 0, 0, 0]"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "response.json()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>V0</th>\n",
       "      <th>V1</th>\n",
       "      <th>V2</th>\n",
       "      <th>V3</th>\n",
       "      <th>V4</th>\n",
       "      <th>V5</th>\n",
       "      <th>V6</th>\n",
       "      <th>V7</th>\n",
       "      <th>V8</th>\n",
       "      <th>V9</th>\n",
       "      <th>...</th>\n",
       "      <th>cc_exp</th>\n",
       "      <th>lat1</th>\n",
       "      <th>long1</th>\n",
       "      <th>lat2</th>\n",
       "      <th>long2</th>\n",
       "      <th>lat3</th>\n",
       "      <th>long3</th>\n",
       "      <th>ip_address</th>\n",
       "      <th>V100</th>\n",
       "      <th>V200</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>160760.0</td>\n",
       "      <td>-0.674466</td>\n",
       "      <td>1.408105</td>\n",
       "      <td>-1.110622</td>\n",
       "      <td>-1.328366</td>\n",
       "      <td>1.388996</td>\n",
       "      <td>-1.308439</td>\n",
       "      <td>1.885879</td>\n",
       "      <td>-0.614233</td>\n",
       "      <td>0.311652</td>\n",
       "      <td>...</td>\n",
       "      <td>01/23</td>\n",
       "      <td>81.627882</td>\n",
       "      <td>-100.863399</td>\n",
       "      <td>88.659301</td>\n",
       "      <td>179.866492</td>\n",
       "      <td>15.609163</td>\n",
       "      <td>-8.823885</td>\n",
       "      <td>195.127.246.134</td>\n",
       "      <td>0.810034</td>\n",
       "      <td>-0.201895</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>19847.0</td>\n",
       "      <td>-2.829816</td>\n",
       "      <td>-2.765149</td>\n",
       "      <td>2.537793</td>\n",
       "      <td>-1.074580</td>\n",
       "      <td>2.842559</td>\n",
       "      <td>-2.153536</td>\n",
       "      <td>-1.795519</td>\n",
       "      <td>-0.250020</td>\n",
       "      <td>3.073504</td>\n",
       "      <td>...</td>\n",
       "      <td>10/27</td>\n",
       "      <td>-53.558042</td>\n",
       "      <td>-46.830667</td>\n",
       "      <td>24.180878</td>\n",
       "      <td>16.312487</td>\n",
       "      <td>-69.282235</td>\n",
       "      <td>-80.631857</td>\n",
       "      <td>221.9.126.196</td>\n",
       "      <td>0.109305</td>\n",
       "      <td>-0.731945</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>88326.0</td>\n",
       "      <td>-3.576495</td>\n",
       "      <td>2.318422</td>\n",
       "      <td>1.306985</td>\n",
       "      <td>3.263665</td>\n",
       "      <td>1.127818</td>\n",
       "      <td>2.865246</td>\n",
       "      <td>1.444125</td>\n",
       "      <td>-0.718922</td>\n",
       "      <td>1.874046</td>\n",
       "      <td>...</td>\n",
       "      <td>08/30</td>\n",
       "      <td>-33.831702</td>\n",
       "      <td>-142.765298</td>\n",
       "      <td>-73.259027</td>\n",
       "      <td>-44.841458</td>\n",
       "      <td>-70.108155</td>\n",
       "      <td>-154.769389</td>\n",
       "      <td>214.77.126.181</td>\n",
       "      <td>0.016867</td>\n",
       "      <td>-0.118852</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>141734.0</td>\n",
       "      <td>2.060386</td>\n",
       "      <td>-0.015382</td>\n",
       "      <td>-1.082544</td>\n",
       "      <td>0.386019</td>\n",
       "      <td>-0.024331</td>\n",
       "      <td>-1.074935</td>\n",
       "      <td>0.207792</td>\n",
       "      <td>-0.338140</td>\n",
       "      <td>0.455091</td>\n",
       "      <td>...</td>\n",
       "      <td>10/30</td>\n",
       "      <td>-74.512890</td>\n",
       "      <td>-61.924700</td>\n",
       "      <td>-57.572207</td>\n",
       "      <td>-30.457993</td>\n",
       "      <td>24.175755</td>\n",
       "      <td>-83.299702</td>\n",
       "      <td>141.124.211.66</td>\n",
       "      <td>-0.639426</td>\n",
       "      <td>0.298636</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>38741.0</td>\n",
       "      <td>1.209965</td>\n",
       "      <td>1.384303</td>\n",
       "      <td>-1.343531</td>\n",
       "      <td>1.763636</td>\n",
       "      <td>0.662351</td>\n",
       "      <td>-2.113384</td>\n",
       "      <td>0.854039</td>\n",
       "      <td>-0.475963</td>\n",
       "      <td>-0.629658</td>\n",
       "      <td>...</td>\n",
       "      <td>12/22</td>\n",
       "      <td>49.083962</td>\n",
       "      <td>165.716597</td>\n",
       "      <td>-20.392087</td>\n",
       "      <td>-39.059461</td>\n",
       "      <td>-85.666065</td>\n",
       "      <td>169.884538</td>\n",
       "      <td>197.201.179.126</td>\n",
       "      <td>-0.328294</td>\n",
       "      <td>-0.139168</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>151934.0</td>\n",
       "      <td>-0.345588</td>\n",
       "      <td>1.070328</td>\n",
       "      <td>-0.652790</td>\n",
       "      <td>-0.492207</td>\n",
       "      <td>0.749482</td>\n",
       "      <td>-0.949391</td>\n",
       "      <td>0.752520</td>\n",
       "      <td>0.079095</td>\n",
       "      <td>0.061269</td>\n",
       "      <td>...</td>\n",
       "      <td>01/27</td>\n",
       "      <td>87.659946</td>\n",
       "      <td>136.167049</td>\n",
       "      <td>-54.402779</td>\n",
       "      <td>103.832347</td>\n",
       "      <td>74.434146</td>\n",
       "      <td>29.429568</td>\n",
       "      <td>112.180.5.227</td>\n",
       "      <td>-0.843460</td>\n",
       "      <td>0.168722</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>159122.0</td>\n",
       "      <td>-0.864273</td>\n",
       "      <td>0.361412</td>\n",
       "      <td>-0.346667</td>\n",
       "      <td>-1.938227</td>\n",
       "      <td>-1.065839</td>\n",
       "      <td>-1.056202</td>\n",
       "      <td>-0.093590</td>\n",
       "      <td>0.095647</td>\n",
       "      <td>-2.782531</td>\n",
       "      <td>...</td>\n",
       "      <td>10/31</td>\n",
       "      <td>-54.346516</td>\n",
       "      <td>160.390803</td>\n",
       "      <td>82.109498</td>\n",
       "      <td>-140.632704</td>\n",
       "      <td>-39.643957</td>\n",
       "      <td>72.387986</td>\n",
       "      <td>212.216.124.139</td>\n",
       "      <td>-0.162751</td>\n",
       "      <td>0.107555</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>41656.0</td>\n",
       "      <td>-1.679000</td>\n",
       "      <td>0.004724</td>\n",
       "      <td>2.093100</td>\n",
       "      <td>-0.209994</td>\n",
       "      <td>1.112123</td>\n",
       "      <td>-1.254060</td>\n",
       "      <td>0.193846</td>\n",
       "      <td>-0.010860</td>\n",
       "      <td>-0.568361</td>\n",
       "      <td>...</td>\n",
       "      <td>09/23</td>\n",
       "      <td>-67.018063</td>\n",
       "      <td>-68.049700</td>\n",
       "      <td>45.497728</td>\n",
       "      <td>86.709826</td>\n",
       "      <td>-27.672311</td>\n",
       "      <td>177.710189</td>\n",
       "      <td>136.185.169.115</td>\n",
       "      <td>-0.540966</td>\n",
       "      <td>-0.050977</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>52202.0</td>\n",
       "      <td>-0.806445</td>\n",
       "      <td>0.865983</td>\n",
       "      <td>1.105898</td>\n",
       "      <td>1.383957</td>\n",
       "      <td>-0.018110</td>\n",
       "      <td>0.140882</td>\n",
       "      <td>0.124651</td>\n",
       "      <td>0.603672</td>\n",
       "      <td>-0.853074</td>\n",
       "      <td>...</td>\n",
       "      <td>06/26</td>\n",
       "      <td>-38.591268</td>\n",
       "      <td>-42.851621</td>\n",
       "      <td>-37.379534</td>\n",
       "      <td>-8.876108</td>\n",
       "      <td>-63.715060</td>\n",
       "      <td>-127.410471</td>\n",
       "      <td>147.221.195.98</td>\n",
       "      <td>0.494319</td>\n",
       "      <td>-0.002675</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>52081.0</td>\n",
       "      <td>1.169646</td>\n",
       "      <td>-0.823924</td>\n",
       "      <td>0.467031</td>\n",
       "      <td>-0.840968</td>\n",
       "      <td>-0.850688</td>\n",
       "      <td>0.062139</td>\n",
       "      <td>-0.705097</td>\n",
       "      <td>0.094004</td>\n",
       "      <td>-1.020702</td>\n",
       "      <td>...</td>\n",
       "      <td>10/30</td>\n",
       "      <td>50.610906</td>\n",
       "      <td>-32.146348</td>\n",
       "      <td>-40.268943</td>\n",
       "      <td>-136.331650</td>\n",
       "      <td>22.521534</td>\n",
       "      <td>-80.302968</td>\n",
       "      <td>43.97.156.133</td>\n",
       "      <td>0.030735</td>\n",
       "      <td>-0.004528</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>10 rows × 42 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "         V0        V1        V2        V3        V4        V5        V6  \\\n",
       "0  160760.0 -0.674466  1.408105 -1.110622 -1.328366  1.388996 -1.308439   \n",
       "1   19847.0 -2.829816 -2.765149  2.537793 -1.074580  2.842559 -2.153536   \n",
       "2   88326.0 -3.576495  2.318422  1.306985  3.263665  1.127818  2.865246   \n",
       "3  141734.0  2.060386 -0.015382 -1.082544  0.386019 -0.024331 -1.074935   \n",
       "4   38741.0  1.209965  1.384303 -1.343531  1.763636  0.662351 -2.113384   \n",
       "5  151934.0 -0.345588  1.070328 -0.652790 -0.492207  0.749482 -0.949391   \n",
       "6  159122.0 -0.864273  0.361412 -0.346667 -1.938227 -1.065839 -1.056202   \n",
       "7   41656.0 -1.679000  0.004724  2.093100 -0.209994  1.112123 -1.254060   \n",
       "8   52202.0 -0.806445  0.865983  1.105898  1.383957 -0.018110  0.140882   \n",
       "9   52081.0  1.169646 -0.823924  0.467031 -0.840968 -0.850688  0.062139   \n",
       "\n",
       "         V7        V8        V9  ...  cc_exp       lat1       long1  \\\n",
       "0  1.885879 -0.614233  0.311652  ...   01/23  81.627882 -100.863399   \n",
       "1 -1.795519 -0.250020  3.073504  ...   10/27 -53.558042  -46.830667   \n",
       "2  1.444125 -0.718922  1.874046  ...   08/30 -33.831702 -142.765298   \n",
       "3  0.207792 -0.338140  0.455091  ...   10/30 -74.512890  -61.924700   \n",
       "4  0.854039 -0.475963 -0.629658  ...   12/22  49.083962  165.716597   \n",
       "5  0.752520  0.079095  0.061269  ...   01/27  87.659946  136.167049   \n",
       "6 -0.093590  0.095647 -2.782531  ...   10/31 -54.346516  160.390803   \n",
       "7  0.193846 -0.010860 -0.568361  ...   09/23 -67.018063  -68.049700   \n",
       "8  0.124651  0.603672 -0.853074  ...   06/26 -38.591268  -42.851621   \n",
       "9 -0.705097  0.094004 -1.020702  ...   10/30  50.610906  -32.146348   \n",
       "\n",
       "        lat2       long2       lat3       long3       ip_address      V100  \\\n",
       "0  88.659301  179.866492  15.609163   -8.823885  195.127.246.134  0.810034   \n",
       "1  24.180878   16.312487 -69.282235  -80.631857    221.9.126.196  0.109305   \n",
       "2 -73.259027  -44.841458 -70.108155 -154.769389   214.77.126.181  0.016867   \n",
       "3 -57.572207  -30.457993  24.175755  -83.299702   141.124.211.66 -0.639426   \n",
       "4 -20.392087  -39.059461 -85.666065  169.884538  197.201.179.126 -0.328294   \n",
       "5 -54.402779  103.832347  74.434146   29.429568    112.180.5.227 -0.843460   \n",
       "6  82.109498 -140.632704 -39.643957   72.387986  212.216.124.139 -0.162751   \n",
       "7  45.497728   86.709826 -27.672311  177.710189  136.185.169.115 -0.540966   \n",
       "8 -37.379534   -8.876108 -63.715060 -127.410471   147.221.195.98  0.494319   \n",
       "9 -40.268943 -136.331650  22.521534  -80.302968    43.97.156.133  0.030735   \n",
       "\n",
       "       V200  \n",
       "0 -0.201895  \n",
       "1 -0.731945  \n",
       "2 -0.118852  \n",
       "3  0.298636  \n",
       "4 -0.139168  \n",
       "5  0.168722  \n",
       "6  0.107555  \n",
       "7 -0.050977  \n",
       "8 -0.002675  \n",
       "9 -0.004528  \n",
       "\n",
       "[10 rows x 42 columns]"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "input_data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}