drum_overview/Main_Script.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "3QiN-nVjJWg5"
   },
   "source": [
    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/datarobot-community/custom-models/blob/master/drum_overview/Main_Script.ipynb)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ciWd0qY5JWg6"
   },
   "source": [
    "## Using MLOps DRUM to test your custom models\n",
    "\n",
    "**Original Author**: Tim Whittaker\n",
    "\n",
    "**Modified by**: Mandie Quartly\n",
    "\n",
    "#### Scope\n",
    "We'll get our hands dirty by:\n",
    "\n",
    "* Building a simple regression model using Scikit\n",
    "* Using DRUM for Batch Scoring\n",
    "* Using DRUM to get a REST API endpoint\n",
    "* Show a simple example app connected to the REST API\n",
    "* H2O, Keras, XGBoost, and DataRobot\n",
    "* Add a DataRobot remote agent if you are interested in further model monitoring\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "8_4R4i2JJWg7",
    "outputId": "4560dbb7-c809-411f-8982-5c48d94d553f"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Cloning into 'mlops-examples'...\n",
      "remote: Enumerating objects: 1407, done.\u001b[K\n",
      "remote: Counting objects: 100% (157/157), done.\u001b[K\n",
      "remote: Compressing objects: 100% (20/20), done.\u001b[K\n",
      "remote: Total 1407 (delta 142), reused 137 (delta 137), pack-reused 1250\u001b[K\n",
      "Receiving objects: 100% (1407/1407), 110.29 MiB | 19.87 MiB/s, done.\n",
      "Resolving deltas: 100% (637/637), done.\n",
      "Checking out files: 100% (196/196), done.\n"
     ]
    }
   ],
   "source": [
    "#Clone the repository\n",
    "!git clone https://github.com/datarobot-community/mlops-examples"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "CAKydOxLJWg7",
    "outputId": "bcf889c5-ed5d-449c-d3a1-e3aabe1141b2"
   },
   "outputs": [],
   "source": [
    "# !pip install -r ./mlops-examples/drum_overview/requirements.txt\n",
    "\n",
    "%pip install datarobot-drum"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "PmW8wCvxJWg8"
   },
   "source": [
    "## Train a regression model\n",
    "\n",
    "A simple RandomForestRegressor to predict the compressive strength of high-performance concrete using the Concrete compressive strength dataset found in this paper:\n",
    "\n",
    "`I-Cheng Yeh, \"Modeling of strength of high performance concrete using artificial neural networks\" Cement and Concrete Research, Vol. 28, No. 12, pp. 1797-1808 (1998).`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "1slSPuVKJWg8",
    "outputId": "1ec06b1c-ce51-4551-eef6-1ac666911620"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([49.35179576, 16.05376805, 29.9966807 , 11.05407913,  4.31469944,\n",
       "       24.99589748, 41.44826343,  5.22090533, 34.01369494, 36.18736849])"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "from sklearn.ensemble import RandomForestRegressor\n",
    "import pickle\n",
    "import datetime\n",
    "\n",
    "# Read the train and test data\n",
    "TRAIN_DATA_REG = \"./mlops-examples/drum_overview/data/concrete_train.csv\"  # 14 features\n",
    "TEST_DATA_REG = \"./mlops-examples/drum_overview/data/concrete_test.csv\"  # 13 features - target is removed\n",
    "\n",
    "reg_X_train = pd.read_csv(TRAIN_DATA_REG)\n",
    "reg_Y_train = reg_X_train.pop('concrete_compressive_strength')\n",
    "\n",
    "reg_X_test = pd.read_csv(TEST_DATA_REG)\n",
    "\n",
    "# Fit the model\n",
    "rf = RandomForestRegressor()\n",
    "rf.fit(reg_X_train, reg_Y_train)\n",
    "\n",
    "# Pickle the file and write it to the file system\n",
    "with open(\"./mlops-examples/drum_overview/custom_model_reg/reg_rf_model.pkl\", 'wb') as pkl:\n",
    "    pickle.dump(rf, pkl)\n",
    "    \n",
    "# Call predict to confirm it works\n",
    "rf.predict(reg_X_test)[0:10]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "7SHTlEDwJWg9"
   },
   "source": [
    "## Testing the Model\n",
    "\n",
    "Input the prediction dataset that includes all features except the target feature."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "DC4cgK9XJWg9",
    "outputId": "034729c1-e0c8-4a15-ff16-ddfedd807d79"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "DRUM performance test\n",
      "Model:      /content/mlops-examples/drum_overview/custom_model_reg\n",
      "Data:       /content/mlops-examples/drum_overview/data/concrete_test.csv\n",
      "# Features: 8\n",
      "Preparing test data...\n",
      "\n",
      "\n",
      "\n",
      "Running test case with timeout: 600\n",
      "Running test case: 41 bytes - 1 samples, 100 iterations\n",
      "Running test case with timeout: 600\n",
      "Running test case: 0.1MB - 2529 samples, 50 iterations\n",
      "Running test case with timeout: 600\n",
      "Running test case: 10MB - 252956 samples, 5 iterations\n",
      "Running test case with timeout: 600\n",
      "Running test case: 50MB - 1264781 samples, 1 iterations\n",
      "Test is done, stopping drum server\n",
      "\n",
      "  size     samples   iters    min     avg     max     total     used     total p\n",
      "                                                       (s)      (MB)     hysical\n",
      "                                                                          (MB)  \n",
      "================================================================================\n",
      "41 bytes         1     100   0.017   0.018   0.022     1.776        NA        NA\n",
      "0.1MB         2529      50   0.054   0.057   0.083     2.834        NA        NA\n",
      "10MB        252956       5   2.039   2.069   2.096    10.347        NA        NA\n",
      "50MB       1264781       1   9.999   9.999   9.999     9.999        NA        NA\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "tput: terminal attributes: No such device or address\n",
      "\n"
     ]
    }
   ],
   "source": [
    "%%sh \n",
    "drum perf-test --code-dir ./mlops-examples/drum_overview/custom_model_reg \\\n",
    "--input ./mlops-examples/drum_overview/data/concrete_test.csv  --target-type regression"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "9MdfM2aSJWg-"
   },
   "source": [
    "## Validating the Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "id": "W3wN0UCtJWg-"
   },
   "outputs": [],
   "source": [
    "%%sh \n",
    "drum validation --code-dir ./mlops-examples/drum_overview/custom_model_reg \\\n",
    "--input ./mlops-examples/drum_overview/data/concrete_test.csv \\\n",
    "--target-type regression > drum_validation.log"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "HUXNcuNpJWg-",
    "outputId": "032d0620-6eee-4532-fba3-4fa2efb55467"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "Validation checks results\n",
      "      Test case          Status   Details\n",
      "=========================================\n",
      "Basic batch prediction   PASSED          \n",
      "Null value imputation    PASSED          \n"
     ]
    }
   ],
   "source": [
    "%%sh\n",
    "tail drum_validation.log"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "fm0SMGtsJWg-"
   },
   "source": [
    "# Batch Scoring with DRUM\n",
    "<a id=\"setup_complete\"></a>\n",
    "\n",
    "At this point our model has been written to disk and we want to start making predictions with it.  To do this, we'll leverage DRUM and it's ability to natively handle our scikit learn model, all we need to do is tell DRUM where it resides as well as the data we wish to score.  \n",
    "\n",
    "There are a lot of frameworks which DRUM supports natively, but for those which DRUM doesn't support off the shelf, we'll just need to create some custom hooks. In this example, we'll highlight some very simple custom hooks, and will provide links to more complex examples.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "id": "WZNX8J9BJWg_"
   },
   "outputs": [],
   "source": [
    "%%sh \n",
    "drum score --code-dir ./mlops-examples/drum_overview/custom_model_reg \\\n",
    "--input ./mlops-examples/drum_overview/data/concrete_test.csv \\\n",
    "--output ./mlops-examples/drum_overview/data/predictions.csv --target-type regression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 206
    },
    "id": "YNYrrltmJWg_",
    "outputId": "c8156ea9-f406-4c17-85ac-cb5cca2f7a16"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "  <div id=\"df-a4c38d77-bd42-4268-a5c0-b89c1083ee58\">\n",
       "    <div class=\"colab-df-container\">\n",
       "      <div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Predictions</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>49.351796</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>16.053768</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>29.996681</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>11.054079</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4.314699</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>\n",
       "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-a4c38d77-bd42-4268-a5c0-b89c1083ee58')\"\n",
       "              title=\"Convert this dataframe to an interactive table.\"\n",
       "              style=\"display:none;\">\n",
       "        \n",
       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
       "       width=\"24px\">\n",
       "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
       "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
       "  </svg>\n",
       "      </button>\n",
       "      \n",
       "  <style>\n",
       "    .colab-df-container {\n",
       "      display:flex;\n",
       "      flex-wrap:wrap;\n",
       "      gap: 12px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert {\n",
       "      background-color: #E8F0FE;\n",
       "      border: none;\n",
       "      border-radius: 50%;\n",
       "      cursor: pointer;\n",
       "      display: none;\n",
       "      fill: #1967D2;\n",
       "      height: 32px;\n",
       "      padding: 0 0 0 0;\n",
       "      width: 32px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert:hover {\n",
       "      background-color: #E2EBFA;\n",
       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "      fill: #174EA6;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert {\n",
       "      background-color: #3B4455;\n",
       "      fill: #D2E3FC;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert:hover {\n",
       "      background-color: #434B5C;\n",
       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
       "      fill: #FFFFFF;\n",
       "    }\n",
       "  </style>\n",
       "\n",
       "      <script>\n",
       "        const buttonEl =\n",
       "          document.querySelector('#df-a4c38d77-bd42-4268-a5c0-b89c1083ee58 button.colab-df-convert');\n",
       "        buttonEl.style.display =\n",
       "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "\n",
       "        async function convertToInteractive(key) {\n",
       "          const element = document.querySelector('#df-a4c38d77-bd42-4268-a5c0-b89c1083ee58');\n",
       "          const dataTable =\n",
       "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
       "                                                     [key], {});\n",
       "          if (!dataTable) return;\n",
       "\n",
       "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
       "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
       "            + ' to learn more about interactive tables.';\n",
       "          element.innerHTML = '';\n",
       "          dataTable['output_type'] = 'display_data';\n",
       "          await google.colab.output.renderOutput(dataTable, element);\n",
       "          const docLink = document.createElement('div');\n",
       "          docLink.innerHTML = docLinkHtml;\n",
       "          element.appendChild(docLink);\n",
       "        }\n",
       "      </script>\n",
       "    </div>\n",
       "  </div>\n",
       "  "
      ],
      "text/plain": [
       "   Predictions\n",
       "0    49.351796\n",
       "1    16.053768\n",
       "2    29.996681\n",
       "3    11.054079\n",
       "4     4.314699"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.read_csv(\"./mlops-examples/drum_overview/data/predictions.csv\").head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "L6yIfgmsJWg_"
   },
   "source": [
    "# Start the inference server locally\n",
    "\n",
    "Batch scoring can be very useful, but the utility DRUM offers does not stop there.  We can also leverage DRUM to serve our model as a RESTful API endpoint.  The only thing that changes is the way we will structure the command - using the `server` mode instead of `score` model.  We'll also need to provide an address which is NOT in use.  \n",
    "\n",
    "When starting the server, we'll use `subprocess.Popen` so we may interact with the server in this notebook"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "id": "ipDcOzgLJWg_"
   },
   "outputs": [],
   "source": [
    "import subprocess\n",
    "import requests\n",
    "import pandas as pd\n",
    "from io import BytesIO\n",
    "import yaml\n",
    "import time\n",
    "import os\n",
    "import datarobot as dr\n",
    "from pprint import pprint"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "id": "1DLYdYYeJWg_"
   },
   "outputs": [],
   "source": [
    "run_inference_server = [\"drum\",\n",
    "              \"server\",\n",
    "              \"--code-dir\",\"./mlops-examples/drum_overview/custom_model_reg\", \n",
    "              \"--address\", \"0.0.0.0:6789\", \n",
    "              \"--show-perf\",\n",
    "              \"--target-type\", \"regression\",\n",
    "              \"--logging-level\", \"info\",\n",
    "              \"--show-stacktrace\",\n",
    "              \"--verbose\"\n",
    "              ]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "id": "_8kYmvxIJWhA"
   },
   "outputs": [],
   "source": [
    "inference_server = subprocess.Popen(run_inference_server, stdout=subprocess.PIPE)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "HtBozypYJWhA",
    "outputId": "4e9f6919-af6a-4567-acce-20d79b5173b9"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "check status\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "b'{\"message\":\"OK\"}\\n'"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "## confirm the server is running\n",
    "time.sleep(10) ## snoozing before pinging the server to give it time to actually start\n",
    "print('check status')\n",
    "requests.request(\"GET\", \"http://0.0.0.0:6789\").content"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "2qCt32ieJWhA"
   },
   "source": [
    "## Send data to server for inference\n",
    "\n",
    "The request must provide our dataset as form data.  In order to do so, we'll create a simple python function to pass the data over appropriately.  We'll leverage the same function in our simple flask app a little later.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "id": "FViV9824JWhA"
   },
   "outputs": [],
   "source": [
    "def score(data, port = \"6789\"):\n",
    "    b_buf = BytesIO()\n",
    "    b_buf.write(data.to_csv(index=False).encode(\"utf-8\"))\n",
    "    b_buf.seek(0)\n",
    "  \n",
    "    url = \"http://localhost:{}/predict/\".format(port)\n",
    "    files = [\n",
    "        ('X', b_buf)\n",
    "    ]\n",
    "    response = requests.request(\"POST\", url, files = files, timeout=None, verify=False)\n",
    "    return response"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "Xfsfr5pWJWhA",
    "outputId": "9b755def-a74b-4d9b-ed88-b3e04d409c4a"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'predictions': [49.3517957612,\n",
      "                 16.0537680487,\n",
      "                 29.9966806992,\n",
      "                 11.0540791281,\n",
      "                 4.3146994394,\n",
      "                 24.9958974817,\n",
      "                 41.4482634256,\n",
      "                 5.2209053253,\n",
      "                 34.0136949358,\n",
      "                 36.1873684923,\n",
      "                 50.8947809964,\n",
      "                 20.0421857757,\n",
      "                 9.6950116159,\n",
      "                 9.480639738,\n",
      "                 40.1518692818,\n",
      "                 9.7036231712,\n",
      "                 6.1237465682,\n",
      "                 10.056552146,\n",
      "                 44.0038475882,\n",
      "                 47.1183072068,\n",
      "                 12.461933962,\n",
      "                 16.6101318422,\n",
      "                 27.8160542698,\n",
      "                 36.0255622646,\n",
      "                 38.4931692896,\n",
      "                 14.2099417805,\n",
      "                 16.896988332,\n",
      "                 31.9657138391,\n",
      "                 38.0365569138,\n",
      "                 39.4245134704,\n",
      "                 14.4808024271,\n",
      "                 12.0126024528,\n",
      "                 33.4010994497,\n",
      "                 40.1003240561,\n",
      "                 40.733235445,\n",
      "                 11.6870594646,\n",
      "                 17.652509238,\n",
      "                 17.2351142572,\n",
      "                 24.8541412161,\n",
      "                 27.5833423302,\n",
      "                 44.7060306308,\n",
      "                 52.9070029128,\n",
      "                 37.1115007542,\n",
      "                 14.7878605637,\n",
      "                 22.7482401955,\n",
      "                 17.9431223871,\n",
      "                 25.9549911015,\n",
      "                 34.208265063,\n",
      "                 36.5132315943,\n",
      "                 44.6587256825,\n",
      "                 52.178164728,\n",
      "                 56.555303114,\n",
      "                 66.4684511468,\n",
      "                 69.6478810742,\n",
      "                 71.1655753444,\n",
      "                 72.9466297476,\n",
      "                 18.9022819997,\n",
      "                 22.5256428689,\n",
      "                 28.7197297787,\n",
      "                 30.3314971396,\n",
      "                 31.9848330086,\n",
      "                 33.0515902758,\n",
      "                 10.5530073699,\n",
      "                 16.9610741413,\n",
      "                 23.480622287,\n",
      "                 26.8391357254,\n",
      "                 33.2905419731,\n",
      "                 39.2663407812,\n",
      "                 40.4721446734,\n",
      "                 14.9438958773,\n",
      "                 13.5707147914,\n",
      "                 24.4901323619,\n",
      "                 37.1624047672,\n",
      "                 13.7672912938,\n",
      "                 22.6366140311,\n",
      "                 20.859089745,\n",
      "                 31.8388916236,\n",
      "                 15.6646602982,\n",
      "                 25.6777340876,\n",
      "                 17.6743794168,\n",
      "                 15.1334604087,\n",
      "                 23.7456154928,\n",
      "                 30.1823359018,\n",
      "                 13.655073165,\n",
      "                 16.7695111144,\n",
      "                 31.7567129792,\n",
      "                 37.4742892358,\n",
      "                 17.793969029,\n",
      "                 33.0373319121,\n",
      "                 39.7227118404,\n",
      "                 40.7437637436,\n",
      "                 41.5730585816,\n",
      "                 19.3713108532,\n",
      "                 28.314366154,\n",
      "                 61.8917094588,\n",
      "                 48.4668523302,\n",
      "                 50.7221490105,\n",
      "                 54.1816440888,\n",
      "                 41.3833541851,\n",
      "                 13.1000853686,\n",
      "                 19.6741218176,\n",
      "                 32.1133858088,\n",
      "                 19.4537928671,\n",
      "                 39.136753767,\n",
      "                 47.3538184189,\n",
      "                 24.0696485286,\n",
      "                 39.0419577118,\n",
      "                 11.9056647252,\n",
      "                 19.6741218176,\n",
      "                 15.3225492017,\n",
      "                 27.7343375742,\n",
      "                 34.9813922312,\n",
      "                 37.0026600728,\n",
      "                 37.8945386474,\n",
      "                 35.1942334724,\n",
      "                 41.9039174297,\n",
      "                 45.9530238192,\n",
      "                 56.8711520696,\n",
      "                 59.3568371553,\n",
      "                 65.690170638,\n",
      "                 20.89629387,\n",
      "                 26.3959405526,\n",
      "                 30.684439904,\n",
      "                 23.7939408657,\n",
      "                 39.9119798972,\n",
      "                 24.8477635631,\n",
      "                 46.7689044092,\n",
      "                 70.5611806828,\n",
      "                 37.8975585522,\n",
      "                 34.2479759257,\n",
      "                 18.7471764937,\n",
      "                 20.734756538,\n",
      "                 36.1420078663,\n",
      "                 37.8335958637,\n",
      "                 15.9185667299,\n",
      "                 36.5888227881,\n",
      "                 39.9831200308,\n",
      "                 29.1208186522,\n",
      "                 27.3967494404,\n",
      "                 44.735588467,\n",
      "                 30.3697423733,\n",
      "                 15.741916084,\n",
      "                 43.3390272499,\n",
      "                 52.4079670788,\n",
      "                 55.5040866358,\n",
      "                 63.4677068048,\n",
      "                 51.323759174,\n",
      "                 33.29666452,\n",
      "                 21.5480417433,\n",
      "                 36.5468888578,\n",
      "                 33.5102228162,\n",
      "                 39.8583041906,\n",
      "                 38.4183749331,\n",
      "                 42.6372372086,\n",
      "                 59.8339752316,\n",
      "                 41.3275430728,\n",
      "                 32.7147191969,\n",
      "                 50.8531435407,\n",
      "                 15.5638037489,\n",
      "                 38.9267738512,\n",
      "                 38.3320663274,\n",
      "                 34.2523472036,\n",
      "                 42.3056337264,\n",
      "                 31.9666997898,\n",
      "                 40.9068730809,\n",
      "                 38.8462154754,\n",
      "                 38.6811687105,\n",
      "                 43.8622636916,\n",
      "                 32.1466530258,\n",
      "                 24.005485892,\n",
      "                 32.4943488778,\n",
      "                 33.7323857729,\n",
      "                 29.4687902946,\n",
      "                 36.4378620179,\n",
      "                 19.7372019769,\n",
      "                 32.1497970363,\n",
      "                 31.9937961966,\n",
      "                 30.3841593165,\n",
      "                 43.156226478,\n",
      "                 12.38456786,\n",
      "                 25.4003027348,\n",
      "                 33.4281682774,\n",
      "                 33.4328498195,\n",
      "                 25.3688005763,\n",
      "                 25.740035139,\n",
      "                 18.4918980047,\n",
      "                 39.178756645,\n",
      "                 20.3028411925,\n",
      "                 34.8288663505,\n",
      "                 9.3435098564,\n",
      "                 14.2477733286,\n",
      "                 31.9797102019,\n",
      "                 23.6562800875,\n",
      "                 29.378889519,\n",
      "                 47.8762274893,\n",
      "                 52.7108814646,\n",
      "                 42.1305895595,\n",
      "                 43.4915669202,\n",
      "                 13.7071758813,\n",
      "                 37.0569287288,\n",
      "                 30.0114010118,\n",
      "                 55.9253151033,\n",
      "                 12.1639562243,\n",
      "                 23.7624249177,\n",
      "                 13.4518904976,\n",
      "                 40.2732515316,\n",
      "                 45.8492369969,\n",
      "                 47.0876806829,\n",
      "                 23.8704244384,\n",
      "                 26.0948256991,\n",
      "                 15.4455034568,\n",
      "                 43.7581597103,\n",
      "                 37.2110404043,\n",
      "                 41.6903177649,\n",
      "                 28.4497309771,\n",
      "                 45.2948155558,\n",
      "                 27.0924285235,\n",
      "                 10.6016449769,\n",
      "                 28.3101120871,\n",
      "                 29.9768100009,\n",
      "                 10.1747559115,\n",
      "                 33.5915602999,\n",
      "                 36.8604832268,\n",
      "                 28.4785372844,\n",
      "                 29.7211178263,\n",
      "                 33.0345602186,\n",
      "                 32.594460793,\n",
      "                 25.5355089784,\n",
      "                 21.5259164584,\n",
      "                 31.8109402666,\n",
      "                 15.8963656027,\n",
      "                 26.1526106826,\n",
      "                 29.3479803099,\n",
      "                 25.5841032468,\n",
      "                 28.789732277,\n",
      "                 29.4699210352,\n",
      "                 36.8395093669,\n",
      "                 19.6259688138,\n",
      "                 32.22079238,\n",
      "                 31.980454836,\n",
      "                 29.5854978971,\n",
      "                 41.870470949,\n",
      "                 12.3865949195,\n",
      "                 25.3999442072,\n",
      "                 33.4298574936,\n",
      "                 33.4317604474,\n",
      "                 24.9716624003,\n",
      "                 25.8002746571,\n",
      "                 18.5395614806,\n",
      "                 38.350751127,\n",
      "                 19.8685264706,\n",
      "                 34.4886582078,\n",
      "                 9.3437442782,\n",
      "                 14.2727599389,\n",
      "                 31.9813028914,\n",
      "                 23.6570385111,\n",
      "                 29.4290971613,\n",
      "                 47.7691587613,\n",
      "                 51.9807401702,\n",
      "                 41.6486251512,\n",
      "                 43.6248012624,\n",
      "                 13.7091615722,\n",
      "                 37.036727082,\n",
      "                 30.1414913435,\n",
      "                 55.9701310433,\n",
      "                 12.1431202596,\n",
      "                 23.755233683,\n",
      "                 13.452166288,\n",
      "                 41.1416879222,\n",
      "                 45.9954955408,\n",
      "                 46.2416108938,\n",
      "                 24.0814592524,\n",
      "                 26.229418309,\n",
      "                 15.4443106633,\n",
      "                 43.7597799789,\n",
      "                 37.2137707292,\n",
      "                 42.0324288614,\n",
      "                 28.4401334712,\n",
      "                 45.2706425272,\n",
      "                 27.0940832659,\n",
      "                 10.5752518356,\n",
      "                 26.8978515015,\n",
      "                 30.2894597879,\n",
      "                 10.124610322,\n",
      "                 34.1556481946,\n",
      "                 37.8979998169,\n",
      "                 34.2505131974,\n",
      "                 18.6709480272,\n",
      "                 20.7691682851,\n",
      "                 35.9345031693,\n",
      "                 39.1650843359,\n",
      "                 15.7180808986,\n",
      "                 35.4980372823,\n",
      "                 39.8069589128,\n",
      "                 28.83029415,\n",
      "                 27.4122764399,\n",
      "                 44.6188532854,\n",
      "                 30.2440026352,\n",
      "                 15.712840881,\n",
      "                 43.4588857578,\n",
      "                 52.1361825344,\n",
      "                 55.5233505953,\n",
      "                 63.0532007284,\n",
      "                 51.652811595,\n",
      "                 33.2617356658,\n",
      "                 20.3796281346,\n",
      "                 36.4361314331,\n",
      "                 34.4788331748,\n",
      "                 39.6976631773,\n",
      "                 38.120638512,\n",
      "                 43.5164845828,\n",
      "                 59.834940498,\n",
      "                 41.3066381604,\n",
      "                 32.701674311,\n",
      "                 51.1371524946,\n",
      "                 15.5617215314,\n",
      "                 38.7817563642,\n",
      "                 38.277321933,\n",
      "                 34.179538538,\n",
      "                 42.3433687478,\n",
      "                 31.8075204656,\n",
      "                 41.0198092497,\n",
      "                 38.9100540582,\n",
      "                 38.6823821883,\n",
      "                 44.028558408,\n",
      "                 32.1267547484,\n",
      "                 23.9685092941,\n",
      "                 33.3505332798,\n",
      "                 33.7323512991]}\n"
     ]
    }
   ],
   "source": [
    "# %%timeit\n",
    "scoring_data = pd.read_csv(\"./mlops-examples/drum_overview/data/concrete_test.csv\")\n",
    "predictions = score(scoring_data).json() ## score entire dataset but only show first 5 records\n",
    "pprint(predictions)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "5uVmEOQhJWhA",
    "outputId": "c0150d0e-60d4-4ec4-f549-f2680d4d6ac2"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "b'{\"message\":\"OK\"}\\n'"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "requests.request(\"GET\", \"http://0.0.0.0:6789/\").content"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "Sdkw1wYoJWhA",
    "outputId": "2deb7b16-0714-4638-807d-5e5e8682aaab"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[b'Detected REST server mode - this is an advanced option\\n',\n",
       " b'Detected /content/mlops-examples/drum_overview/custom_model_reg/custom.py .. trying to load hooks\\n',\n",
       " b'\\x1b[32m \\x1b[0m\\n',\n",
       " b'\\x1b[32m \\x1b[0m\\n',\n",
       " b'\\x1b[32m============================================================\\x1b[0m\\n',\n",
       " b'\\x1b[32mComponent: Prediction Server\\x1b[0m\\n',\n",
       " b'\\x1b[32mOutput:\\x1b[0m\\n',\n",
       " b'\\x1b[32m------------------------------------------------------------\\x1b[0m\\n']"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "inference_server.terminate()\n",
    "inference_server.stdout.readlines()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "id": "w6SQwG4vJWhA"
   },
   "outputs": [],
   "source": [
    "#Stop the flask server\n",
    "# %%sh fuser -n tcp -k 6789\n",
    "! lsof -i tcp:6789 | grep -v PID | awk '{print $2}' | xargs kill"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "xxFKPaytJWhA"
   },
   "source": [
    "## Value Prop\n",
    "\n",
    "One may ask, what is the benefit to be had here?  Well, first off, there is not need for me to write an api to get the model up and running.  Second, DRUM allows me to abstract the framework away (provided I'm using one that is natively supported, or I can write enough python so that DRUM understands how to hook up to the model.  \n",
    "\n",
    "For example, I could hot swap models as I see fit.\n",
    "\n",
    "While we will run through several other frameworks with in `score` you can bet they are supported in `server` mode as well!"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "collapsed_sections": [],
   "name": "220428_MQ_DRUM_Main_Script.ipynb",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}