24Feb2022 - DRUM_Lab.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ePVVRJ_7FkEg"
   },
   "source": [
    "<a href=\"https://colab.research.google.com/github/datarobot-community/DRU-MLOps/blob/master/24Feb2022 - DRUM_Lab.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "VLxOyJ2xMrlL"
   },
   "source": [
    "# Custom Model Integration with MLOps - DRUM Laboratory"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "_OUbruxUMrlM"
   },
   "source": [
    "Welcome to the MLOps III DRUM Lab! \n",
    " \n",
    " In this notebook we will\n",
    "\n",
    "* Build a simple regression model using Scikit-Learn\n",
    "* Use DRUM to test & validate the model\n",
    "* Use DRUM to score data in batch mode\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "tFS-eBOmsi-F"
   },
   "source": [
    "## Use case to be addressed:\n",
    "\n",
    "We will build a regression model to predict depression scores in older adults. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "VeC6ZQmYnIYl"
   },
   "source": [
    "Let's begin by uploading a few resources we will need:\n",
    "\n",
    "1. Training set: **mental_health_training.csv**\n",
    "2. Scoring set: **mental_health_inference.csv**\n",
    "3. Requirements file: **colab_requirements.txt**\n",
    "4. File with hooks used by the model: **custom.py**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "-IDgpv8NMs7r"
   },
   "outputs": [],
   "source": [
    "from google.colab import files\n",
    "uploaded = files.upload()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "4sN_vquZolv_"
   },
   "outputs": [],
   "source": [
    "!ls"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will now create parameters to pass the names of the training and inference datasets to the next cells. As per best practices, the names of these parameters will be written in UPPERCASE.\n",
    "\n",
    "First we will create a parameter for the name of the training dataset:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "TRAINING = 'mental_health_training.csv'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will define an environment variable to pass the name of the inference dataset to the DRUM command:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "INFERENCE = 'mental_health_inference.csv'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!export INFERENCE"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "1sdCt94rqk17"
   },
   "source": [
    "Let's install the Python modules we need using the requirements file:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "x86AAdVeMjl6"
   },
   "outputs": [],
   "source": [
    "!pip install -r colab_requirements.txt -q"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "os.environ['PYTHONWARNINGS'] = 'ignore'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "AyiVy3Zv3eEP"
   },
   "source": [
    "# 1.- Model Training\n",
    "\n",
    "We will now build a very simple Scikit-Learn Regression model using the training data that we loaded into the Colab environment."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "pdT60Pcp38kl"
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "from sklearn.ensemble import RandomForestRegressor\n",
    "import pickle\n",
    "import datetime\n",
    "\n",
    "## load data\n",
    "df = pd.read_csv(TRAINING)\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "vAkNyQLWMrlN"
   },
   "outputs": [],
   "source": [
    "## set features and target\n",
    "X = df.drop('depression_score', axis=1)\n",
    "y = df['depression_score']\n",
    "\n",
    "## train the model\n",
    "rf = RandomForestRegressor(n_estimators = 20, random_state = 801)\n",
    "rf.fit(X,y)\n",
    "\n",
    "## serialize the model\n",
    "with open('rf.pkl', 'wb') as pkl:\n",
    "    pickle.dump(rf, pkl)\n",
    "\n",
    "print(\"Done!\")    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "DKi7ywP8L192"
   },
   "source": [
    "# 2.- Model Testing\n",
    "\n",
    "We will now use DRUM to test how the model performs by computing latency times and memory usage for several different test case sizes. A report is generated after this process is completed.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "FM60vZNHL1cu"
   },
   "outputs": [],
   "source": [
    "!drum perf-test "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "79e-llL0L7Aj"
   },
   "source": [
    "# 3.- Model Validation: Handling of Missing Values\n",
    "\n",
    "We will now validate the model to detect and address issues before deployment. It’s highly encouraged that you run these tests, which are the same ones that DataRobot performs automatically before deploying models.\n",
    "\n",
    "Especifically, DRUM will test null values imputation by setting each feature in the dataset to \"missing\" and then feeding the features to the model. We will send the results to **validation.log**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "ms1EF-eeL1MT"
   },
   "outputs": [],
   "source": [
    "!drum validation "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "I_hENDrAHJkR"
   },
   "outputs": [],
   "source": [
    "!cat validation.log"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "t4p0zDG-VWJP"
   },
   "source": [
    "# 4.- Batch Scoring with DRUM\n",
    "<a id=\"setup_complete\"></a>\n",
    "\n",
    "We want to use our model to make predictions; to do this, we'll leverage DRUM and its ability to natively handle our Scikit-Learn model. All we need to do is tell DRUM where the model resides and what data we wish to score.  \n",
    "\n",
    "DRUM provides native support for many frameworks. To use DRUM with model frameworks that are not supported out-of-the box, we'll just need to create some custom hooks so DRUM.  In this example, we'll explain some very simple custom hooks and provide links to more complex examples.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "C_OOeqEx6hqH"
   },
   "outputs": [],
   "source": [
    "!drum score "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "fNNrCQvix1fm"
   },
   "source": [
    "Let's have a look at the predictions:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!cat predictions.csv"
   ]
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "collapsed_sections": [],
   "name": "MLOps III - DRUM",
   "provenance": [],
   "toc_visible": true
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}