Exercises/Ex.5 - Data Science with Jupyter.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src = http://i1377.photobucket.com/albums/ah57/drmingle1/Jupyter%20-%20Healthcare%20Data%20Science/BosDataFestival2015_zpsqpbcvzw8.png alt=\"Matrix\" style=\"width: 600px;\"/>\n",
    "\n",
    "\n",
    "# Finding Type II Diabetes In A Patient Population\n",
    "Given a year of EHR data for patients without Diabetes, we predict which patients will be diagnosed with Diabetes in the next year."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "***************************\n",
    "#Excercise 5\n",
    "\n",
    "Choose between A and B.\n",
    "***************************"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Data Scientist A**\n",
    "\n",
    "#PHASE V: Evaluation\n",
    "\n",
    "######Evaluate Results\n",
    "This step is meant to evaluate the model from a business perspective and seeks to determine if there is some business reason why this model is deficient. \n",
    "\n",
    "1. Assessment of Data Science results\n",
    "    * Summarize assessment results in terms of business criteria:\n",
    "2. Approved models\n",
    "    * A list of generated models which are approved for business reasons:\n",
    "\n",
    "######Review Process\n",
    "This step is a quality assurance step insuring we used all the the appropriate items in the models.\n",
    "\n",
    "1. Review of process\n",
    "    * Summarize the process review and highlight activities that have been missed and those that should be repeated.\n",
    "\n",
    "######Determine Next Steps\n",
    "Team decides whether to finish the project and move on to deployment, initiate further iterations, or set up new data science projects. \n",
    "\n",
    "1. List of possible actions\n",
    "    * Reasons for each option:\n",
    "    * Reasons against each option:\n",
    "    \n",
    "2. Decisions\n",
    "    * How to proceed, along with rationale:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "**===================================================================================================**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Data Scientist B**\n",
    "\n",
    "#PHASE V: Evaluation\n",
    "\n",
    "######Evaluate Results\n",
    "1. Assessment of Data Science results\n",
    "    *  Our organization felt that we were unable to capture most of the diabetic type II population through standard channels and that by developing a predictive model which could classify patients with a high-likelihood of type II diabetis that could reduce cost and increase quality of care to patients. Currently our organization does not have any predictive model built for this. However, the models we constructed are able to produce a result of 0.31490 which is far lower than the 0.40 benchmark set by the business to be useful.\n",
    "\n",
    "2. Approved models\n",
    "\n",
    "Below is our list of approved models: \n",
    "\n",
    "***Graident Boosting Machine***\n",
    "\n",
    "GBM Model | CV Error | Folds | Trees | Depth | Shrinkage | Bag Frac | Node Size | Tolerance\n",
    "-|-|-|-|-|-|-|-\n",
    "    gbm 10_5 0.003_0.80_30 | 0.31210 | 10|7,550| 5 | 0.003|0.80|30|0.001\n",
    "    gbm10_5_0.003_0.80_30_tolhalf_ext |\t0.31153|\t10|\t8,500|\t5|\t0.003|\t0.80|\t30|\t0.0005\n",
    "    gbm10_5_0.0025_0.80_30\t|0.31319\t|10\t|8,000|\t5|\t0.003|\t0.80|\t30|\t0.001\n",
    "    gbm10_5_0.0025_0.80_30_ext|\t0.31312\t|10|\t7,750|\t5|\t0.0025|\t0.80|\t30|\t0.001\n",
    "    gbm10_5_0.0025_0.80_30_tolhalf\t|0.31122\t|10\t|11,000|\t5|\t0.0025|\t0.80|\t30|\t0.0005\n",
    "    gbm10_5_0.0025_0.80_30_tolhalf_ext\t|0.31139|\t10\t|10,450|\t5|\t0.0025|\t0.80|\t30|\t0.0005\n",
    "    gbm20_5_0.002_0.80_10\t|0.31049\t|20|\t13,450|\t5\t|0.002|\t0.80|\t10|\t0.001\n",
    "    gbm20_5_0.002_0.80_15\t|0.31040\t|20\t|12,950\t|5\t|0.002\t|0.80\t|15\t|0.001\n",
    "    gbm20_5_0.0025_0.80_20\t|0.30878\t|20\t|12,300\t|5\t|0.0025\t|0.80\t|20\t|0.001\n",
    "    gbm20_5_0.0025_0.80_40\t|0.31040\t|20\t|10,500\t|5\t|0.0025\t|0.80\t|40\t|0.001\n",
    "    gbm20_6_0.002_0.80_30\t|0.30931\t|20\t|12,200\t|6\t|0.002\t|0.80\t|30\t|0.001\n",
    "\n",
    "    \n",
    "***Random Forest***\n",
    "\n",
    "RF Model | OOB MSE | Trees | Node Size\n",
    "-|-|-|-\n",
    "    RF1 | 0.10055| 15,000 | 5\n",
    "    RF5\t|0.10076|\t30,000|\t15\n",
    "    RF2\t|0.10089|\t15,000|\t20\n",
    "    RF3\t|0.10102|\t15,000|\t40\n",
    "\n",
    "######Review Process\n",
    "* We have gone back through each step to confirm that we were able to reproduce the results described. No activities have been missed or repeated. \n",
    "    \n",
    "######Determine Next Steps\n",
    "\n",
    "1. List of possible actions\n",
    "    * Reasons for each option:\n",
    "        * We could move forward with the completion of the Type II Diabetis model\n",
    "        * We could stop the Type II Diabetis model\n",
    "        * We could develop faster learning models\n",
    "    * Reasons against each option:\n",
    "        * Moving forward could mean we miss a significantly more accurate model\n",
    "        * Stopping the Type II model at this point would not accomplish the business goal of uncovering these population, reducing cost, and improving healthcare quality.\n",
    "        * The business concern was not about building faster models, but meeting the accuracy benchmark of 0.40 logloss. Developing faster learning models would take additional company resources and with no guarantee of besting the predictive accuracy of the model already developed.\n",
    "    \n",
    "2. Decisions\n",
    "    * We recommend placing in production our Type II Diabetis model as it meets both the business and data science objectives set out at the start of the project. This model will help to uncover populations with Type II Diabetis which have gone underdiagnosed and our organization can reduce cost and improve patient care by discovering patients in this subclassification.\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.4.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}