python/wine-quality-classification_logistic-regression.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div style=\"text-align: center; line-height: 0; padding-top: 2px;\">\n",
    "  <img src=\"http://bit.ly/pyDA4vodafone01_qh\" alt=\"Quantia Consulting\" style=\"width: 480px; height: 200px\">\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Dataset\n",
    "[Wine Quality Repository](https://archive.ics.uci.edu/ml/datasets/wine+quality)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline \n",
    "\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import scipy.stats as stats\n",
    "import matplotlib.pyplot as plt\n",
    "import sklearn\n",
    "import statsmodels.api as sm\n",
    "\n",
    "import seaborn as sns\n",
    "sns.set_style(\"whitegrid\")\n",
    "sns.set_context(\"poster\")\n",
    "\n",
    "# special matplotlib argument for improved plots\n",
    "from matplotlib import rcParams"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The Dataset can be loaded directly from the library sklearn.\n",
    "\n",
    "The loaded dataset has four different attributes:\n",
    "* the `DESCR` that contains the description of the dataset itself;\n",
    "* the `feature_names` that contains the name of the columns;\n",
    "* the `data` that contains the data.\n",
    "* the `target` that contains the data to be predicted.\n",
    "\n",
    "A brief description of the characteristics can be found in the DESCR file. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Wine Data Database\n",
      "====================\n",
      "\n",
      "Notes\n",
      "-----\n",
      "Data Set Characteristics:\n",
      "    :Number of Instances: 178 (50 in each of three classes)\n",
      "    :Number of Attributes: 13 numeric, predictive attributes and the class\n",
      "    :Attribute Information:\n",
      " \t\t- 1) Alcohol\n",
      " \t\t- 2) Malic acid\n",
      " \t\t- 3) Ash\n",
      "\t\t- 4) Alcalinity of ash  \n",
      " \t\t- 5) Magnesium\n",
      "\t\t- 6) Total phenols\n",
      " \t\t- 7) Flavanoids\n",
      " \t\t- 8) Nonflavanoid phenols\n",
      " \t\t- 9) Proanthocyanins\n",
      "\t\t- 10)Color intensity\n",
      " \t\t- 11)Hue\n",
      " \t\t- 12)OD280/OD315 of diluted wines\n",
      " \t\t- 13)Proline\n",
      "        \t- class:\n",
      "                - class_0\n",
      "                - class_1\n",
      "                - class_2\n",
      "\t\t\n",
      "    :Summary Statistics:\n",
      "    \n",
      "    ============================= ==== ===== ======= =====\n",
      "                                   Min   Max   Mean     SD\n",
      "    ============================= ==== ===== ======= =====\n",
      "    Alcohol:                      11.0  14.8    13.0   0.8\n",
      "    Malic Acid:                   0.74  5.80    2.34  1.12\n",
      "    Ash:                          1.36  3.23    2.36  0.27\n",
      "    Alcalinity of Ash:            10.6  30.0    19.5   3.3\n",
      "    Magnesium:                    70.0 162.0    99.7  14.3\n",
      "    Total Phenols:                0.98  3.88    2.29  0.63\n",
      "    Flavanoids:                   0.34  5.08    2.03  1.00\n",
      "    Nonflavanoid Phenols:         0.13  0.66    0.36  0.12\n",
      "    Proanthocyanins:              0.41  3.58    1.59  0.57\n",
      "    Colour Intensity:              1.3  13.0     5.1   2.3\n",
      "    Hue:                          0.48  1.71    0.96  0.23\n",
      "    OD280/OD315 of diluted wines: 1.27  4.00    2.61  0.71\n",
      "    Proline:                       278  1680     746   315\n",
      "    ============================= ==== ===== ======= =====\n",
      "\n",
      "    :Missing Attribute Values: None\n",
      "    :Class Distribution: class_0 (59), class_1 (71), class_2 (48)\n",
      "    :Creator: R.A. Fisher\n",
      "    :Donor: Michael Marshall (MARSHALL%PLU@io.arc.nasa.gov)\n",
      "    :Date: July, 1988\n",
      "\n",
      "This is a copy of UCI ML Wine recognition datasets.\n",
      "https://archive.ics.uci.edu/ml/machine-learning-databases/wine/wine.data\n",
      "\n",
      "The data is the results of a chemical analysis of wines grown in the same\n",
      "region in Italy by three different cultivators. There are thirteen different\n",
      "measurements taken for different constituents found in the three types of\n",
      "wine.\n",
      "\n",
      "Original Owners: \n",
      "\n",
      "Forina, M. et al, PARVUS - \n",
      "An Extendible Package for Data Exploration, Classification and Correlation. \n",
      "Institute of Pharmaceutical and Food Analysis and Technologies,\n",
      "Via Brigata Salerno, 16147 Genoa, Italy.\n",
      "\n",
      "Citation:\n",
      "\n",
      "Lichman, M. (2013). UCI Machine Learning Repository\n",
      "[http://archive.ics.uci.edu/ml]. Irvine, CA: University of California,\n",
      "School of Information and Computer Science. \n",
      "\n",
      "References\n",
      "----------\n",
      "(1) \n",
      "S. Aeberhard, D. Coomans and O. de Vel, \n",
      "Comparison of Classifiers in High Dimensional Settings, \n",
      "Tech. Rep. no. 92-02, (1992), Dept. of Computer Science and Dept. of \n",
      "Mathematics and Statistics, James Cook University of North Queensland. \n",
      "(Also submitted to Technometrics). \n",
      "\n",
      "The data was used with many others for comparing various \n",
      "classifiers. The classes are separable, though only RDA \n",
      "has achieved 100% correct classification. \n",
      "(RDA : 100%, QDA 99.4%, LDA 98.9%, 1NN 96.1% (z-transformed data)) \n",
      "(All results using the leave-one-out technique) \n",
      "\n",
      "(2) \n",
      "S. Aeberhard, D. Coomans and O. de Vel, \n",
      "\"THE CLASSIFICATION PERFORMANCE OF RDA\" \n",
      "Tech. Rep. no. 92-01, (1992), Dept. of Computer Science and Dept. of \n",
      "Mathematics and Statistics, James Cook University of North Queensland. \n",
      "(Also submitted to Journal of Chemometrics). \n",
      "\n"
     ]
    }
   ],
   "source": [
    "from sklearn.datasets import load_wine\n",
    "wine = load_wine()\n",
    "print(wine.DESCR)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let’s create a pandas dataframe from the data.\n",
    "\n",
    "We can do that by calling the method `pd.DataFrame()` on the `boston.data`. \n",
    "\n",
    "Let's than check the first 5 lines using the `head()` method.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "      0     1     2     3      4     5     6     7     8     9     10    11  \\\n",
      "0  14.23  1.71  2.43  15.6  127.0  2.80  3.06  0.28  2.29  5.64  1.04  3.92   \n",
      "1  13.20  1.78  2.14  11.2  100.0  2.65  2.76  0.26  1.28  4.38  1.05  3.40   \n",
      "2  13.16  2.36  2.67  18.6  101.0  2.80  3.24  0.30  2.81  5.68  1.03  3.17   \n",
      "3  14.37  1.95  2.50  16.8  113.0  3.85  3.49  0.24  2.18  7.80  0.86  3.45   \n",
      "4  13.24  2.59  2.87  21.0  118.0  2.80  2.69  0.39  1.82  4.32  1.04  2.93   \n",
      "\n",
      "       12  \n",
      "0  1065.0  \n",
      "1  1050.0  \n",
      "2  1185.0  \n",
      "3  1480.0  \n",
      "4   735.0  \n"
     ]
    }
   ],
   "source": [
    "df = pd.DataFrame(wine.data)\n",
    "print(df.head())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's now convert the index to the column names exploiting the `feature_names`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "   alcohol  malic_acid   ash  alcalinity_of_ash  magnesium  total_phenols  \\\n",
      "0    14.23        1.71  2.43               15.6      127.0           2.80   \n",
      "1    13.20        1.78  2.14               11.2      100.0           2.65   \n",
      "2    13.16        2.36  2.67               18.6      101.0           2.80   \n",
      "3    14.37        1.95  2.50               16.8      113.0           3.85   \n",
      "4    13.24        2.59  2.87               21.0      118.0           2.80   \n",
      "\n",
      "   flavanoids  nonflavanoid_phenols  proanthocyanins  color_intensity   hue  \\\n",
      "0        3.06                  0.28             2.29             5.64  1.04   \n",
      "1        2.76                  0.26             1.28             4.38  1.05   \n",
      "2        3.24                  0.30             2.81             5.68  1.03   \n",
      "3        3.49                  0.24             2.18             7.80  0.86   \n",
      "4        2.69                  0.39             1.82             4.32  1.04   \n",
      "\n",
      "   od280/od315_of_diluted_wines  proline  \n",
      "0                          3.92   1065.0  \n",
      "1                          3.40   1050.0  \n",
      "2                          3.17   1185.0  \n",
      "3                          3.45   1480.0  \n",
      "4                          2.93    735.0  \n"
     ]
    }
   ],
   "source": [
    "df.columns = wine.feature_names\n",
    "print(df.head())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's now add the target column to the dataframe and split it into target and train"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
      " 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1\n",
      " 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1\n",
      " 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2\n",
      " 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2]\n"
     ]
    }
   ],
   "source": [
    "print(wine.target)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn import preprocessing\n",
    "\n",
    "binarized_target = preprocessing.binarize(wine.target.reshape(-1, 1), threshold=0.5, copy=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "df['quality'] = binarized_target\n",
    "X = df.drop('quality', axis=1)\n",
    "Y = df.quality\n",
    "\n",
    "X_train, X_test, Y_train, Y_test = train_test_split(X, Y, test_size = 0.2, random_state = 5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's finally run the logistic regression."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.linear_model import LogisticRegression\n",
    "\n",
    "lr = LogisticRegression(random_state=5)\n",
    "lr.fit(X_train, Y_train)\n",
    "Y_pred = lr.predict(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy score: 0.9166666666666666\n",
      "Confusion matrix:\n",
      "[[15  2]\n",
      " [ 1 18]]\n"
     ]
    }
   ],
   "source": [
    "from sklearn.metrics import accuracy_score, confusion_matrix\n",
    "\n",
    "print(\"Accuracy score: {}\".format(accuracy_score(Y_test, Y_pred)))\n",
    "print(\"Confusion matrix:\\n{}\".format(confusion_matrix(Y_test, Y_pred)))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}