Books/Machine Learning Mastery With Python - Jason Brownlee/Chapter14.AutomateMachineLearningWorkflowsWithPipelines.ipynb

{
  "cells": [
    {
      "metadata": {
        "collapsed": true
      },
      "cell_type": "markdown",
      "source": "Date: 2018/11/10"
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "1. How to use pipelines to minimize data leakage.\n2. How to construct a data preparation and modeling pipeline.\n3. How to construct a feature extraction and modeling pipeline."
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "### 14.1 Automating Machine Learning Workflows\nPython scikit-learn provides a Pipeline utility to help automate machine learning workflows.\nPipelines work by allowing for a linear sequence of data transforms to be chained together culminating in a modeling process that can be evaluated.\n\nThe goal is to ensure that all of the steps in the pipeline are constrained to the data available for the evaluation, such as the training dataset or each fold of the cross validation procedure."
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "### 14.2 Data Preparation and Modeling Pipeline\nData preparation is one easy way to leak knowledge of the whole training dataset to the algorithm.\n For example, preparing your data using normalization or standardization on the entire training dataset before learning would not be a valid test because the training dataset would have been influenced by the scale of the data in the test set.\n Pipelines help you prevent data leakage in your test harness by ensuring that data preparation like standardization is constrained to each fold of your cross validation procedure. \n \n The pipeline is defined with two steps:\n1. Standardize the data.\n2. Learn a Linear Discriminant Analysis model."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "# Create a pipeline that standardizes the data then creates a model\nfrom pandas import read_csv\nfrom sklearn.model_selection import KFold\nfrom sklearn.model_selection import cross_val_score\nfrom sklearn.preprocessing import StandardScaler\nfrom sklearn.pipeline import Pipeline\nfrom sklearn.discriminant_analysis import LinearDiscriminantAnalysis",
      "execution_count": 1,
      "outputs": []
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "# load data\nurl = \"http://ftp.ics.uci.edu/pub/machine-learning-databases/pima-indians-diabetes/pima-indians-diabetes.data\"\nnames=[\"preg\", \"plas\", \"pres\", \"skin\", \"test\", \"mass\", \"pedi\", \"age\", \"class\"] \ndataframe = read_csv(url, names=names)\narray = dataframe.values\nX = array[:,0:8]\nY = array[:,8]",
      "execution_count": 2,
      "outputs": []
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "# create pipeline\nestimators = []\nestimators.append(( 'standardize' , StandardScaler()))\nestimators.append(( 'lda' , LinearDiscriminantAnalysis()))\nmodel = Pipeline(estimators)",
      "execution_count": 4,
      "outputs": []
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "# evaluate pipeline\nnum_folds = 10\nseed = 7\nkfold = KFold(n_splits=num_folds, random_state=seed)\nresults = cross_val_score(model, X, Y, cv=kfold)\nprint(results.mean())",
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "stream",
          "text": "0.773462064251538\n",
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "Also notice how the Pipeline itself is treated like an estimator and is evaluated in its entirety by the k-fold cross validation procedure."
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "### 14.3 Feature Extraction and Modeling Pipeline\nFeature extraction is another procedure that is susceptible to data leakage.\nLike data preparation, feature extraction procedures must be restricted to the data in your training dataset.\n\nThe pipeline provides a handy tool called the FeatureUnion which allows the results of multiple feature selection and extraction procedures to be combined into a larger dataset on which a model can be trained. \nImportantly, all the feature extraction and the feature union occurs within each fold of the cross validation procedure. "
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "1. Feature Extraction with Principal Component Analysis (3 features).\n2. Feature Extraction with Statistical Selection (6 features).\n3. Feature Union.\n4. Learn a Logistic Regression Model."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "# Create a pipeline that extracts features from the data then creates a model\nfrom pandas import read_csv\nfrom sklearn.model_selection import KFold\nfrom sklearn.model_selection import cross_val_score\nfrom sklearn.pipeline import Pipeline\nfrom sklearn.pipeline import FeatureUnion\nfrom sklearn.linear_model import LogisticRegression\nfrom sklearn.decomposition import PCA\nfrom sklearn.feature_selection import SelectKBest",
      "execution_count": 6,
      "outputs": []
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "# load data\nurl = \"http://ftp.ics.uci.edu/pub/machine-learning-databases/pima-indians-diabetes/pima-indians-diabetes.data\"\nnames=[\"preg\", \"plas\", \"pres\", \"skin\", \"test\", \"mass\", \"pedi\", \"age\", \"class\"] \ndataframe = read_csv(url, names=names)\narray = dataframe.values\nX = array[:,0:8]\nY = array[:,8]",
      "execution_count": 7,
      "outputs": []
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "# create feature union\nfeatures = []\nfeatures.append(( 'pca' , PCA(n_components=3)))\nfeatures.append(( 'select_best' , SelectKBest(k=6)))\nfeature_union = FeatureUnion(features)",
      "execution_count": 9,
      "outputs": []
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "# create pipeline\nestimators = []\nestimators.append(( 'feature_union' , feature_union))\nestimators.append(( 'logistic' , LogisticRegression()))\nmodel = Pipeline(estimators)",
      "execution_count": 10,
      "outputs": []
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "# evaluate pipeline\nnum_folds = 10\nseed = 7\nkfold = KFold(n_splits=num_folds, random_state=seed)\nresults = cross_val_score(model, X, Y, cv=kfold)\nprint(results.mean())",
      "execution_count": 11,
      "outputs": [
        {
          "output_type": "stream",
          "text": "0.7760423786739576\n",
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "Notice how the FeatureUnion is it’s own Pipeline that in turn is a single step in the final Pipeline used to feed Logistic Regression. This might get you thinking about how you can start embedding pipelines within pipelines."
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "#### 14.4 Summary\n- Data preparation and modeling constrained to each fold of the cross validation procedure.\n- Feature extraction and feature union constrained to each fold of the cross validation procedure."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "",
      "execution_count": null,
      "outputs": []
    }
  ],
  "metadata": {
    "kernelspec": {
      "name": "python36",
      "display_name": "Python 3.6",
      "language": "python"
    },
    "language_info": {
      "mimetype": "text/x-python",
      "nbconvert_exporter": "python",
      "name": "python",
      "pygments_lexer": "ipython3",
      "version": "3.6.6",
      "file_extension": ".py",
      "codemirror_mode": {
        "version": 3,
        "name": "ipython"
      }
    }
  },
  "nbformat": 4,
  "nbformat_minor": 2
}