Machine Learning/3.1 Random Forest/RandomForestRegression.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 构建随机森林回归模型"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 0.import工具库"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "from sklearn import preprocessing\n",
    "from sklearn.ensemble import RandomForestRegressor\n",
    "from sklearn.datasets import load_boston"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 1.加载数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "boston_house = load_boston()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "boston_feature_name = boston_house.feature_names\n",
    "boston_features = boston_house.data\n",
    "boston_target = boston_house.target"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['CRIM', 'ZN', 'INDUS', 'CHAS', 'NOX', 'RM', 'AGE', 'DIS', 'RAD',\n",
       "       'TAX', 'PTRATIO', 'B', 'LSTAT'],\n",
       "      dtype='|S7')"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "boston_feature_name"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Boston House Prices dataset\n",
      "===========================\n",
      "\n",
      "Notes\n",
      "------\n",
      "Data Set Characteristics:  \n",
      "\n",
      "    :Number of Instances: 506 \n",
      "\n",
      "    :Number of Attributes: 13 numeric/categorical predictive\n",
      "    \n",
      "    :Median Value (attribute 14) is usually the target\n",
      "\n",
      "    :Attribute Information (in order):\n",
      "        - CRIM     per capita crime rate by town\n",
      "        - ZN       proportion of residential land zoned for lots over 25,000 sq.ft.\n",
      "        - INDUS    proportion of non-retail business acres per town\n",
      "        - CHAS     Charles River dummy variable (= 1 if tract bounds river; 0 otherwise)\n",
      "        - NOX      nitric oxides concentration (parts per 10 million)\n",
      "        - RM       average number of rooms per dwelling\n",
      "        - AGE      proportion of owner-occupied units built prior to 1940\n",
      "        - DIS      weighted distances to five Boston employment centres\n",
      "        - RAD      index of accessibility to radial highways\n",
      "        - TAX      full-value property-tax rate per $10,000\n",
      "        - PTRATIO  pupil-teacher ratio by town\n",
      "        - B        1000(Bk - 0.63)^2 where Bk is the proportion of blacks by town\n",
      "        - LSTAT    % lower status of the population\n",
      "        - MEDV     Median value of owner-occupied homes in $1000's\n",
      "\n",
      "    :Missing Attribute Values: None\n",
      "\n",
      "    :Creator: Harrison, D. and Rubinfeld, D.L.\n",
      "\n",
      "This is a copy of UCI ML housing dataset.\n",
      "http://archive.ics.uci.edu/ml/datasets/Housing\n",
      "\n",
      "\n",
      "This dataset was taken from the StatLib library which is maintained at Carnegie Mellon University.\n",
      "\n",
      "The Boston house-price data of Harrison, D. and Rubinfeld, D.L. 'Hedonic\n",
      "prices and the demand for clean air', J. Environ. Economics & Management,\n",
      "vol.5, 81-102, 1978.   Used in Belsley, Kuh & Welsch, 'Regression diagnostics\n",
      "...', Wiley, 1980.   N.B. Various transformations are used in the table on\n",
      "pages 244-261 of the latter.\n",
      "\n",
      "The Boston house-price data has been used in many machine learning papers that address regression\n",
      "problems.   \n",
      "     \n",
      "**References**\n",
      "\n",
      "   - Belsley, Kuh & Welsch, 'Regression diagnostics: Identifying Influential Data and Sources of Collinearity', Wiley, 1980. 244-261.\n",
      "   - Quinlan,R. (1993). Combining Instance-Based and Model-Based Learning. In Proceedings on the Tenth International Conference of Machine Learning, 236-243, University of Massachusetts, Amherst. Morgan Kaufmann.\n",
      "   - many more! (see http://archive.ics.uci.edu/ml/datasets/Housing)\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print(boston_house.DESCR)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[  6.32000000e-03,   1.80000000e+01,   2.31000000e+00,\n",
       "          0.00000000e+00,   5.38000000e-01,   6.57500000e+00,\n",
       "          6.52000000e+01,   4.09000000e+00,   1.00000000e+00,\n",
       "          2.96000000e+02,   1.53000000e+01,   3.96900000e+02,\n",
       "          4.98000000e+00],\n",
       "       [  2.73100000e-02,   0.00000000e+00,   7.07000000e+00,\n",
       "          0.00000000e+00,   4.69000000e-01,   6.42100000e+00,\n",
       "          7.89000000e+01,   4.96710000e+00,   2.00000000e+00,\n",
       "          2.42000000e+02,   1.78000000e+01,   3.96900000e+02,\n",
       "          9.14000000e+00],\n",
       "       [  2.72900000e-02,   0.00000000e+00,   7.07000000e+00,\n",
       "          0.00000000e+00,   4.69000000e-01,   7.18500000e+00,\n",
       "          6.11000000e+01,   4.96710000e+00,   2.00000000e+00,\n",
       "          2.42000000e+02,   1.78000000e+01,   3.92830000e+02,\n",
       "          4.03000000e+00],\n",
       "       [  3.23700000e-02,   0.00000000e+00,   2.18000000e+00,\n",
       "          0.00000000e+00,   4.58000000e-01,   6.99800000e+00,\n",
       "          4.58000000e+01,   6.06220000e+00,   3.00000000e+00,\n",
       "          2.22000000e+02,   1.87000000e+01,   3.94630000e+02,\n",
       "          2.94000000e+00],\n",
       "       [  6.90500000e-02,   0.00000000e+00,   2.18000000e+00,\n",
       "          0.00000000e+00,   4.58000000e-01,   7.14700000e+00,\n",
       "          5.42000000e+01,   6.06220000e+00,   3.00000000e+00,\n",
       "          2.22000000e+02,   1.87000000e+01,   3.96900000e+02,\n",
       "          5.33000000e+00]])"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "boston_features[:5,:]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 24. ,  21.6,  34.7,  33.4,  36.2,  28.7,  22.9,  27.1,  16.5,\n",
       "        18.9,  15. ,  18.9,  21.7,  20.4,  18.2,  19.9,  23.1,  17.5,\n",
       "        20.2,  18.2,  13.6,  19.6,  15.2,  14.5,  15.6,  13.9,  16.6,\n",
       "        14.8,  18.4,  21. ,  12.7,  14.5,  13.2,  13.1,  13.5,  18.9,\n",
       "        20. ,  21. ,  24.7,  30.8,  34.9,  26.6,  25.3,  24.7,  21.2,\n",
       "        19.3,  20. ,  16.6,  14.4,  19.4,  19.7,  20.5,  25. ,  23.4,\n",
       "        18.9,  35.4,  24.7,  31.6,  23.3,  19.6,  18.7,  16. ,  22.2,\n",
       "        25. ,  33. ,  23.5,  19.4,  22. ,  17.4,  20.9,  24.2,  21.7,\n",
       "        22.8,  23.4,  24.1,  21.4,  20. ,  20.8,  21.2,  20.3,  28. ,\n",
       "        23.9,  24.8,  22.9,  23.9,  26.6,  22.5,  22.2,  23.6,  28.7,\n",
       "        22.6,  22. ,  22.9,  25. ,  20.6,  28.4,  21.4,  38.7,  43.8,\n",
       "        33.2,  27.5,  26.5,  18.6,  19.3,  20.1,  19.5,  19.5,  20.4,\n",
       "        19.8,  19.4,  21.7,  22.8,  18.8,  18.7,  18.5,  18.3,  21.2,\n",
       "        19.2,  20.4,  19.3,  22. ,  20.3,  20.5,  17.3,  18.8,  21.4,\n",
       "        15.7,  16.2,  18. ,  14.3,  19.2,  19.6,  23. ,  18.4,  15.6,\n",
       "        18.1,  17.4,  17.1,  13.3,  17.8,  14. ,  14.4,  13.4,  15.6,\n",
       "        11.8,  13.8,  15.6,  14.6,  17.8,  15.4,  21.5,  19.6,  15.3,\n",
       "        19.4,  17. ,  15.6,  13.1,  41.3,  24.3,  23.3,  27. ,  50. ,\n",
       "        50. ,  50. ,  22.7,  25. ,  50. ,  23.8,  23.8,  22.3,  17.4,\n",
       "        19.1,  23.1,  23.6,  22.6,  29.4,  23.2,  24.6,  29.9,  37.2,\n",
       "        39.8,  36.2,  37.9,  32.5,  26.4,  29.6,  50. ,  32. ,  29.8,\n",
       "        34.9,  37. ,  30.5,  36.4,  31.1,  29.1,  50. ,  33.3,  30.3,\n",
       "        34.6,  34.9,  32.9,  24.1,  42.3,  48.5,  50. ,  22.6,  24.4,\n",
       "        22.5,  24.4,  20. ,  21.7,  19.3,  22.4,  28.1,  23.7,  25. ,\n",
       "        23.3,  28.7,  21.5,  23. ,  26.7,  21.7,  27.5,  30.1,  44.8,\n",
       "        50. ,  37.6,  31.6,  46.7,  31.5,  24.3,  31.7,  41.7,  48.3,\n",
       "        29. ,  24. ,  25.1,  31.5,  23.7,  23.3,  22. ,  20.1,  22.2,\n",
       "        23.7,  17.6,  18.5,  24.3,  20.5,  24.5,  26.2,  24.4,  24.8,\n",
       "        29.6,  42.8,  21.9,  20.9,  44. ,  50. ,  36. ,  30.1,  33.8,\n",
       "        43.1,  48.8,  31. ,  36.5,  22.8,  30.7,  50. ,  43.5,  20.7,\n",
       "        21.1,  25.2,  24.4,  35.2,  32.4,  32. ,  33.2,  33.1,  29.1,\n",
       "        35.1,  45.4,  35.4,  46. ,  50. ,  32.2,  22. ,  20.1,  23.2,\n",
       "        22.3,  24.8,  28.5,  37.3,  27.9,  23.9,  21.7,  28.6,  27.1,\n",
       "        20.3,  22.5,  29. ,  24.8,  22. ,  26.4,  33.1,  36.1,  28.4,\n",
       "        33.4,  28.2,  22.8,  20.3,  16.1,  22.1,  19.4,  21.6,  23.8,\n",
       "        16.2,  17.8,  19.8,  23.1,  21. ,  23.8,  23.1,  20.4,  18.5,\n",
       "        25. ,  24.6,  23. ,  22.2,  19.3,  22.6,  19.8,  17.1,  19.4,\n",
       "        22.2,  20.7,  21.1,  19.5,  18.5,  20.6,  19. ,  18.7,  32.7,\n",
       "        16.5,  23.9,  31.2,  17.5,  17.2,  23.1,  24.5,  26.6,  22.9,\n",
       "        24.1,  18.6,  30.1,  18.2,  20.6,  17.8,  21.7,  22.7,  22.6,\n",
       "        25. ,  19.9,  20.8,  16.8,  21.9,  27.5,  21.9,  23.1,  50. ,\n",
       "        50. ,  50. ,  50. ,  50. ,  13.8,  13.8,  15. ,  13.9,  13.3,\n",
       "        13.1,  10.2,  10.4,  10.9,  11.3,  12.3,   8.8,   7.2,  10.5,\n",
       "         7.4,  10.2,  11.5,  15.1,  23.2,   9.7,  13.8,  12.7,  13.1,\n",
       "        12.5,   8.5,   5. ,   6.3,   5.6,   7.2,  12.1,   8.3,   8.5,\n",
       "         5. ,  11.9,  27.9,  17.2,  27.5,  15. ,  17.2,  17.9,  16.3,\n",
       "         7. ,   7.2,   7.5,  10.4,   8.8,   8.4,  16.7,  14.2,  20.8,\n",
       "        13.4,  11.7,   8.3,  10.2,  10.9,  11. ,   9.5,  14.5,  14.1,\n",
       "        16.1,  14.3,  11.7,  13.4,   9.6,   8.7,   8.4,  12.8,  10.5,\n",
       "        17.1,  18.4,  15.4,  10.8,  11.8,  14.9,  12.6,  14.1,  13. ,\n",
       "        13.4,  15.2,  16.1,  17.8,  14.9,  14.1,  12.7,  13.5,  14.9,\n",
       "        20. ,  16.4,  17.7,  19.5,  20.2,  21.4,  19.9,  19. ,  19.1,\n",
       "        19.1,  20.1,  19.9,  19.6,  23.2,  29.8,  13.8,  13.3,  16.7,\n",
       "        12. ,  14.6,  21.4,  23. ,  23.7,  25. ,  21.8,  20.6,  21.2,\n",
       "        19.1,  20.6,  15.2,   7. ,   8.1,  13.6,  20.1,  21.8,  24.5,\n",
       "        23.1,  19.7,  18.3,  21.2,  17.5,  16.8,  22.4,  20.6,  23.9,\n",
       "        22. ,  11.9])"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "boston_target"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 构建模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Help on class RandomForestRegressor in module sklearn.ensemble.forest:\n",
      "\n",
      "class RandomForestRegressor(ForestRegressor)\n",
      " |  A random forest regressor.\n",
      " |  \n",
      " |  A random forest is a meta estimator that fits a number of classifying\n",
      " |  decision trees on various sub-samples of the dataset and use averaging\n",
      " |  to improve the predictive accuracy and control over-fitting.\n",
      " |  The sub-sample size is always the same as the original\n",
      " |  input sample size but the samples are drawn with replacement if\n",
      " |  `bootstrap=True` (default).\n",
      " |  \n",
      " |  Read more in the :ref:`User Guide <forest>`.\n",
      " |  \n",
      " |  Parameters\n",
      " |  ----------\n",
      " |  n_estimators : integer, optional (default=10)\n",
      " |      The number of trees in the forest.\n",
      " |  \n",
      " |  criterion : string, optional (default=\"mse\")\n",
      " |      The function to measure the quality of a split. Supported criteria\n",
      " |      are \"mse\" for the mean squared error, which is equal to variance\n",
      " |      reduction as feature selection criterion, and \"mae\" for the mean\n",
      " |      absolute error.\n",
      " |  \n",
      " |      .. versionadded:: 0.18\n",
      " |         Mean Absolute Error (MAE) criterion.\n",
      " |  \n",
      " |  max_features : int, float, string or None, optional (default=\"auto\")\n",
      " |      The number of features to consider when looking for the best split:\n",
      " |  \n",
      " |      - If int, then consider `max_features` features at each split.\n",
      " |      - If float, then `max_features` is a percentage and\n",
      " |        `int(max_features * n_features)` features are considered at each\n",
      " |        split.\n",
      " |      - If \"auto\", then `max_features=n_features`.\n",
      " |      - If \"sqrt\", then `max_features=sqrt(n_features)`.\n",
      " |      - If \"log2\", then `max_features=log2(n_features)`.\n",
      " |      - If None, then `max_features=n_features`.\n",
      " |  \n",
      " |      Note: the search for a split does not stop until at least one\n",
      " |      valid partition of the node samples is found, even if it requires to\n",
      " |      effectively inspect more than ``max_features`` features.\n",
      " |  \n",
      " |  max_depth : integer or None, optional (default=None)\n",
      " |      The maximum depth of the tree. If None, then nodes are expanded until\n",
      " |      all leaves are pure or until all leaves contain less than\n",
      " |      min_samples_split samples.\n",
      " |  \n",
      " |  min_samples_split : int, float, optional (default=2)\n",
      " |      The minimum number of samples required to split an internal node:\n",
      " |  \n",
      " |      - If int, then consider `min_samples_split` as the minimum number.\n",
      " |      - If float, then `min_samples_split` is a percentage and\n",
      " |        `ceil(min_samples_split * n_samples)` are the minimum\n",
      " |        number of samples for each split.\n",
      " |  \n",
      " |      .. versionchanged:: 0.18\n",
      " |         Added float values for percentages.\n",
      " |  \n",
      " |  min_samples_leaf : int, float, optional (default=1)\n",
      " |      The minimum number of samples required to be at a leaf node:\n",
      " |  \n",
      " |      - If int, then consider `min_samples_leaf` as the minimum number.\n",
      " |      - If float, then `min_samples_leaf` is a percentage and\n",
      " |        `ceil(min_samples_leaf * n_samples)` are the minimum\n",
      " |        number of samples for each node.\n",
      " |  \n",
      " |      .. versionchanged:: 0.18\n",
      " |         Added float values for percentages.\n",
      " |  \n",
      " |  min_weight_fraction_leaf : float, optional (default=0.)\n",
      " |      The minimum weighted fraction of the sum total of weights (of all\n",
      " |      the input samples) required to be at a leaf node. Samples have\n",
      " |      equal weight when sample_weight is not provided.\n",
      " |  \n",
      " |  max_leaf_nodes : int or None, optional (default=None)\n",
      " |      Grow trees with ``max_leaf_nodes`` in best-first fashion.\n",
      " |      Best nodes are defined as relative reduction in impurity.\n",
      " |      If None then unlimited number of leaf nodes.\n",
      " |  \n",
      " |  min_impurity_split : float,\n",
      " |      Threshold for early stopping in tree growth. A node will split\n",
      " |      if its impurity is above the threshold, otherwise it is a leaf.\n",
      " |  \n",
      " |      .. deprecated:: 0.19\n",
      " |         ``min_impurity_split`` has been deprecated in favor of\n",
      " |         ``min_impurity_decrease`` in 0.19 and will be removed in 0.21.\n",
      " |         Use ``min_impurity_decrease`` instead.\n",
      " |  \n",
      " |  min_impurity_decrease : float, optional (default=0.)\n",
      " |      A node will be split if this split induces a decrease of the impurity\n",
      " |      greater than or equal to this value.\n",
      " |  \n",
      " |      The weighted impurity decrease equation is the following::\n",
      " |  \n",
      " |          N_t / N * (impurity - N_t_R / N_t * right_impurity\n",
      " |                              - N_t_L / N_t * left_impurity)\n",
      " |  \n",
      " |      where ``N`` is the total number of samples, ``N_t`` is the number of\n",
      " |      samples at the current node, ``N_t_L`` is the number of samples in the\n",
      " |      left child, and ``N_t_R`` is the number of samples in the right child.\n",
      " |  \n",
      " |      ``N``, ``N_t``, ``N_t_R`` and ``N_t_L`` all refer to the weighted sum,\n",
      " |      if ``sample_weight`` is passed.\n",
      " |  \n",
      " |      .. versionadded:: 0.19\n",
      " |  \n",
      " |  bootstrap : boolean, optional (default=True)\n",
      " |      Whether bootstrap samples are used when building trees.\n",
      " |  \n",
      " |  oob_score : bool, optional (default=False)\n",
      " |      whether to use out-of-bag samples to estimate\n",
      " |      the R^2 on unseen data.\n",
      " |  \n",
      " |  n_jobs : integer, optional (default=1)\n",
      " |      The number of jobs to run in parallel for both `fit` and `predict`.\n",
      " |      If -1, then the number of jobs is set to the number of cores.\n",
      " |  \n",
      " |  random_state : int, RandomState instance or None, optional (default=None)\n",
      " |      If int, random_state is the seed used by the random number generator;\n",
      " |      If RandomState instance, random_state is the random number generator;\n",
      " |      If None, the random number generator is the RandomState instance used\n",
      " |      by `np.random`.\n",
      " |  \n",
      " |  verbose : int, optional (default=0)\n",
      " |      Controls the verbosity of the tree building process.\n",
      " |  \n",
      " |  warm_start : bool, optional (default=False)\n",
      " |      When set to ``True``, reuse the solution of the previous call to fit\n",
      " |      and add more estimators to the ensemble, otherwise, just fit a whole\n",
      " |      new forest.\n",
      " |  \n",
      " |  Attributes\n",
      " |  ----------\n",
      " |  estimators_ : list of DecisionTreeRegressor\n",
      " |      The collection of fitted sub-estimators.\n",
      " |  \n",
      " |  feature_importances_ : array of shape = [n_features]\n",
      " |      The feature importances (the higher, the more important the feature).\n",
      " |  \n",
      " |  n_features_ : int\n",
      " |      The number of features when ``fit`` is performed.\n",
      " |  \n",
      " |  n_outputs_ : int\n",
      " |      The number of outputs when ``fit`` is performed.\n",
      " |  \n",
      " |  oob_score_ : float\n",
      " |      Score of the training dataset obtained using an out-of-bag estimate.\n",
      " |  \n",
      " |  oob_prediction_ : array of shape = [n_samples]\n",
      " |      Prediction computed with out-of-bag estimate on the training set.\n",
      " |  \n",
      " |  Examples\n",
      " |  --------\n",
      " |  >>> from sklearn.ensemble import RandomForestRegressor\n",
      " |  >>> from sklearn.datasets import make_regression\n",
      " |  >>>\n",
      " |  >>> X, y = make_regression(n_features=4, n_informative=2,\n",
      " |  ...                        random_state=0, shuffle=False)\n",
      " |  >>> regr = RandomForestRegressor(max_depth=2, random_state=0)\n",
      " |  >>> regr.fit(X, y)\n",
      " |  RandomForestRegressor(bootstrap=True, criterion='mse', max_depth=2,\n",
      " |             max_features='auto', max_leaf_nodes=None,\n",
      " |             min_impurity_decrease=0.0, min_impurity_split=None,\n",
      " |             min_samples_leaf=1, min_samples_split=2,\n",
      " |             min_weight_fraction_leaf=0.0, n_estimators=10, n_jobs=1,\n",
      " |             oob_score=False, random_state=0, verbose=0, warm_start=False)\n",
      " |  >>> print(regr.feature_importances_)\n",
      " |  [ 0.17339552  0.81594114  0.          0.01066333]\n",
      " |  >>> print(regr.predict([[0, 0, 0, 0]]))\n",
      " |  [-2.50699856]\n",
      " |  \n",
      " |  Notes\n",
      " |  -----\n",
      " |  The default values for the parameters controlling the size of the trees\n",
      " |  (e.g. ``max_depth``, ``min_samples_leaf``, etc.) lead to fully grown and\n",
      " |  unpruned trees which can potentially be very large on some data sets. To\n",
      " |  reduce memory consumption, the complexity and size of the trees should be\n",
      " |  controlled by setting those parameter values.\n",
      " |  \n",
      " |  The features are always randomly permuted at each split. Therefore,\n",
      " |  the best found split may vary, even with the same training data,\n",
      " |  ``max_features=n_features`` and ``bootstrap=False``, if the improvement\n",
      " |  of the criterion is identical for several splits enumerated during the\n",
      " |  search of the best split. To obtain a deterministic behaviour during\n",
      " |  fitting, ``random_state`` has to be fixed.\n",
      " |  \n",
      " |  References\n",
      " |  ----------\n",
      " |  \n",
      " |  .. [1] L. Breiman, \"Random Forests\", Machine Learning, 45(1), 5-32, 2001.\n",
      " |  \n",
      " |  See also\n",
      " |  --------\n",
      " |  DecisionTreeRegressor, ExtraTreesRegressor\n",
      " |  \n",
      " |  Method resolution order:\n",
      " |      RandomForestRegressor\n",
      " |      ForestRegressor\n",
      " |      abc.NewBase\n",
      " |      BaseForest\n",
      " |      abc.NewBase\n",
      " |      sklearn.ensemble.base.BaseEnsemble\n",
      " |      abc.NewBase\n",
      " |      sklearn.base.BaseEstimator\n",
      " |      sklearn.base.MetaEstimatorMixin\n",
      " |      sklearn.base.RegressorMixin\n",
      " |      __builtin__.object\n",
      " |  \n",
      " |  Methods defined here:\n",
      " |  \n",
      " |  __init__(self, n_estimators=10, criterion='mse', max_depth=None, min_samples_split=2, min_samples_leaf=1, min_weight_fraction_leaf=0.0, max_features='auto', max_leaf_nodes=None, min_impurity_decrease=0.0, min_impurity_split=None, bootstrap=True, oob_score=False, n_jobs=1, random_state=None, verbose=0, warm_start=False)\n",
      " |  \n",
      " |  ----------------------------------------------------------------------\n",
      " |  Data and other attributes defined here:\n",
      " |  \n",
      " |  __abstractmethods__ = frozenset([])\n",
      " |  \n",
      " |  ----------------------------------------------------------------------\n",
      " |  Methods inherited from ForestRegressor:\n",
      " |  \n",
      " |  predict(self, X)\n",
      " |      Predict regression target for X.\n",
      " |      \n",
      " |      The predicted regression target of an input sample is computed as the\n",
      " |      mean predicted regression targets of the trees in the forest.\n",
      " |      \n",
      " |      Parameters\n",
      " |      ----------\n",
      " |      X : array-like or sparse matrix of shape = [n_samples, n_features]\n",
      " |          The input samples. Internally, its dtype will be converted to\n",
      " |          ``dtype=np.float32``. If a sparse matrix is provided, it will be\n",
      " |          converted into a sparse ``csr_matrix``.\n",
      " |      \n",
      " |      Returns\n",
      " |      -------\n",
      " |      y : array of shape = [n_samples] or [n_samples, n_outputs]\n",
      " |          The predicted values.\n",
      " |  \n",
      " |  ----------------------------------------------------------------------\n",
      " |  Methods inherited from BaseForest:\n",
      " |  \n",
      " |  apply(self, X)\n",
      " |      Apply trees in the forest to X, return leaf indices.\n",
      " |      \n",
      " |      Parameters\n",
      " |      ----------\n",
      " |      X : array-like or sparse matrix, shape = [n_samples, n_features]\n",
      " |          The input samples. Internally, its dtype will be converted to\n",
      " |          ``dtype=np.float32``. If a sparse matrix is provided, it will be\n",
      " |          converted into a sparse ``csr_matrix``.\n",
      " |      \n",
      " |      Returns\n",
      " |      -------\n",
      " |      X_leaves : array_like, shape = [n_samples, n_estimators]\n",
      " |          For each datapoint x in X and for each tree in the forest,\n",
      " |          return the index of the leaf x ends up in.\n",
      " |  \n",
      " |  decision_path(self, X)\n",
      " |      Return the decision path in the forest\n",
      " |      \n",
      " |      .. versionadded:: 0.18\n",
      " |      \n",
      " |      Parameters\n",
      " |      ----------\n",
      " |      X : array-like or sparse matrix, shape = [n_samples, n_features]\n",
      " |          The input samples. Internally, its dtype will be converted to\n",
      " |          ``dtype=np.float32``. If a sparse matrix is provided, it will be\n",
      " |          converted into a sparse ``csr_matrix``.\n",
      " |      \n",
      " |      Returns\n",
      " |      -------\n",
      " |      indicator : sparse csr array, shape = [n_samples, n_nodes]\n",
      " |          Return a node indicator matrix where non zero elements\n",
      " |          indicates that the samples goes through the nodes.\n",
      " |      \n",
      " |      n_nodes_ptr : array of size (n_estimators + 1, )\n",
      " |          The columns from indicator[n_nodes_ptr[i]:n_nodes_ptr[i+1]]\n",
      " |          gives the indicator value for the i-th estimator.\n",
      " |  \n",
      " |  fit(self, X, y, sample_weight=None)\n",
      " |      Build a forest of trees from the training set (X, y).\n",
      " |      \n",
      " |      Parameters\n",
      " |      ----------\n",
      " |      X : array-like or sparse matrix of shape = [n_samples, n_features]\n",
      " |          The training input samples. Internally, its dtype will be converted to\n",
      " |          ``dtype=np.float32``. If a sparse matrix is provided, it will be\n",
      " |          converted into a sparse ``csc_matrix``.\n",
      " |      \n",
      " |      y : array-like, shape = [n_samples] or [n_samples, n_outputs]\n",
      " |          The target values (class labels in classification, real numbers in\n",
      " |          regression).\n",
      " |      \n",
      " |      sample_weight : array-like, shape = [n_samples] or None\n",
      " |          Sample weights. If None, then samples are equally weighted. Splits\n",
      " |          that would create child nodes with net zero or negative weight are\n",
      " |          ignored while searching for a split in each node. In the case of\n",
      " |          classification, splits are also ignored if they would result in any\n",
      " |          single class carrying a negative weight in either child node.\n",
      " |      \n",
      " |      Returns\n",
      " |      -------\n",
      " |      self : object\n",
      " |          Returns self.\n",
      " |  \n",
      " |  ----------------------------------------------------------------------\n",
      " |  Data descriptors inherited from BaseForest:\n",
      " |  \n",
      " |  feature_importances_\n",
      " |      Return the feature importances (the higher, the more important the\n",
      " |         feature).\n",
      " |      \n",
      " |      Returns\n",
      " |      -------\n",
      " |      feature_importances_ : array, shape = [n_features]\n",
      " |  \n",
      " |  ----------------------------------------------------------------------\n",
      " |  Methods inherited from sklearn.ensemble.base.BaseEnsemble:\n",
      " |  \n",
      " |  __getitem__(self, index)\n",
      " |      Returns the index'th estimator in the ensemble.\n",
      " |  \n",
      " |  __iter__(self)\n",
      " |      Returns iterator over estimators in the ensemble.\n",
      " |  \n",
      " |  __len__(self)\n",
      " |      Returns the number of estimators in the ensemble.\n",
      " |  \n",
      " |  ----------------------------------------------------------------------\n",
      " |  Methods inherited from sklearn.base.BaseEstimator:\n",
      " |  \n",
      " |  __getstate__(self)\n",
      " |  \n",
      " |  __repr__(self)\n",
      " |  \n",
      " |  __setstate__(self, state)\n",
      " |  \n",
      " |  get_params(self, deep=True)\n",
      " |      Get parameters for this estimator.\n",
      " |      \n",
      " |      Parameters\n",
      " |      ----------\n",
      " |      deep : boolean, optional\n",
      " |          If True, will return the parameters for this estimator and\n",
      " |          contained subobjects that are estimators.\n",
      " |      \n",
      " |      Returns\n",
      " |      -------\n",
      " |      params : mapping of string to any\n",
      " |          Parameter names mapped to their values.\n",
      " |  \n",
      " |  set_params(self, **params)\n",
      " |      Set the parameters of this estimator.\n",
      " |      \n",
      " |      The method works on simple estimators as well as on nested objects\n",
      " |      (such as pipelines). The latter have parameters of the form\n",
      " |      ``<component>__<parameter>`` so that it's possible to update each\n",
      " |      component of a nested object.\n",
      " |      \n",
      " |      Returns\n",
      " |      -------\n",
      " |      self\n",
      " |  \n",
      " |  ----------------------------------------------------------------------\n",
      " |  Data descriptors inherited from sklearn.base.BaseEstimator:\n",
      " |  \n",
      " |  __dict__\n",
      " |      dictionary for instance variables (if defined)\n",
      " |  \n",
      " |  __weakref__\n",
      " |      list of weak references to the object (if defined)\n",
      " |  \n",
      " |  ----------------------------------------------------------------------\n",
      " |  Methods inherited from sklearn.base.RegressorMixin:\n",
      " |  \n",
      " |  score(self, X, y, sample_weight=None)\n",
      " |      Returns the coefficient of determination R^2 of the prediction.\n",
      " |      \n",
      " |      The coefficient R^2 is defined as (1 - u/v), where u is the residual\n",
      " |      sum of squares ((y_true - y_pred) ** 2).sum() and v is the total\n",
      " |      sum of squares ((y_true - y_true.mean()) ** 2).sum().\n",
      " |      The best possible score is 1.0 and it can be negative (because the\n",
      " |      model can be arbitrarily worse). A constant model that always\n",
      " |      predicts the expected value of y, disregarding the input features,\n",
      " |      would get a R^2 score of 0.0.\n",
      " |      \n",
      " |      Parameters\n",
      " |      ----------\n",
      " |      X : array-like, shape = (n_samples, n_features)\n",
      " |          Test samples.\n",
      " |      \n",
      " |      y : array-like, shape = (n_samples) or (n_samples, n_outputs)\n",
      " |          True values for X.\n",
      " |      \n",
      " |      sample_weight : array-like, shape = [n_samples], optional\n",
      " |          Sample weights.\n",
      " |      \n",
      " |      Returns\n",
      " |      -------\n",
      " |      score : float\n",
      " |          R^2 of self.predict(X) wrt. y.\n",
      "\n"
     ]
    }
   ],
   "source": [
    "help(RandomForestRegressor)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "rgs = RandomForestRegressor(n_estimators=15)  ##随机森林模型\n",
    "rgs = rgs.fit(boston_features, boston_target)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "RandomForestRegressor(bootstrap=True, criterion='mse', max_depth=None,\n",
       "           max_features='auto', max_leaf_nodes=None,\n",
       "           min_impurity_decrease=0.0, min_impurity_split=None,\n",
       "           min_samples_leaf=1, min_samples_split=2,\n",
       "           min_weight_fraction_leaf=0.0, n_estimators=15, n_jobs=1,\n",
       "           oob_score=False, random_state=None, verbose=0, warm_start=False)"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rgs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 26.16666667,  22.24      ,  33.76666667,  33.67333333,\n",
       "        35.46      ,  26.74      ,  21.84      ,  26.59333333,\n",
       "        20.31333333,  19.88      ,  18.53333333,  19.57333333,\n",
       "        21.64      ,  19.64666667,  19.37333333,  19.92      ,\n",
       "        22.3       ,  17.8       ,  19.64      ,  18.73333333,\n",
       "        13.80666667,  18.42666667,  15.5       ,  14.46666667,\n",
       "        15.54666667,  14.18      ,  16.34666667,  14.58666667,\n",
       "        18.25333333,  21.46      ,  13.38      ,  15.86      ,\n",
       "        14.09333333,  13.8       ,  13.71333333,  19.51333333,\n",
       "        19.96666667,  21.49333333,  23.56666667,  30.07333333,\n",
       "        34.7       ,  28.31333333,  25.04      ,  24.64666667,\n",
       "        21.45333333,  19.42      ,  19.74      ,  18.26666667,\n",
       "        19.57333333,  20.02666667,  20.74666667,  20.84666667,\n",
       "        25.42666667,  22.25333333,  19.12666667,  34.66666667,\n",
       "        24.3       ,  32.18666667,  23.42666667,  19.78      ,\n",
       "        18.78666667,  17.82666667,  23.03333333,  25.81333333,\n",
       "        32.30666667,  23.85333333,  20.3       ,  21.85333333,\n",
       "        18.59333333,  21.10666667,  24.42      ,  21.25333333,\n",
       "        23.2       ,  23.56666667,  24.22      ,  22.24      ,\n",
       "        20.08      ,  21.34666667,  21.24      ,  20.49333333,\n",
       "        27.87333333,  24.4       ,  24.16666667,  22.98      ,\n",
       "        23.48666667,  27.13333333,  21.23333333,  22.28666667,\n",
       "        26.63333333,  29.01333333,  22.4       ,  22.06666667,\n",
       "        22.75333333,  24.77333333,  21.38666667,  26.91333333,\n",
       "        21.57333333,  40.41333333,  44.06      ,  32.64666667,\n",
       "        26.9       ,  25.90666667,  19.05333333,  19.85333333,\n",
       "        20.00666667,  19.73333333,  19.09333333,  20.34666667,\n",
       "        20.21333333,  19.20666667,  21.24      ,  24.06      ,\n",
       "        18.76666667,  18.66      ,  19.18666667,  18.37333333,\n",
       "        21.08      ,  20.04      ,  19.32      ,  19.11333333,\n",
       "        21.79333333,  21.07333333,  19.68      ,  16.92      ,\n",
       "        19.00666667,  20.57333333,  15.87333333,  16.04666667,\n",
       "        17.64666667,  14.80666667,  19.72666667,  19.89333333,\n",
       "        21.52      ,  18.16666667,  15.46      ,  18.89333333,\n",
       "        16.71333333,  18.36      ,  13.54666667,  16.38666667,\n",
       "        13.9       ,  14.04      ,  13.41333333,  14.58666667,\n",
       "        12.49333333,  13.95333333,  15.80666667,  14.44      ,\n",
       "        16.67333333,  15.32      ,  21.67333333,  19.6       ,\n",
       "        16.92666667,  17.62666667,  16.9       ,  15.82666667,\n",
       "        14.26666667,  34.84666667,  23.87333333,  24.14      ,\n",
       "        26.14666667,  48.45333333,  49.56666667,  49.44666667,\n",
       "        21.82666667,  24.29333333,  49.80666667,  22.55333333,\n",
       "        22.67333333,  21.97333333,  18.66666667,  20.72      ,\n",
       "        21.75333333,  23.42666667,  22.56666667,  29.07333333,\n",
       "        23.02666667,  24.20666667,  28.74666667,  35.60666667,\n",
       "        42.8       ,  33.5       ,  36.97333333,  32.09333333,\n",
       "        26.45333333,  28.88      ,  47.63333333,  30.38666667,\n",
       "        29.18666667,  34.84666667,  34.12666667,  29.24666667,\n",
       "        35.76666667,  31.82666667,  29.34      ,  48.67333333,\n",
       "        33.6       ,  30.52666667,  34.26666667,  34.84      ,\n",
       "        33.73333333,  23.38      ,  42.36666667,  48.6       ,\n",
       "        48.47333333,  22.15333333,  24.27333333,  21.84      ,\n",
       "        23.44      ,  20.54666667,  21.4       ,  20.68      ,\n",
       "        22.51333333,  26.79333333,  22.83333333,  24.82666667,\n",
       "        22.53333333,  27.07333333,  20.67333333,  22.8       ,\n",
       "        27.        ,  21.24      ,  26.67333333,  29.24      ,\n",
       "        45.48      ,  48.93333333,  41.64666667,  31.81333333,\n",
       "        46.19333333,  32.8       ,  23.28      ,  31.92666667,\n",
       "        43.86666667,  45.86      ,  28.46      ,  23.44      ,\n",
       "        25.91333333,  31.70666667,  23.57333333,  24.63333333,\n",
       "        24.99333333,  20.82666667,  22.10666667,  23.97333333,\n",
       "        18.09333333,  18.52      ,  23.64666667,  20.46      ,\n",
       "        23.2       ,  26.64      ,  24.36      ,  27.22      ,\n",
       "        30.37333333,  40.96      ,  22.07333333,  21.23333333,\n",
       "        44.58666667,  49.24666667,  36.1       ,  30.28      ,\n",
       "        33.12666667,  43.98      ,  47.82      ,  30.35333333,\n",
       "        36.10666667,  21.79333333,  29.96      ,  48.74666667,\n",
       "        43.60666667,  20.66666667,  21.44      ,  25.01333333,\n",
       "        24.76      ,  38.78      ,  33.61333333,  33.6       ,\n",
       "        33.22666667,  32.66666667,  26.78      ,  34.47333333,\n",
       "        46.01333333,  34.73333333,  45.41333333,  48.90666667,\n",
       "        31.54      ,  22.25333333,  20.25333333,  23.65333333,\n",
       "        22.66      ,  24.64666667,  31.19333333,  34.18      ,\n",
       "        28.34666667,  23.28666667,  22.12      ,  27.36      ,\n",
       "        26.49333333,  20.88666667,  24.1       ,  29.98666667,\n",
       "        26.59333333,  23.92      ,  25.13333333,  32.73333333,\n",
       "        35.57333333,  28.28666667,  34.6       ,  29.93333333,\n",
       "        25.48666667,  20.17333333,  17.52      ,  22.84      ,\n",
       "        19.96      ,  21.66666667,  23.65333333,  16.72666667,\n",
       "        17.96666667,  19.64      ,  23.01333333,  21.31333333,\n",
       "        23.89333333,  23.42      ,  21.14666667,  18.29333333,\n",
       "        24.74666667,  24.94666667,  23.97333333,  22.04      ,\n",
       "        20.17333333,  22.75333333,  19.79333333,  17.60666667,\n",
       "        20.96      ,  22.32      ,  22.04666667,  20.34666667,\n",
       "        19.46666667,  19.18      ,  20.54666667,  19.31333333,\n",
       "        18.98      ,  32.66666667,  18.45333333,  25.26666667,\n",
       "        30.28      ,  17.92666667,  18.08      ,  23.3       ,\n",
       "        24.58666667,  27.50666667,  23.44      ,  25.18      ,\n",
       "        20.55333333,  29.7       ,  19.15333333,  20.94      ,\n",
       "        16.68      ,  21.38666667,  21.86666667,  22.14      ,\n",
       "        23.73333333,  20.13333333,  20.        ,  17.24666667,\n",
       "        26.76666667,  22.62666667,  19.58      ,  21.64      ,\n",
       "        46.78666667,  47.02      ,  41.21333333,  45.14666667,\n",
       "        42.74666667,  13.46      ,  13.32      ,  22.26666667,\n",
       "        13.04666667,  12.56      ,  12.65333333,  10.74      ,\n",
       "        15.79333333,  10.94666667,  11.46      ,  11.27333333,\n",
       "         9.1       ,   8.34666667,   8.96      ,   7.56      ,\n",
       "         9.76      ,  11.43333333,  14.76      ,  20.18      ,\n",
       "         9.70666667,  14.08      ,  11.27333333,  13.44666667,\n",
       "        13.02666667,  10.52      ,   5.72666667,   7.14666667,\n",
       "         7.20666667,   8.32666667,  11.5       ,   9.32      ,\n",
       "         8.04666667,   6.98666667,  12.48666667,  31.44666667,\n",
       "        16.7       ,  25.96666667,  21.21333333,  17.04      ,\n",
       "        17.02666667,  16.3       ,   7.26666667,   7.3       ,\n",
       "         8.25333333,  10.02      ,   8.87333333,  11.07333333,\n",
       "        15.46666667,  14.47333333,  18.96666667,  12.74      ,\n",
       "        12.3       ,   8.9       ,  11.50666667,  12.76      ,\n",
       "        12.12      ,  10.05333333,  14.50666667,  17.65333333,\n",
       "        17.75333333,  14.52      ,  11.85333333,  11.67333333,\n",
       "        10.94      ,   8.68666667,   8.02      ,  12.10666667,\n",
       "        10.29333333,  16.21333333,  17.60666667,  15.25333333,\n",
       "        10.71333333,  12.16666667,  14.80666667,  14.68666667,\n",
       "        14.30666667,  13.55333333,  14.02      ,  15.37333333,\n",
       "        16.08666667,  19.74666667,  14.77333333,  13.98      ,\n",
       "        13.53333333,  13.98      ,  15.16      ,  19.47333333,\n",
       "        16.33333333,  18.45333333,  20.74      ,  20.99333333,\n",
       "        21.57333333,  19.66666667,  17.67333333,  15.99333333,\n",
       "        18.84666667,  20.22      ,  18.75333333,  19.84      ,\n",
       "        22.87333333,  27.68666667,  13.87333333,  14.12      ,\n",
       "        16.30666667,  11.87333333,  14.16      ,  20.96      ,\n",
       "        22.48666667,  24.33333333,  26.02666667,  20.92      ,\n",
       "        20.38      ,  22.08      ,  18.89333333,  20.89333333,\n",
       "        14.98666667,   9.30666667,   9.91333333,  13.98666667,\n",
       "        20.46      ,  21.08666667,  23.27333333,  19.56      ,\n",
       "        19.76666667,  18.92666667,  21.03333333,  18.05333333,\n",
       "        17.84666667,  22.7       ,  20.64666667,  25.46666667,\n",
       "        23.96666667,  14.72666667])"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rgs.predict(boston_features)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn import tree"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "DecisionTreeRegressor(criterion='mse', max_depth=None, max_features=None,\n",
       "           max_leaf_nodes=None, min_impurity_decrease=0.0,\n",
       "           min_impurity_split=None, min_samples_leaf=1,\n",
       "           min_samples_split=2, min_weight_fraction_leaf=0.0,\n",
       "           presort=False, random_state=None, splitter='best')"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rgs2 = tree.DecisionTreeRegressor()           ##决策树模型，比较两个模型的预测结果！\n",
    "rgs2.fit(boston_features, boston_target)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 24. ,  21.6,  34.7,  33.4,  36.2,  28.7,  22.9,  27.1,  16.5,\n",
       "        18.9,  15. ,  18.9,  21.7,  20.4,  18.2,  19.9,  23.1,  17.5,\n",
       "        20.2,  18.2,  13.6,  19.6,  15.2,  14.5,  15.6,  13.9,  16.6,\n",
       "        14.8,  18.4,  21. ,  12.7,  14.5,  13.2,  13.1,  13.5,  18.9,\n",
       "        20. ,  21. ,  24.7,  30.8,  34.9,  26.6,  25.3,  24.7,  21.2,\n",
       "        19.3,  20. ,  16.6,  14.4,  19.4,  19.7,  20.5,  25. ,  23.4,\n",
       "        18.9,  35.4,  24.7,  31.6,  23.3,  19.6,  18.7,  16. ,  22.2,\n",
       "        25. ,  33. ,  23.5,  19.4,  22. ,  17.4,  20.9,  24.2,  21.7,\n",
       "        22.8,  23.4,  24.1,  21.4,  20. ,  20.8,  21.2,  20.3,  28. ,\n",
       "        23.9,  24.8,  22.9,  23.9,  26.6,  22.5,  22.2,  23.6,  28.7,\n",
       "        22.6,  22. ,  22.9,  25. ,  20.6,  28.4,  21.4,  38.7,  43.8,\n",
       "        33.2,  27.5,  26.5,  18.6,  19.3,  20.1,  19.5,  19.5,  20.4,\n",
       "        19.8,  19.4,  21.7,  22.8,  18.8,  18.7,  18.5,  18.3,  21.2,\n",
       "        19.2,  20.4,  19.3,  22. ,  20.3,  20.5,  17.3,  18.8,  21.4,\n",
       "        15.7,  16.2,  18. ,  14.3,  19.2,  19.6,  23. ,  18.4,  15.6,\n",
       "        18.1,  17.4,  17.1,  13.3,  17.8,  14. ,  14.4,  13.4,  15.6,\n",
       "        11.8,  13.8,  15.6,  14.6,  17.8,  15.4,  21.5,  19.6,  15.3,\n",
       "        19.4,  17. ,  15.6,  13.1,  41.3,  24.3,  23.3,  27. ,  50. ,\n",
       "        50. ,  50. ,  22.7,  25. ,  50. ,  23.8,  23.8,  22.3,  17.4,\n",
       "        19.1,  23.1,  23.6,  22.6,  29.4,  23.2,  24.6,  29.9,  37.2,\n",
       "        39.8,  36.2,  37.9,  32.5,  26.4,  29.6,  50. ,  32. ,  29.8,\n",
       "        34.9,  37. ,  30.5,  36.4,  31.1,  29.1,  50. ,  33.3,  30.3,\n",
       "        34.6,  34.9,  32.9,  24.1,  42.3,  48.5,  50. ,  22.6,  24.4,\n",
       "        22.5,  24.4,  20. ,  21.7,  19.3,  22.4,  28.1,  23.7,  25. ,\n",
       "        23.3,  28.7,  21.5,  23. ,  26.7,  21.7,  27.5,  30.1,  44.8,\n",
       "        50. ,  37.6,  31.6,  46.7,  31.5,  24.3,  31.7,  41.7,  48.3,\n",
       "        29. ,  24. ,  25.1,  31.5,  23.7,  23.3,  22. ,  20.1,  22.2,\n",
       "        23.7,  17.6,  18.5,  24.3,  20.5,  24.5,  26.2,  24.4,  24.8,\n",
       "        29.6,  42.8,  21.9,  20.9,  44. ,  50. ,  36. ,  30.1,  33.8,\n",
       "        43.1,  48.8,  31. ,  36.5,  22.8,  30.7,  50. ,  43.5,  20.7,\n",
       "        21.1,  25.2,  24.4,  35.2,  32.4,  32. ,  33.2,  33.1,  29.1,\n",
       "        35.1,  45.4,  35.4,  46. ,  50. ,  32.2,  22. ,  20.1,  23.2,\n",
       "        22.3,  24.8,  28.5,  37.3,  27.9,  23.9,  21.7,  28.6,  27.1,\n",
       "        20.3,  22.5,  29. ,  24.8,  22. ,  26.4,  33.1,  36.1,  28.4,\n",
       "        33.4,  28.2,  22.8,  20.3,  16.1,  22.1,  19.4,  21.6,  23.8,\n",
       "        16.2,  17.8,  19.8,  23.1,  21. ,  23.8,  23.1,  20.4,  18.5,\n",
       "        25. ,  24.6,  23. ,  22.2,  19.3,  22.6,  19.8,  17.1,  19.4,\n",
       "        22.2,  20.7,  21.1,  19.5,  18.5,  20.6,  19. ,  18.7,  32.7,\n",
       "        16.5,  23.9,  31.2,  17.5,  17.2,  23.1,  24.5,  26.6,  22.9,\n",
       "        24.1,  18.6,  30.1,  18.2,  20.6,  17.8,  21.7,  22.7,  22.6,\n",
       "        25. ,  19.9,  20.8,  16.8,  21.9,  27.5,  21.9,  23.1,  50. ,\n",
       "        50. ,  50. ,  50. ,  50. ,  13.8,  13.8,  15. ,  13.9,  13.3,\n",
       "        13.1,  10.2,  10.4,  10.9,  11.3,  12.3,   8.8,   7.2,  10.5,\n",
       "         7.4,  10.2,  11.5,  15.1,  23.2,   9.7,  13.8,  12.7,  13.1,\n",
       "        12.5,   8.5,   5. ,   6.3,   5.6,   7.2,  12.1,   8.3,   8.5,\n",
       "         5. ,  11.9,  27.9,  17.2,  27.5,  15. ,  17.2,  17.9,  16.3,\n",
       "         7. ,   7.2,   7.5,  10.4,   8.8,   8.4,  16.7,  14.2,  20.8,\n",
       "        13.4,  11.7,   8.3,  10.2,  10.9,  11. ,   9.5,  14.5,  14.1,\n",
       "        16.1,  14.3,  11.7,  13.4,   9.6,   8.7,   8.4,  12.8,  10.5,\n",
       "        17.1,  18.4,  15.4,  10.8,  11.8,  14.9,  12.6,  14.1,  13. ,\n",
       "        13.4,  15.2,  16.1,  17.8,  14.9,  14.1,  12.7,  13.5,  14.9,\n",
       "        20. ,  16.4,  17.7,  19.5,  20.2,  21.4,  19.9,  19. ,  19.1,\n",
       "        19.1,  20.1,  19.9,  19.6,  23.2,  29.8,  13.8,  13.3,  16.7,\n",
       "        12. ,  14.6,  21.4,  23. ,  23.7,  25. ,  21.8,  20.6,  21.2,\n",
       "        19.1,  20.6,  15.2,   7. ,   8.1,  13.6,  20.1,  21.8,  24.5,\n",
       "        23.1,  19.7,  18.3,  21.2,  17.5,  16.8,  22.4,  20.6,  23.9,\n",
       "        22. ,  11.9])"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rgs2.predict(boston_features)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}