Titanic.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 经典又兼具备趣味性的Kaggle案例[泰坦尼克号问题](https://www.kaggle.com/c/titanic)\n",
    "大家都熟悉的『Jack and Rose』的故事，豪华游艇倒了，大家都惊恐逃生，可是救生艇的数量有限，无法人人都有，副船长发话了『lady and kid first！』，所以是否获救其实并非随机，而是基于一些背景有rank先后的。<br>\n",
    "训练和测试数据是一些乘客的个人信息以及存活状况，要尝试根据它生成合适的模型并预测其他人的存活状况。<br>\n",
    "对，这是一个二分类问题，很多分类算法都可以解决。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**<font color=red>看看数据长什么样</font>**<br>\n",
    "还是用pandas加载数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index([u'PassengerId', u'Survived', u'Pclass', u'Name', u'Sex', u'Age',\n",
       "       u'SibSp', u'Parch', u'Ticket', u'Fare', u'Cabin', u'Embarked'],\n",
       "      dtype='object')"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 这个ipython notebook主要是我解决Kaggle Titanic问题的思路和过程\n",
    "\n",
    "import pandas as pd #数据分析\n",
    "import numpy as np #科学计算\n",
    "from pandas import Series,DataFrame\n",
    "\n",
    "data_train = pd.read_csv(\"Train.csv\")\n",
    "data_train.columns\n",
    "#data_train[data_train.Cabin.notnull()]['Survived'].value_counts()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**<font color=red>我们看大概有以下这些字段</font>**<br>\n",
    "PassengerId => 乘客ID<br>\n",
    "Pclass => 乘客等级(1/2/3等舱位)<br>\n",
    "Name => 乘客姓名<br>\n",
    "Sex => 性别<br>\n",
    "Age => 年龄<br>\n",
    "SibSp => 堂兄弟/妹个数<br>\n",
    "Parch => 父母与小孩个数<br>\n",
    "Ticket => 船票信息<br>\n",
    "Fare => 票价<br>\n",
    "Cabin => 客舱<br>\n",
    "Embarked => 登船港口"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**<font color=red>我这么懒的人显然会让pandas自己先告诉我们一些信息<font>**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 891 entries, 0 to 890\n",
      "Data columns (total 12 columns):\n",
      "PassengerId    891 non-null int64\n",
      "Survived       891 non-null int64\n",
      "Pclass         891 non-null int64\n",
      "Name           891 non-null object\n",
      "Sex            891 non-null object\n",
      "Age            714 non-null float64\n",
      "SibSp          891 non-null int64\n",
      "Parch          891 non-null int64\n",
      "Ticket         891 non-null object\n",
      "Fare           891 non-null float64\n",
      "Cabin          204 non-null object\n",
      "Embarked       889 non-null object\n",
      "dtypes: float64(2), int64(5), object(5)\n",
      "memory usage: 83.6+ KB\n"
     ]
    }
   ],
   "source": [
    "data_train.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<font color=red>上面的数据说啥了？它告诉我们，训练数据中总共有891名乘客，但是很不幸，我们有些属性的数据不全，比如说：<font><br>\n",
    "\n",
    "* <font color=red>Age（年龄）属性只有714名乘客有记录<font>\n",
    "* <font color=red>Cabin（客舱）更是只有204名乘客是已知的<font>\n",
    "\n",
    "<font color=red>似乎信息略少啊，想再瞄一眼具体数据数值情况呢？恩，我们用下列的方法，得到数值型数据的一些分布(因为有些属性，比如姓名，是文本型；而另外一些属性，比如登船港口，是类目型。这些我们用下面的函数是看不到的)<font>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Fare</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>891.000000</td>\n",
       "      <td>891.000000</td>\n",
       "      <td>891.000000</td>\n",
       "      <td>714.000000</td>\n",
       "      <td>891.000000</td>\n",
       "      <td>891.000000</td>\n",
       "      <td>891.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>446.000000</td>\n",
       "      <td>0.383838</td>\n",
       "      <td>2.308642</td>\n",
       "      <td>29.699118</td>\n",
       "      <td>0.523008</td>\n",
       "      <td>0.381594</td>\n",
       "      <td>32.204208</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>257.353842</td>\n",
       "      <td>0.486592</td>\n",
       "      <td>0.836071</td>\n",
       "      <td>14.526497</td>\n",
       "      <td>1.102743</td>\n",
       "      <td>0.806057</td>\n",
       "      <td>49.693429</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.420000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>223.500000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>7.910400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>446.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>14.454200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>668.500000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>31.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>891.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>80.000000</td>\n",
       "      <td>8.000000</td>\n",
       "      <td>6.000000</td>\n",
       "      <td>512.329200</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       PassengerId    Survived      Pclass         Age       SibSp  \\\n",
       "count   891.000000  891.000000  891.000000  714.000000  891.000000   \n",
       "mean    446.000000    0.383838    2.308642   29.699118    0.523008   \n",
       "std     257.353842    0.486592    0.836071   14.526497    1.102743   \n",
       "min       1.000000    0.000000    1.000000    0.420000    0.000000   \n",
       "25%     223.500000    0.000000    2.000000         NaN    0.000000   \n",
       "50%     446.000000    0.000000    3.000000         NaN    0.000000   \n",
       "75%     668.500000    1.000000    3.000000         NaN    1.000000   \n",
       "max     891.000000    1.000000    3.000000   80.000000    8.000000   \n",
       "\n",
       "            Parch        Fare  \n",
       "count  891.000000  891.000000  \n",
       "mean     0.381594   32.204208  \n",
       "std      0.806057   49.693429  \n",
       "min      0.000000    0.000000  \n",
       "25%      0.000000    7.910400  \n",
       "50%      0.000000   14.454200  \n",
       "75%      0.000000   31.000000  \n",
       "max      6.000000  512.329200  "
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data_train.describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<font color=red>mean字段告诉我们，大概0.383838的人最后获救了，2/3等舱的人数比1等舱要多，平均乘客年龄大概是29.7岁(计算这个时候会略掉无记录的)等等…<font>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* <font color=red>『对数据的认识太重要了！』<font>\n",
    "* <font color=red>『对数据的认识太重要了！』<font>\n",
    "* <font color=red>『对数据的认识太重要了！』<font>\n",
    "\n",
    "<font color=red>口号喊完了，上面的简单描述信息并没有什么卵用啊，咱们得再细一点分析下数据啊。<font><br>\n",
    "<font color=red>看看**每个/多个 属性和最后的Survived**之间有着什么样的关系<font>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "fig = plt.figure()\n",
    "fig.set(alpha=0.2)  # 设定图表颜色alpha参数\n",
    "\n",
    "plt.subplot2grid((2,3),(0,0))             # 在一张大图里分列几个小图\n",
    "data_train.Survived.value_counts().plot(kind='bar')# plots a bar graph of those who surived vs those who did not. \n",
    "plt.title(u\"获救情况 (1为获救)\") # puts a title on our graph\n",
    "plt.ylabel(u\"人数\")  \n",
    "\n",
    "plt.subplot2grid((2,3),(0,1))\n",
    "data_train.Pclass.value_counts().plot(kind=\"bar\")\n",
    "plt.ylabel(u\"人数\")\n",
    "plt.title(u\"乘客等级分布\")\n",
    "\n",
    "plt.subplot2grid((2,3),(0,2))\n",
    "plt.scatter(data_train.Survived, data_train.Age)\n",
    "plt.ylabel(u\"年龄\")                         # sets the y axis lable\n",
    "plt.grid(b=True, which='major', axis='y') # formats the grid line style of our graphs\n",
    "plt.title(u\"按年龄看获救分布 (1为获救)\")\n",
    "\n",
    "\n",
    "plt.subplot2grid((2,3),(1,0), colspan=2)\n",
    "data_train.Age[data_train.Pclass == 1].plot(kind='kde')   # plots a kernel desnsity estimate of the subset of the 1st class passanges's age\n",
    "data_train.Age[data_train.Pclass == 2].plot(kind='kde')\n",
    "data_train.Age[data_train.Pclass == 3].plot(kind='kde')\n",
    "plt.xlabel(u\"年龄\")# plots an axis lable\n",
    "plt.ylabel(u\"密度\") \n",
    "plt.title(u\"各等级的乘客年龄分布\")\n",
    "plt.legend((u'头等舱', u'2等舱',u'3等舱'),loc='best') # sets our legend for our graph.\n",
    "\n",
    "\n",
    "plt.subplot2grid((2,3),(1,2))\n",
    "data_train.Embarked.value_counts().plot(kind='bar')\n",
    "plt.title(u\"各登船口岸上船人数\")\n",
    "plt.ylabel(u\"人数\")  \n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "于是得到了像下面这样一张图：<br>\n",
    "![](http://7xo0y8.com1.z0.glb.clouddn.com/2_titanic/8.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<font color=red>bingo，图还是比数字好看多了。所以我们在图上可以看出来:<font><br>\n",
    "* <font color=red>被救的人300多点，不到半数；<font><br>\n",
    "* <font color=red>3等舱乘客灰常多；遇难和获救的人年龄似乎跨度都很广；<font><br>\n",
    "* <font color=red>3个不同的舱年龄总体趋势似乎也一致，2/3等舱乘客20岁多点的人最多，1等舱40岁左右的最多(→_→似乎符合财富和年龄的分配哈，咳咳，别理我，我瞎扯的)；<font><br>\n",
    "* <font color=red>登船港口人数按照S、C、Q递减，而且S远多于另外俩港口。<font><br><br>\n",
    "\n",
    "<font color=red>这个时候我们可能会有一些想法了：<font><br><br>\n",
    "\n",
    "1. <font color=red>不同舱位/乘客等级可能和财富/地位有关系，最后获救概率可能会不一样<font><br>\n",
    "2. <font color=red>年龄对获救概率也一定是有影响的，毕竟前面说了，副船长还说『小孩和女士先走』呢<font><br>\n",
    "3. <font color=red>和登船港口是不是有关系呢？也许登船港口不同，人的出身地位不同？<font><br>\n",
    "\n",
    "<font color=red>口说无凭，空想无益。老老实实再来统计统计，看看这些属性值的统计分布吧。<font>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 552,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "#看看各乘客等级的获救情况\n",
    "fig = plt.figure()\n",
    "fig.set(alpha=0.2)  # 设定图表颜色alpha参数\n",
    "\n",
    "Survived_0 = data_train.Pclass[data_train.Survived == 0].value_counts()\n",
    "Survived_1 = data_train.Pclass[data_train.Survived == 1].value_counts()\n",
    "df=pd.DataFrame({u'获救':Survived_1, u'未获救':Survived_0})\n",
    "df.plot(kind='bar', stacked=True)\n",
    "plt.title(u\"各乘客等级的获救情况\")\n",
    "plt.xlabel(u\"乘客等级\") \n",
    "plt.ylabel(u\"人数\") \n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "<font color=red>得到这个图：<font><br>\n",
    "![](http://7xo0y8.com1.z0.glb.clouddn.com/2_titanic/9.png?imageView/2/w/450/q/100)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<font color=red>啧啧，果然，钱和地位对舱位有影响，进而对获救的可能性也有影响啊←_← <font><br>\n",
    "<font color=red>咳咳，跑题了，我想说的是，明显等级为1的乘客，获救的概率高很多。恩，这个一定是影响最后获救结果的一个特征。<font>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 553,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#看看各登录港口的获救情况\n",
    "fig = plt.figure()\n",
    "fig.set(alpha=0.2)  # 设定图表颜色alpha参数\n",
    "\n",
    "Survived_0 = data_train.Embarked[data_train.Survived == 0].value_counts()\n",
    "Survived_1 = data_train.Embarked[data_train.Survived == 1].value_counts()\n",
    "df=pd.DataFrame({u'获救':Survived_1, u'未获救':Survived_0})\n",
    "df.plot(kind='bar', stacked=True)\n",
    "plt.title(u\"各登录港口乘客的获救情况\")\n",
    "plt.xlabel(u\"登录港口\") \n",
    "plt.ylabel(u\"人数\") \n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](http://7xo0y8.com1.z0.glb.clouddn.com/2_titanic/Embarked.png?imageView/2/w/500/q/100)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<font color=red>并没有看出什么...<font><br>\n",
    "\n",
    "<font color=red>那个，看看性别好了<font><br>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 507,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "#看看各性别的获救情况\n",
    "fig = plt.figure()\n",
    "fig.set(alpha=0.2)  # 设定图表颜色alpha参数\n",
    "\n",
    "Survived_m = data_train.Survived[data_train.Sex == 'male'].value_counts()\n",
    "Survived_f = data_train.Survived[data_train.Sex == 'female'].value_counts()\n",
    "df=pd.DataFrame({u'男性':Survived_m, u'女性':Survived_f})\n",
    "df.plot(kind='bar', stacked=True)\n",
    "plt.title(u\"按性别看获救情况\")\n",
    "plt.xlabel(u\"性别\") \n",
    "plt.ylabel(u\"人数\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](http://7xo0y8.com1.z0.glb.clouddn.com/2_titanic/10.png?imageView/2/w/450/q/100)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<font color=red>歪果盆友果然很尊重lady，lady first践行得不错。性别无疑也要作为重要特征加入最后的模型之中。<font><br>\n",
    "\n",
    "<font color=red>再来个详细版的好了<font><br>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 508,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "#然后我们再来看看各种舱级别情况下各性别的获救情况\n",
    "fig=plt.figure()\n",
    "fig.set(alpha=0.65) # 设置图像透明度，无所谓\n",
    "plt.title(u\"根据舱等级和性别的获救情况\")\n",
    "\n",
    "ax1=fig.add_subplot(141)\n",
    "data_train.Survived[data_train.Sex == 'female'][data_train.Pclass != 3].value_counts().plot(kind='bar', label=\"female highclass\", color='#FA2479')\n",
    "ax1.set_xticklabels([u\"获救\", u\"未获救\"], rotation=0)\n",
    "ax1.legend([u\"女性/高级舱\"], loc='best')\n",
    "\n",
    "ax2=fig.add_subplot(142, sharey=ax1)\n",
    "data_train.Survived[data_train.Sex == 'female'][data_train.Pclass == 3].value_counts().plot(kind='bar', label='female, low class', color='pink')\n",
    "ax2.set_xticklabels([u\"未获救\", u\"获救\"], rotation=0)\n",
    "plt.legend([u\"女性/低级舱\"], loc='best')\n",
    "\n",
    "ax3=fig.add_subplot(143, sharey=ax1)\n",
    "data_train.Survived[data_train.Sex == 'male'][data_train.Pclass != 3].value_counts().plot(kind='bar', label='male, high class',color='lightblue')\n",
    "ax3.set_xticklabels([u\"未获救\", u\"获救\"], rotation=0)\n",
    "plt.legend([u\"男性/高级舱\"], loc='best')\n",
    "\n",
    "ax4=fig.add_subplot(144, sharey=ax1)\n",
    "data_train.Survived[data_train.Sex == 'male'][data_train.Pclass == 3].value_counts().plot(kind='bar', label='male low class', color='steelblue')\n",
    "ax4.set_xticklabels([u\"未获救\", u\"获救\"], rotation=0)\n",
    "plt.legend([u\"男性/低级舱\"], loc='best')\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](http://7xo0y8.com1.z0.glb.clouddn.com/2_titanic/11.png?imageView/2/w/700/q/100)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<font color=red>那堂兄弟和父母呢？<font>\n",
    "<font color=red>大家族会有优势么？<font><br>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 509,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div style=\"max-height:1000px;max-width:1500px;overflow:auto;\">\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>SibSp</th>\n",
       "      <th>Survived</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th rowspan=\"2\" valign=\"top\">0</th>\n",
       "      <th>0</th>\n",
       "      <td>398</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>210</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"2\" valign=\"top\">1</th>\n",
       "      <th>0</th>\n",
       "      <td>97</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>112</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"2\" valign=\"top\">2</th>\n",
       "      <th>0</th>\n",
       "      <td>15</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"2\" valign=\"top\">3</th>\n",
       "      <th>0</th>\n",
       "      <td>12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"2\" valign=\"top\">4</th>\n",
       "      <th>0</th>\n",
       "      <td>15</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <th>0</th>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <th>0</th>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                PassengerId\n",
       "SibSp Survived             \n",
       "0     0                 398\n",
       "      1                 210\n",
       "1     0                  97\n",
       "      1                 112\n",
       "2     0                  15\n",
       "      1                  13\n",
       "3     0                  12\n",
       "      1                   4\n",
       "4     0                  15\n",
       "      1                   3\n",
       "5     0                   5\n",
       "8     0                   7"
      ]
     },
     "execution_count": 509,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "g = data_train.groupby(['SibSp','Survived'])\n",
    "df = pd.DataFrame(g.count()['PassengerId'])\n",
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 168,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div style=\"max-height:1000px;max-width:1500px;overflow:auto;\">\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Parch</th>\n",
       "      <th>Survived</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th rowspan=\"2\" valign=\"top\">0</th>\n",
       "      <th>0</th>\n",
       "      <td>445</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>233</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"2\" valign=\"top\">1</th>\n",
       "      <th>0</th>\n",
       "      <td>53</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>65</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"2\" valign=\"top\">2</th>\n",
       "      <th>0</th>\n",
       "      <td>40</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>40</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"2\" valign=\"top\">3</th>\n",
       "      <th>0</th>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <th>0</th>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"2\" valign=\"top\">5</th>\n",
       "      <th>0</th>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                PassengerId\n",
       "Parch Survived             \n",
       "0     0                 445\n",
       "      1                 233\n",
       "1     0                  53\n",
       "      1                  65\n",
       "2     0                  40\n",
       "      1                  40\n",
       "3     0                   2\n",
       "      1                   3\n",
       "4     0                   4\n",
       "5     0                   4\n",
       "      1                   1\n",
       "6     0                   1"
      ]
     },
     "execution_count": 168,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "g = data_train.groupby(['Parch','Survived'])\n",
    "df = pd.DataFrame(g.count()['PassengerId'])\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<font color=red>好吧，没看出特别特别明显的规律(为自己的智商感到捉急…)，先作为备选特征，放一放。<font><br>\n",
    "<font color=red>看看船票好了<font><br>\n",
    "<font color=red>ticket是船票编号，应该是unique的，和最后的结果没有太大的关系，不纳入考虑的特征范畴<font><br>\n",
    "<font color=red>cabin只有204个乘客有值，我们先看看它的一个分布<font><br>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 191,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "C23 C25 C27        4\n",
       "G6                 4\n",
       "B96 B98            4\n",
       "D                  3\n",
       "C22 C26            3\n",
       "E101               3\n",
       "F2                 3\n",
       "F33                3\n",
       "B57 B59 B63 B66    2\n",
       "C68                2\n",
       "B58 B60            2\n",
       "E121               2\n",
       "D20                2\n",
       "E8                 2\n",
       "E44                2\n",
       "B77                2\n",
       "C65                2\n",
       "D26                2\n",
       "E24                2\n",
       "E25                2\n",
       "B20                2\n",
       "C93                2\n",
       "D33                2\n",
       "E67                2\n",
       "D35                2\n",
       "D36                2\n",
       "C52                2\n",
       "F4                 2\n",
       "C125               2\n",
       "C124               2\n",
       "                  ..\n",
       "F G63              1\n",
       "A6                 1\n",
       "D45                1\n",
       "D6                 1\n",
       "D56                1\n",
       "C101               1\n",
       "C54                1\n",
       "D28                1\n",
       "D37                1\n",
       "B102               1\n",
       "D30                1\n",
       "E17                1\n",
       "E58                1\n",
       "F E69              1\n",
       "D10 D12            1\n",
       "E50                1\n",
       "A14                1\n",
       "C91                1\n",
       "A16                1\n",
       "B38                1\n",
       "B39                1\n",
       "C95                1\n",
       "B78                1\n",
       "B79                1\n",
       "C99                1\n",
       "B37                1\n",
       "A19                1\n",
       "E12                1\n",
       "A7                 1\n",
       "D15                1\n",
       "dtype: int64"
      ]
     },
     "execution_count": 191,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#ticket是船票编号，应该是unique的，和最后的结果没有太大的关系，不纳入考虑的特征范畴\n",
    "#cabin只有204个乘客有值，我们先看看它的一个分布\n",
    "data_train.Cabin.value_counts()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<font color=red>这三三两两的…如此不集中…我们猜一下，也许，前面的ABCDE是指的甲板位置、然后编号是房间号？…好吧，我瞎说的，别当真…<font><br>\n",
    "<font color=red>关键是Cabin这鬼属性，应该算作类目型的，本来缺失值就多，还如此不集中，注定是个棘手货…第一感觉，这玩意儿如果直接按照类目特征处理的话，太散了，估计每个因子化后的特征都拿不到什么权重。加上有那么多缺失值，要不我们先把Cabin缺失与否作为条件(虽然这部分信息缺失可能并非未登记，maybe只是丢失了而已，所以这样做未必妥当)，先在有无Cabin信息这个粗粒度上看看Survived的情况好了。<font><br>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 511,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "#cabin的值计数太分散了，绝大多数Cabin值只出现一次。感觉上作为类目，加入特征未必会有效\n",
    "#那我们一起看看这个值的有无，对于survival的分布状况，影响如何吧\n",
    "fig = plt.figure()\n",
    "fig.set(alpha=0.2)  # 设定图表颜色alpha参数\n",
    "\n",
    "Survived_cabin = data_train.Survived[pd.notnull(data_train.Cabin)].value_counts()\n",
    "Survived_nocabin = data_train.Survived[pd.isnull(data_train.Cabin)].value_counts()\n",
    "df=pd.DataFrame({u'有':Survived_cabin, u'无':Survived_nocabin}).transpose()\n",
    "df.plot(kind='bar', stacked=True)\n",
    "plt.title(u\"按Cabin有无看获救情况\")\n",
    "plt.xlabel(u\"Cabin有无\") \n",
    "plt.ylabel(u\"人数\")\n",
    "plt.show()\n",
    "\n",
    "#似乎有cabin记录的乘客survival比例稍高，那先试试把这个值分为两类，有cabin值/无cabin值，一会儿加到类别特征好了"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](http://7xo0y8.com1.z0.glb.clouddn.com/2_titanic/15.png?imageView/2/w/400/q/100)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<font color=red>有Cabin记录的似乎获救概率稍高一些，先这么着放一放吧。<font><br><br>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<font color=red>先从最突出的数据属性开始吧，对，Cabin和Age，有丢失数据实在是对下一步工作影响太大。<font><br>\n",
    "\n",
    "<font color=red>先说Cabin，暂时我们就按照刚才说的，按Cabin有无数据，将这个属性处理成Yes和No两种类型吧。<font><br>\n",
    "\n",
    "<font color=red>再说Age：<font><br>\n",
    "\n",
    "<font color=red>通常遇到缺值的情况，我们会有几种常见的处理方式<font><br>\n",
    "\n",
    "1. <font color=red>如果缺值的样本占总数比例极高，我们可能就直接舍弃了，作为特征加入的话，可能反倒带入noise，影响最后的结果了<font><br>\n",
    "2. <font color=red>如果缺值的样本适中，而该属性非连续值特征属性(比如说类目属性)，那就把NaN作为一个新类别，加到类别特征中<font><br>\n",
    "3. <font color=red>如果缺值的样本适中，而该属性为连续值特征属性，有时候我们会考虑给定一个step(比如这里的age，我们可以考虑每隔2/3岁为一个步长)，然后把它离散化，之后把NaN作为一个type加到属性类目中。<font><br>\n",
    "4. <font color=red>有些情况下，缺失的值个数并不是特别多，那我们也可以试着根据已有的值，拟合一下数据，补充上。<font><br>\n",
    "<font color=red>本例中，后两种处理方式应该都是可行的，我们先试试拟合补全吧(虽然说没有特别多的背景可供我们拟合，这不一定是一个多么好的选择)<font><br>\n",
    "\n",
    "<font color=red>我们这里用scikit-learn中的RandomForest来拟合一下缺失的年龄数据<font><br>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 564,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div style=\"max-height:1000px;max-width:1500px;overflow:auto;\">\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Name</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Ticket</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Embarked</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Braund, Mr. Owen Harris</td>\n",
       "      <td>male</td>\n",
       "      <td>22.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>A/5 21171</td>\n",
       "      <td>7.2500</td>\n",
       "      <td>No</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Cumings, Mrs. John Bradley (Florence Briggs Th...</td>\n",
       "      <td>female</td>\n",
       "      <td>38.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>PC 17599</td>\n",
       "      <td>71.2833</td>\n",
       "      <td>Yes</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>Heikkinen, Miss. Laina</td>\n",
       "      <td>female</td>\n",
       "      <td>26.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>STON/O2. 3101282</td>\n",
       "      <td>7.9250</td>\n",
       "      <td>No</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Futrelle, Mrs. Jacques Heath (Lily May Peel)</td>\n",
       "      <td>female</td>\n",
       "      <td>35.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>113803</td>\n",
       "      <td>53.1000</td>\n",
       "      <td>Yes</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Allen, Mr. William Henry</td>\n",
       "      <td>male</td>\n",
       "      <td>35.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>373450</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>No</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>6</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Moran, Mr. James</td>\n",
       "      <td>male</td>\n",
       "      <td>23.828953</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>330877</td>\n",
       "      <td>8.4583</td>\n",
       "      <td>No</td>\n",
       "      <td>Q</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>7</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>McCarthy, Mr. Timothy J</td>\n",
       "      <td>male</td>\n",
       "      <td>54.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>17463</td>\n",
       "      <td>51.8625</td>\n",
       "      <td>Yes</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>8</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Palsson, Master. Gosta Leonard</td>\n",
       "      <td>male</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>349909</td>\n",
       "      <td>21.0750</td>\n",
       "      <td>No</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>9</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>Johnson, Mrs. Oscar W (Elisabeth Vilhelmina Berg)</td>\n",
       "      <td>female</td>\n",
       "      <td>27.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>347742</td>\n",
       "      <td>11.1333</td>\n",
       "      <td>No</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>10</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>Nasser, Mrs. Nicholas (Adele Achem)</td>\n",
       "      <td>female</td>\n",
       "      <td>14.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>237736</td>\n",
       "      <td>30.0708</td>\n",
       "      <td>No</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>11</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>Sandstrom, Miss. Marguerite Rut</td>\n",
       "      <td>female</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>PP 9549</td>\n",
       "      <td>16.7000</td>\n",
       "      <td>Yes</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>12</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Bonnell, Miss. Elizabeth</td>\n",
       "      <td>female</td>\n",
       "      <td>58.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>113783</td>\n",
       "      <td>26.5500</td>\n",
       "      <td>Yes</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>13</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Saundercock, Mr. William Henry</td>\n",
       "      <td>male</td>\n",
       "      <td>20.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>A/5. 2151</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>No</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>14</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Andersson, Mr. Anders Johan</td>\n",
       "      <td>male</td>\n",
       "      <td>39.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>347082</td>\n",
       "      <td>31.2750</td>\n",
       "      <td>No</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>15</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Vestrom, Miss. Hulda Amanda Adolfina</td>\n",
       "      <td>female</td>\n",
       "      <td>14.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>350406</td>\n",
       "      <td>7.8542</td>\n",
       "      <td>No</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>16</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>Hewlett, Mrs. (Mary D Kingcome)</td>\n",
       "      <td>female</td>\n",
       "      <td>55.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>248706</td>\n",
       "      <td>16.0000</td>\n",
       "      <td>No</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>17</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Rice, Master. Eugene</td>\n",
       "      <td>male</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>382652</td>\n",
       "      <td>29.1250</td>\n",
       "      <td>No</td>\n",
       "      <td>Q</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>18</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>Williams, Mr. Charles Eugene</td>\n",
       "      <td>male</td>\n",
       "      <td>32.066493</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>244373</td>\n",
       "      <td>13.0000</td>\n",
       "      <td>No</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>19</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Vander Planke, Mrs. Julius (Emelia Maria Vande...</td>\n",
       "      <td>female</td>\n",
       "      <td>31.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>345763</td>\n",
       "      <td>18.0000</td>\n",
       "      <td>No</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>20</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>Masselmani, Mrs. Fatima</td>\n",
       "      <td>female</td>\n",
       "      <td>29.518205</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2649</td>\n",
       "      <td>7.2250</td>\n",
       "      <td>No</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>21</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>Fynney, Mr. Joseph J</td>\n",
       "      <td>male</td>\n",
       "      <td>35.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>239865</td>\n",
       "      <td>26.0000</td>\n",
       "      <td>No</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>22</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>Beesley, Mr. Lawrence</td>\n",
       "      <td>male</td>\n",
       "      <td>34.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>248698</td>\n",
       "      <td>13.0000</td>\n",
       "      <td>Yes</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>23</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>McGowan, Miss. Anna \"Annie\"</td>\n",
       "      <td>female</td>\n",
       "      <td>15.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>330923</td>\n",
       "      <td>8.0292</td>\n",
       "      <td>No</td>\n",
       "      <td>Q</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>24</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Sloper, Mr. William Thompson</td>\n",
       "      <td>male</td>\n",
       "      <td>28.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>113788</td>\n",
       "      <td>35.5000</td>\n",
       "      <td>Yes</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>25</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Palsson, Miss. Torborg Danira</td>\n",
       "      <td>female</td>\n",
       "      <td>8.000000</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>349909</td>\n",
       "      <td>21.0750</td>\n",
       "      <td>No</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>26</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>Asplund, Mrs. Carl Oscar (Selma Augusta Emilia...</td>\n",
       "      <td>female</td>\n",
       "      <td>38.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>347077</td>\n",
       "      <td>31.3875</td>\n",
       "      <td>No</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>27</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Emir, Mr. Farred Chehab</td>\n",
       "      <td>male</td>\n",
       "      <td>29.518205</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2631</td>\n",
       "      <td>7.2250</td>\n",
       "      <td>No</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>28</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>Fortune, Mr. Charles Alexander</td>\n",
       "      <td>male</td>\n",
       "      <td>19.000000</td>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "      <td>19950</td>\n",
       "      <td>263.0000</td>\n",
       "      <td>Yes</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>29</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>O'Dwyer, Miss. Ellen \"Nellie\"</td>\n",
       "      <td>female</td>\n",
       "      <td>22.380113</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>330959</td>\n",
       "      <td>7.8792</td>\n",
       "      <td>No</td>\n",
       "      <td>Q</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>30</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Todoroff, Mr. Lalio</td>\n",
       "      <td>male</td>\n",
       "      <td>27.947206</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>349216</td>\n",
       "      <td>7.8958</td>\n",
       "      <td>No</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>861</th>\n",
       "      <td>862</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>Giles, Mr. Frederick Edward</td>\n",
       "      <td>male</td>\n",
       "      <td>21.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>28134</td>\n",
       "      <td>11.5000</td>\n",
       "      <td>No</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>862</th>\n",
       "      <td>863</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Swift, Mrs. Frederick Joel (Margaret Welles Ba...</td>\n",
       "      <td>female</td>\n",
       "      <td>48.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>17466</td>\n",
       "      <td>25.9292</td>\n",
       "      <td>Yes</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>863</th>\n",
       "      <td>864</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Sage, Miss. Dorothy Edith \"Dolly\"</td>\n",
       "      <td>female</td>\n",
       "      <td>10.888325</td>\n",
       "      <td>8</td>\n",
       "      <td>2</td>\n",
       "      <td>CA. 2343</td>\n",
       "      <td>69.5500</td>\n",
       "      <td>No</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>864</th>\n",
       "      <td>865</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>Gill, Mr. John William</td>\n",
       "      <td>male</td>\n",
       "      <td>24.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>233866</td>\n",
       "      <td>13.0000</td>\n",
       "      <td>No</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>865</th>\n",
       "      <td>866</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>Bystrom, Mrs. (Karolina)</td>\n",
       "      <td>female</td>\n",
       "      <td>42.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>236852</td>\n",
       "      <td>13.0000</td>\n",
       "      <td>No</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>866</th>\n",
       "      <td>867</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>Duran y More, Miss. Asuncion</td>\n",
       "      <td>female</td>\n",
       "      <td>27.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>SC/PARIS 2149</td>\n",
       "      <td>13.8583</td>\n",
       "      <td>No</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>867</th>\n",
       "      <td>868</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>Roebling, Mr. Washington Augustus II</td>\n",
       "      <td>male</td>\n",
       "      <td>31.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>PC 17590</td>\n",
       "      <td>50.4958</td>\n",
       "      <td>Yes</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>868</th>\n",
       "      <td>869</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>van Melkebeke, Mr. Philemon</td>\n",
       "      <td>male</td>\n",
       "      <td>25.977889</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>345777</td>\n",
       "      <td>9.5000</td>\n",
       "      <td>No</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>869</th>\n",
       "      <td>870</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>Johnson, Master. Harold Theodor</td>\n",
       "      <td>male</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>347742</td>\n",
       "      <td>11.1333</td>\n",
       "      <td>No</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>870</th>\n",
       "      <td>871</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Balkic, Mr. Cerin</td>\n",
       "      <td>male</td>\n",
       "      <td>26.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>349248</td>\n",
       "      <td>7.8958</td>\n",
       "      <td>No</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>871</th>\n",
       "      <td>872</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Beckwith, Mrs. Richard Leonard (Sallie Monypeny)</td>\n",
       "      <td>female</td>\n",
       "      <td>47.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>11751</td>\n",
       "      <td>52.5542</td>\n",
       "      <td>Yes</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>872</th>\n",
       "      <td>873</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>Carlsson, Mr. Frans Olof</td>\n",
       "      <td>male</td>\n",
       "      <td>33.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>695</td>\n",
       "      <td>5.0000</td>\n",
       "      <td>Yes</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>873</th>\n",
       "      <td>874</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Vander Cruyssen, Mr. Victor</td>\n",
       "      <td>male</td>\n",
       "      <td>47.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>345765</td>\n",
       "      <td>9.0000</td>\n",
       "      <td>No</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>874</th>\n",
       "      <td>875</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>Abelson, Mrs. Samuel (Hannah Wizosky)</td>\n",
       "      <td>female</td>\n",
       "      <td>28.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>P/PP 3381</td>\n",
       "      <td>24.0000</td>\n",
       "      <td>No</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>875</th>\n",
       "      <td>876</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>Najib, Miss. Adele Kiamie \"Jane\"</td>\n",
       "      <td>female</td>\n",
       "      <td>15.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2667</td>\n",
       "      <td>7.2250</td>\n",
       "      <td>No</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>876</th>\n",
       "      <td>877</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Gustafsson, Mr. Alfred Ossian</td>\n",
       "      <td>male</td>\n",
       "      <td>20.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7534</td>\n",
       "      <td>9.8458</td>\n",
       "      <td>No</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>877</th>\n",
       "      <td>878</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Petroff, Mr. Nedelio</td>\n",
       "      <td>male</td>\n",
       "      <td>19.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>349212</td>\n",
       "      <td>7.8958</td>\n",
       "      <td>No</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>878</th>\n",
       "      <td>879</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Laleff, Mr. Kristo</td>\n",
       "      <td>male</td>\n",
       "      <td>27.947206</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>349217</td>\n",
       "      <td>7.8958</td>\n",
       "      <td>No</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>879</th>\n",
       "      <td>880</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Potter, Mrs. Thomas Jr (Lily Alexenia Wilson)</td>\n",
       "      <td>female</td>\n",
       "      <td>56.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>11767</td>\n",
       "      <td>83.1583</td>\n",
       "      <td>Yes</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>880</th>\n",
       "      <td>881</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>Shelley, Mrs. William (Imanita Parrish Hall)</td>\n",
       "      <td>female</td>\n",
       "      <td>25.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>230433</td>\n",
       "      <td>26.0000</td>\n",
       "      <td>No</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>881</th>\n",
       "      <td>882</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Markun, Mr. Johann</td>\n",
       "      <td>male</td>\n",
       "      <td>33.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>349257</td>\n",
       "      <td>7.8958</td>\n",
       "      <td>No</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>882</th>\n",
       "      <td>883</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Dahlberg, Miss. Gerda Ulrika</td>\n",
       "      <td>female</td>\n",
       "      <td>22.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7552</td>\n",
       "      <td>10.5167</td>\n",
       "      <td>No</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>883</th>\n",
       "      <td>884</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>Banfield, Mr. Frederick James</td>\n",
       "      <td>male</td>\n",
       "      <td>28.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>C.A./SOTON 34068</td>\n",
       "      <td>10.5000</td>\n",
       "      <td>No</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>884</th>\n",
       "      <td>885</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Sutehall, Mr. Henry Jr</td>\n",
       "      <td>male</td>\n",
       "      <td>25.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>SOTON/OQ 392076</td>\n",
       "      <td>7.0500</td>\n",
       "      <td>No</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>885</th>\n",
       "      <td>886</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Rice, Mrs. William (Margaret Norton)</td>\n",
       "      <td>female</td>\n",
       "      <td>39.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>382652</td>\n",
       "      <td>29.1250</td>\n",
       "      <td>No</td>\n",
       "      <td>Q</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>886</th>\n",
       "      <td>887</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>Montvila, Rev. Juozas</td>\n",
       "      <td>male</td>\n",
       "      <td>27.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>211536</td>\n",
       "      <td>13.0000</td>\n",
       "      <td>No</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>887</th>\n",
       "      <td>888</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Graham, Miss. Margaret Edith</td>\n",
       "      <td>female</td>\n",
       "      <td>19.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>112053</td>\n",
       "      <td>30.0000</td>\n",
       "      <td>Yes</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>888</th>\n",
       "      <td>889</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Johnston, Miss. Catherine Helen \"Carrie\"</td>\n",
       "      <td>female</td>\n",
       "      <td>16.232379</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>W./C. 6607</td>\n",
       "      <td>23.4500</td>\n",
       "      <td>No</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>889</th>\n",
       "      <td>890</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Behr, Mr. Karl Howell</td>\n",
       "      <td>male</td>\n",
       "      <td>26.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>111369</td>\n",
       "      <td>30.0000</td>\n",
       "      <td>Yes</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>890</th>\n",
       "      <td>891</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Dooley, Mr. Patrick</td>\n",
       "      <td>male</td>\n",
       "      <td>32.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>370376</td>\n",
       "      <td>7.7500</td>\n",
       "      <td>No</td>\n",
       "      <td>Q</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>891 rows × 12 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     PassengerId  Survived  Pclass  \\\n",
       "0              1         0       3   \n",
       "1              2         1       1   \n",
       "2              3         1       3   \n",
       "3              4         1       1   \n",
       "4              5         0       3   \n",
       "5              6         0       3   \n",
       "6              7         0       1   \n",
       "7              8         0       3   \n",
       "8              9         1       3   \n",
       "9             10         1       2   \n",
       "10            11         1       3   \n",
       "11            12         1       1   \n",
       "12            13         0       3   \n",
       "13            14         0       3   \n",
       "14            15         0       3   \n",
       "15            16         1       2   \n",
       "16            17         0       3   \n",
       "17            18         1       2   \n",
       "18            19         0       3   \n",
       "19            20         1       3   \n",
       "20            21         0       2   \n",
       "21            22         1       2   \n",
       "22            23         1       3   \n",
       "23            24         1       1   \n",
       "24            25         0       3   \n",
       "25            26         1       3   \n",
       "26            27         0       3   \n",
       "27            28         0       1   \n",
       "28            29         1       3   \n",
       "29            30         0       3   \n",
       "..           ...       ...     ...   \n",
       "861          862         0       2   \n",
       "862          863         1       1   \n",
       "863          864         0       3   \n",
       "864          865         0       2   \n",
       "865          866         1       2   \n",
       "866          867         1       2   \n",
       "867          868         0       1   \n",
       "868          869         0       3   \n",
       "869          870         1       3   \n",
       "870          871         0       3   \n",
       "871          872         1       1   \n",
       "872          873         0       1   \n",
       "873          874         0       3   \n",
       "874          875         1       2   \n",
       "875          876         1       3   \n",
       "876          877         0       3   \n",
       "877          878         0       3   \n",
       "878          879         0       3   \n",
       "879          880         1       1   \n",
       "880          881         1       2   \n",
       "881          882         0       3   \n",
       "882          883         0       3   \n",
       "883          884         0       2   \n",
       "884          885         0       3   \n",
       "885          886         0       3   \n",
       "886          887         0       2   \n",
       "887          888         1       1   \n",
       "888          889         0       3   \n",
       "889          890         1       1   \n",
       "890          891         0       3   \n",
       "\n",
       "                                                  Name     Sex        Age  \\\n",
       "0                              Braund, Mr. Owen Harris    male  22.000000   \n",
       "1    Cumings, Mrs. John Bradley (Florence Briggs Th...  female  38.000000   \n",
       "2                               Heikkinen, Miss. Laina  female  26.000000   \n",
       "3         Futrelle, Mrs. Jacques Heath (Lily May Peel)  female  35.000000   \n",
       "4                             Allen, Mr. William Henry    male  35.000000   \n",
       "5                                     Moran, Mr. James    male  23.828953   \n",
       "6                              McCarthy, Mr. Timothy J    male  54.000000   \n",
       "7                       Palsson, Master. Gosta Leonard    male   2.000000   \n",
       "8    Johnson, Mrs. Oscar W (Elisabeth Vilhelmina Berg)  female  27.000000   \n",
       "9                  Nasser, Mrs. Nicholas (Adele Achem)  female  14.000000   \n",
       "10                     Sandstrom, Miss. Marguerite Rut  female   4.000000   \n",
       "11                            Bonnell, Miss. Elizabeth  female  58.000000   \n",
       "12                      Saundercock, Mr. William Henry    male  20.000000   \n",
       "13                         Andersson, Mr. Anders Johan    male  39.000000   \n",
       "14                Vestrom, Miss. Hulda Amanda Adolfina  female  14.000000   \n",
       "15                    Hewlett, Mrs. (Mary D Kingcome)   female  55.000000   \n",
       "16                                Rice, Master. Eugene    male   2.000000   \n",
       "17                        Williams, Mr. Charles Eugene    male  32.066493   \n",
       "18   Vander Planke, Mrs. Julius (Emelia Maria Vande...  female  31.000000   \n",
       "19                             Masselmani, Mrs. Fatima  female  29.518205   \n",
       "20                                Fynney, Mr. Joseph J    male  35.000000   \n",
       "21                               Beesley, Mr. Lawrence    male  34.000000   \n",
       "22                         McGowan, Miss. Anna \"Annie\"  female  15.000000   \n",
       "23                        Sloper, Mr. William Thompson    male  28.000000   \n",
       "24                       Palsson, Miss. Torborg Danira  female   8.000000   \n",
       "25   Asplund, Mrs. Carl Oscar (Selma Augusta Emilia...  female  38.000000   \n",
       "26                             Emir, Mr. Farred Chehab    male  29.518205   \n",
       "27                      Fortune, Mr. Charles Alexander    male  19.000000   \n",
       "28                       O'Dwyer, Miss. Ellen \"Nellie\"  female  22.380113   \n",
       "29                                 Todoroff, Mr. Lalio    male  27.947206   \n",
       "..                                                 ...     ...        ...   \n",
       "861                        Giles, Mr. Frederick Edward    male  21.000000   \n",
       "862  Swift, Mrs. Frederick Joel (Margaret Welles Ba...  female  48.000000   \n",
       "863                  Sage, Miss. Dorothy Edith \"Dolly\"  female  10.888325   \n",
       "864                             Gill, Mr. John William    male  24.000000   \n",
       "865                           Bystrom, Mrs. (Karolina)  female  42.000000   \n",
       "866                       Duran y More, Miss. Asuncion  female  27.000000   \n",
       "867               Roebling, Mr. Washington Augustus II    male  31.000000   \n",
       "868                        van Melkebeke, Mr. Philemon    male  25.977889   \n",
       "869                    Johnson, Master. Harold Theodor    male   4.000000   \n",
       "870                                  Balkic, Mr. Cerin    male  26.000000   \n",
       "871   Beckwith, Mrs. Richard Leonard (Sallie Monypeny)  female  47.000000   \n",
       "872                           Carlsson, Mr. Frans Olof    male  33.000000   \n",
       "873                        Vander Cruyssen, Mr. Victor    male  47.000000   \n",
       "874              Abelson, Mrs. Samuel (Hannah Wizosky)  female  28.000000   \n",
       "875                   Najib, Miss. Adele Kiamie \"Jane\"  female  15.000000   \n",
       "876                      Gustafsson, Mr. Alfred Ossian    male  20.000000   \n",
       "877                               Petroff, Mr. Nedelio    male  19.000000   \n",
       "878                                 Laleff, Mr. Kristo    male  27.947206   \n",
       "879      Potter, Mrs. Thomas Jr (Lily Alexenia Wilson)  female  56.000000   \n",
       "880       Shelley, Mrs. William (Imanita Parrish Hall)  female  25.000000   \n",
       "881                                 Markun, Mr. Johann    male  33.000000   \n",
       "882                       Dahlberg, Miss. Gerda Ulrika  female  22.000000   \n",
       "883                      Banfield, Mr. Frederick James    male  28.000000   \n",
       "884                             Sutehall, Mr. Henry Jr    male  25.000000   \n",
       "885               Rice, Mrs. William (Margaret Norton)  female  39.000000   \n",
       "886                              Montvila, Rev. Juozas    male  27.000000   \n",
       "887                       Graham, Miss. Margaret Edith  female  19.000000   \n",
       "888           Johnston, Miss. Catherine Helen \"Carrie\"  female  16.232379   \n",
       "889                              Behr, Mr. Karl Howell    male  26.000000   \n",
       "890                                Dooley, Mr. Patrick    male  32.000000   \n",
       "\n",
       "     SibSp  Parch            Ticket      Fare Cabin Embarked  \n",
       "0        1      0         A/5 21171    7.2500    No        S  \n",
       "1        1      0          PC 17599   71.2833   Yes        C  \n",
       "2        0      0  STON/O2. 3101282    7.9250    No        S  \n",
       "3        1      0            113803   53.1000   Yes        S  \n",
       "4        0      0            373450    8.0500    No        S  \n",
       "5        0      0            330877    8.4583    No        Q  \n",
       "6        0      0             17463   51.8625   Yes        S  \n",
       "7        3      1            349909   21.0750    No        S  \n",
       "8        0      2            347742   11.1333    No        S  \n",
       "9        1      0            237736   30.0708    No        C  \n",
       "10       1      1           PP 9549   16.7000   Yes        S  \n",
       "11       0      0            113783   26.5500   Yes        S  \n",
       "12       0      0         A/5. 2151    8.0500    No        S  \n",
       "13       1      5            347082   31.2750    No        S  \n",
       "14       0      0            350406    7.8542    No        S  \n",
       "15       0      0            248706   16.0000    No        S  \n",
       "16       4      1            382652   29.1250    No        Q  \n",
       "17       0      0            244373   13.0000    No        S  \n",
       "18       1      0            345763   18.0000    No        S  \n",
       "19       0      0              2649    7.2250    No        C  \n",
       "20       0      0            239865   26.0000    No        S  \n",
       "21       0      0            248698   13.0000   Yes        S  \n",
       "22       0      0            330923    8.0292    No        Q  \n",
       "23       0      0            113788   35.5000   Yes        S  \n",
       "24       3      1            349909   21.0750    No        S  \n",
       "25       1      5            347077   31.3875    No        S  \n",
       "26       0      0              2631    7.2250    No        C  \n",
       "27       3      2             19950  263.0000   Yes        S  \n",
       "28       0      0            330959    7.8792    No        Q  \n",
       "29       0      0            349216    7.8958    No        S  \n",
       "..     ...    ...               ...       ...   ...      ...  \n",
       "861      1      0             28134   11.5000    No        S  \n",
       "862      0      0             17466   25.9292   Yes        S  \n",
       "863      8      2          CA. 2343   69.5500    No        S  \n",
       "864      0      0            233866   13.0000    No        S  \n",
       "865      0      0            236852   13.0000    No        S  \n",
       "866      1      0     SC/PARIS 2149   13.8583    No        C  \n",
       "867      0      0          PC 17590   50.4958   Yes        S  \n",
       "868      0      0            345777    9.5000    No        S  \n",
       "869      1      1            347742   11.1333    No        S  \n",
       "870      0      0            349248    7.8958    No        S  \n",
       "871      1      1             11751   52.5542   Yes        S  \n",
       "872      0      0               695    5.0000   Yes        S  \n",
       "873      0      0            345765    9.0000    No        S  \n",
       "874      1      0         P/PP 3381   24.0000    No        C  \n",
       "875      0      0              2667    7.2250    No        C  \n",
       "876      0      0              7534    9.8458    No        S  \n",
       "877      0      0            349212    7.8958    No        S  \n",
       "878      0      0            349217    7.8958    No        S  \n",
       "879      0      1             11767   83.1583   Yes        C  \n",
       "880      0      1            230433   26.0000    No        S  \n",
       "881      0      0            349257    7.8958    No        S  \n",
       "882      0      0              7552   10.5167    No        S  \n",
       "883      0      0  C.A./SOTON 34068   10.5000    No        S  \n",
       "884      0      0   SOTON/OQ 392076    7.0500    No        S  \n",
       "885      0      5            382652   29.1250    No        Q  \n",
       "886      0      0            211536   13.0000    No        S  \n",
       "887      0      0            112053   30.0000   Yes        S  \n",
       "888      1      2        W./C. 6607   23.4500    No        S  \n",
       "889      0      0            111369   30.0000   Yes        C  \n",
       "890      0      0            370376    7.7500    No        Q  \n",
       "\n",
       "[891 rows x 12 columns]"
      ]
     },
     "execution_count": 564,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.ensemble import RandomForestRegressor\n",
    " \n",
    "### 使用 RandomForestClassifier 填补缺失的年龄属性\n",
    "def set_missing_ages(df):\n",
    "    \n",
    "    # 把已有的数值型特征取出来丢进Random Forest Regressor中\n",
    "    age_df = df[['Age','Fare', 'Parch', 'SibSp', 'Pclass']]\n",
    "\n",
    "    # 乘客分成已知年龄和未知年龄两部分\n",
    "    known_age = age_df[age_df.Age.notnull()].as_matrix()\n",
    "    unknown_age = age_df[age_df.Age.isnull()].as_matrix()\n",
    "\n",
    "    # y即目标年龄\n",
    "    y = known_age[:, 0]\n",
    "\n",
    "    # X即特征属性值\n",
    "    X = known_age[:, 1:]\n",
    "\n",
    "    # fit到RandomForestRegressor之中\n",
    "    rfr = RandomForestRegressor(random_state=0, n_estimators=2000, n_jobs=-1)\n",
    "    rfr.fit(X, y)\n",
    "    \n",
    "    # 用得到的模型进行未知年龄结果预测\n",
    "    predictedAges = rfr.predict(unknown_age[:, 1::])\n",
    "    \n",
    "    # 用得到的预测结果填补原缺失数据\n",
    "    df.loc[ (df.Age.isnull()), 'Age' ] = predictedAges \n",
    "    \n",
    "    return df, rfr\n",
    "\n",
    "def set_Cabin_type(df):\n",
    "    df.loc[ (df.Cabin.notnull()), 'Cabin' ] = \"Yes\"\n",
    "    df.loc[ (df.Cabin.isnull()), 'Cabin' ] = \"No\"\n",
    "    return df\n",
    "\n",
    "data_train, rfr = set_missing_ages(data_train)\n",
    "data_train = set_Cabin_type(data_train)\n",
    "data_train"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<font color=red>因为逻辑回归建模时，需要输入的特征都是数值型特征，我们通常会先对类目型的特征因子化/one-hot编码。 <font><br>\n",
    "<font color=red>什么叫做因子化/one-hot编码？举个例子：<font><br>\n",
    "\n",
    "<font color=red>以Embarked为例，原本一个属性维度，因为其取值可以是[‘S’,’C’,’Q‘]，而将其平展开为’Embarked_C’,’Embarked_S’, ‘Embarked_Q’三个属性<font><br>\n",
    "\n",
    "* <font color=red>原本Embarked取值为S的，在此处的”Embarked_S”下取值为1，在’Embarked_C’, ‘Embarked_Q’下取值为0<font><br>\n",
    "* <font color=red>原本Embarked取值为C的，在此处的”Embarked_C”下取值为1，在’Embarked_S’, ‘Embarked_Q’下取值为0<font><br>\n",
    "* <font color=red>原本Embarked取值为Q的，在此处的”Embarked_Q”下取值为1，在’Embarked_C’, ‘Embarked_S’下取值为0<font><br>\n",
    "\n",
    "<font color=red>我们使用pandas的”get_dummies”来完成这个工作，并拼接在原来的”data_train”之上，如下所示。<font><br>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 565,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div style=\"max-height:1000px;max-width:1500px;overflow:auto;\">\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Survived</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Cabin_No</th>\n",
       "      <th>Cabin_Yes</th>\n",
       "      <th>Embarked_C</th>\n",
       "      <th>Embarked_Q</th>\n",
       "      <th>Embarked_S</th>\n",
       "      <th>Sex_female</th>\n",
       "      <th>Sex_male</th>\n",
       "      <th>Pclass_1</th>\n",
       "      <th>Pclass_2</th>\n",
       "      <th>Pclass_3</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>22.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>7.2500</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>38.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>71.2833</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>26.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7.9250</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>35.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>53.1000</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>35.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>6</td>\n",
       "      <td>0</td>\n",
       "      <td>23.828953</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>8.4583</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>7</td>\n",
       "      <td>0</td>\n",
       "      <td>54.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>51.8625</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>8</td>\n",
       "      <td>0</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>21.0750</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>9</td>\n",
       "      <td>1</td>\n",
       "      <td>27.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>11.1333</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>10</td>\n",
       "      <td>1</td>\n",
       "      <td>14.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>30.0708</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>11</td>\n",
       "      <td>1</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>16.7000</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>12</td>\n",
       "      <td>1</td>\n",
       "      <td>58.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>26.5500</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>13</td>\n",
       "      <td>0</td>\n",
       "      <td>20.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>14</td>\n",
       "      <td>0</td>\n",
       "      <td>39.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>31.2750</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>15</td>\n",
       "      <td>0</td>\n",
       "      <td>14.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7.8542</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>16</td>\n",
       "      <td>1</td>\n",
       "      <td>55.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>16.0000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>17</td>\n",
       "      <td>0</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>29.1250</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>18</td>\n",
       "      <td>1</td>\n",
       "      <td>32.066493</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>13.0000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>19</td>\n",
       "      <td>0</td>\n",
       "      <td>31.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>18.0000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>20</td>\n",
       "      <td>1</td>\n",
       "      <td>29.518205</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7.2250</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>21</td>\n",
       "      <td>0</td>\n",
       "      <td>35.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>26.0000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>22</td>\n",
       "      <td>1</td>\n",
       "      <td>34.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>13.0000</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>23</td>\n",
       "      <td>1</td>\n",
       "      <td>15.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>8.0292</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>24</td>\n",
       "      <td>1</td>\n",
       "      <td>28.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>35.5000</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>25</td>\n",
       "      <td>0</td>\n",
       "      <td>8.000000</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>21.0750</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>26</td>\n",
       "      <td>1</td>\n",
       "      <td>38.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>31.3875</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>27</td>\n",
       "      <td>0</td>\n",
       "      <td>29.518205</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7.2250</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>28</td>\n",
       "      <td>0</td>\n",
       "      <td>19.000000</td>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "      <td>263.0000</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>29</td>\n",
       "      <td>1</td>\n",
       "      <td>22.380113</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7.8792</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>30</td>\n",
       "      <td>0</td>\n",
       "      <td>27.947206</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7.8958</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>861</th>\n",
       "      <td>862</td>\n",
       "      <td>0</td>\n",
       "      <td>21.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>11.5000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>862</th>\n",
       "      <td>863</td>\n",
       "      <td>1</td>\n",
       "      <td>48.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>25.9292</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>863</th>\n",
       "      <td>864</td>\n",
       "      <td>0</td>\n",
       "      <td>10.888325</td>\n",
       "      <td>8</td>\n",
       "      <td>2</td>\n",
       "      <td>69.5500</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>864</th>\n",
       "      <td>865</td>\n",
       "      <td>0</td>\n",
       "      <td>24.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>13.0000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>865</th>\n",
       "      <td>866</td>\n",
       "      <td>1</td>\n",
       "      <td>42.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>13.0000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>866</th>\n",
       "      <td>867</td>\n",
       "      <td>1</td>\n",
       "      <td>27.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>13.8583</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>867</th>\n",
       "      <td>868</td>\n",
       "      <td>0</td>\n",
       "      <td>31.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>50.4958</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>868</th>\n",
       "      <td>869</td>\n",
       "      <td>0</td>\n",
       "      <td>25.977889</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>9.5000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>869</th>\n",
       "      <td>870</td>\n",
       "      <td>1</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>11.1333</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>870</th>\n",
       "      <td>871</td>\n",
       "      <td>0</td>\n",
       "      <td>26.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7.8958</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>871</th>\n",
       "      <td>872</td>\n",
       "      <td>1</td>\n",
       "      <td>47.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>52.5542</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>872</th>\n",
       "      <td>873</td>\n",
       "      <td>0</td>\n",
       "      <td>33.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>5.0000</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>873</th>\n",
       "      <td>874</td>\n",
       "      <td>0</td>\n",
       "      <td>47.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>9.0000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>874</th>\n",
       "      <td>875</td>\n",
       "      <td>1</td>\n",
       "      <td>28.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>24.0000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>875</th>\n",
       "      <td>876</td>\n",
       "      <td>1</td>\n",
       "      <td>15.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7.2250</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>876</th>\n",
       "      <td>877</td>\n",
       "      <td>0</td>\n",
       "      <td>20.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>9.8458</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>877</th>\n",
       "      <td>878</td>\n",
       "      <td>0</td>\n",
       "      <td>19.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7.8958</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>878</th>\n",
       "      <td>879</td>\n",
       "      <td>0</td>\n",
       "      <td>27.947206</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7.8958</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>879</th>\n",
       "      <td>880</td>\n",
       "      <td>1</td>\n",
       "      <td>56.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>83.1583</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>880</th>\n",
       "      <td>881</td>\n",
       "      <td>1</td>\n",
       "      <td>25.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>26.0000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>881</th>\n",
       "      <td>882</td>\n",
       "      <td>0</td>\n",
       "      <td>33.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7.8958</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>882</th>\n",
       "      <td>883</td>\n",
       "      <td>0</td>\n",
       "      <td>22.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>10.5167</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>883</th>\n",
       "      <td>884</td>\n",
       "      <td>0</td>\n",
       "      <td>28.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>10.5000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>884</th>\n",
       "      <td>885</td>\n",
       "      <td>0</td>\n",
       "      <td>25.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7.0500</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>885</th>\n",
       "      <td>886</td>\n",
       "      <td>0</td>\n",
       "      <td>39.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>29.1250</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>886</th>\n",
       "      <td>887</td>\n",
       "      <td>0</td>\n",
       "      <td>27.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>13.0000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>887</th>\n",
       "      <td>888</td>\n",
       "      <td>1</td>\n",
       "      <td>19.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>30.0000</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>888</th>\n",
       "      <td>889</td>\n",
       "      <td>0</td>\n",
       "      <td>16.232379</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>23.4500</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>889</th>\n",
       "      <td>890</td>\n",
       "      <td>1</td>\n",
       "      <td>26.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>30.0000</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>890</th>\n",
       "      <td>891</td>\n",
       "      <td>0</td>\n",
       "      <td>32.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7.7500</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>891 rows × 16 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     PassengerId  Survived        Age  SibSp  Parch      Fare  Cabin_No  \\\n",
       "0              1         0  22.000000      1      0    7.2500         1   \n",
       "1              2         1  38.000000      1      0   71.2833         0   \n",
       "2              3         1  26.000000      0      0    7.9250         1   \n",
       "3              4         1  35.000000      1      0   53.1000         0   \n",
       "4              5         0  35.000000      0      0    8.0500         1   \n",
       "5              6         0  23.828953      0      0    8.4583         1   \n",
       "6              7         0  54.000000      0      0   51.8625         0   \n",
       "7              8         0   2.000000      3      1   21.0750         1   \n",
       "8              9         1  27.000000      0      2   11.1333         1   \n",
       "9             10         1  14.000000      1      0   30.0708         1   \n",
       "10            11         1   4.000000      1      1   16.7000         0   \n",
       "11            12         1  58.000000      0      0   26.5500         0   \n",
       "12            13         0  20.000000      0      0    8.0500         1   \n",
       "13            14         0  39.000000      1      5   31.2750         1   \n",
       "14            15         0  14.000000      0      0    7.8542         1   \n",
       "15            16         1  55.000000      0      0   16.0000         1   \n",
       "16            17         0   2.000000      4      1   29.1250         1   \n",
       "17            18         1  32.066493      0      0   13.0000         1   \n",
       "18            19         0  31.000000      1      0   18.0000         1   \n",
       "19            20         1  29.518205      0      0    7.2250         1   \n",
       "20            21         0  35.000000      0      0   26.0000         1   \n",
       "21            22         1  34.000000      0      0   13.0000         0   \n",
       "22            23         1  15.000000      0      0    8.0292         1   \n",
       "23            24         1  28.000000      0      0   35.5000         0   \n",
       "24            25         0   8.000000      3      1   21.0750         1   \n",
       "25            26         1  38.000000      1      5   31.3875         1   \n",
       "26            27         0  29.518205      0      0    7.2250         1   \n",
       "27            28         0  19.000000      3      2  263.0000         0   \n",
       "28            29         1  22.380113      0      0    7.8792         1   \n",
       "29            30         0  27.947206      0      0    7.8958         1   \n",
       "..           ...       ...        ...    ...    ...       ...       ...   \n",
       "861          862         0  21.000000      1      0   11.5000         1   \n",
       "862          863         1  48.000000      0      0   25.9292         0   \n",
       "863          864         0  10.888325      8      2   69.5500         1   \n",
       "864          865         0  24.000000      0      0   13.0000         1   \n",
       "865          866         1  42.000000      0      0   13.0000         1   \n",
       "866          867         1  27.000000      1      0   13.8583         1   \n",
       "867          868         0  31.000000      0      0   50.4958         0   \n",
       "868          869         0  25.977889      0      0    9.5000         1   \n",
       "869          870         1   4.000000      1      1   11.1333         1   \n",
       "870          871         0  26.000000      0      0    7.8958         1   \n",
       "871          872         1  47.000000      1      1   52.5542         0   \n",
       "872          873         0  33.000000      0      0    5.0000         0   \n",
       "873          874         0  47.000000      0      0    9.0000         1   \n",
       "874          875         1  28.000000      1      0   24.0000         1   \n",
       "875          876         1  15.000000      0      0    7.2250         1   \n",
       "876          877         0  20.000000      0      0    9.8458         1   \n",
       "877          878         0  19.000000      0      0    7.8958         1   \n",
       "878          879         0  27.947206      0      0    7.8958         1   \n",
       "879          880         1  56.000000      0      1   83.1583         0   \n",
       "880          881         1  25.000000      0      1   26.0000         1   \n",
       "881          882         0  33.000000      0      0    7.8958         1   \n",
       "882          883         0  22.000000      0      0   10.5167         1   \n",
       "883          884         0  28.000000      0      0   10.5000         1   \n",
       "884          885         0  25.000000      0      0    7.0500         1   \n",
       "885          886         0  39.000000      0      5   29.1250         1   \n",
       "886          887         0  27.000000      0      0   13.0000         1   \n",
       "887          888         1  19.000000      0      0   30.0000         0   \n",
       "888          889         0  16.232379      1      2   23.4500         1   \n",
       "889          890         1  26.000000      0      0   30.0000         0   \n",
       "890          891         0  32.000000      0      0    7.7500         1   \n",
       "\n",
       "     Cabin_Yes  Embarked_C  Embarked_Q  Embarked_S  Sex_female  Sex_male  \\\n",
       "0            0           0           0           1           0         1   \n",
       "1            1           1           0           0           1         0   \n",
       "2            0           0           0           1           1         0   \n",
       "3            1           0           0           1           1         0   \n",
       "4            0           0           0           1           0         1   \n",
       "5            0           0           1           0           0         1   \n",
       "6            1           0           0           1           0         1   \n",
       "7            0           0           0           1           0         1   \n",
       "8            0           0           0           1           1         0   \n",
       "9            0           1           0           0           1         0   \n",
       "10           1           0           0           1           1         0   \n",
       "11           1           0           0           1           1         0   \n",
       "12           0           0           0           1           0         1   \n",
       "13           0           0           0           1           0         1   \n",
       "14           0           0           0           1           1         0   \n",
       "15           0           0           0           1           1         0   \n",
       "16           0           0           1           0           0         1   \n",
       "17           0           0           0           1           0         1   \n",
       "18           0           0           0           1           1         0   \n",
       "19           0           1           0           0           1         0   \n",
       "20           0           0           0           1           0         1   \n",
       "21           1           0           0           1           0         1   \n",
       "22           0           0           1           0           1         0   \n",
       "23           1           0           0           1           0         1   \n",
       "24           0           0           0           1           1         0   \n",
       "25           0           0           0           1           1         0   \n",
       "26           0           1           0           0           0         1   \n",
       "27           1           0           0           1           0         1   \n",
       "28           0           0           1           0           1         0   \n",
       "29           0           0           0           1           0         1   \n",
       "..         ...         ...         ...         ...         ...       ...   \n",
       "861          0           0           0           1           0         1   \n",
       "862          1           0           0           1           1         0   \n",
       "863          0           0           0           1           1         0   \n",
       "864          0           0           0           1           0         1   \n",
       "865          0           0           0           1           1         0   \n",
       "866          0           1           0           0           1         0   \n",
       "867          1           0           0           1           0         1   \n",
       "868          0           0           0           1           0         1   \n",
       "869          0           0           0           1           0         1   \n",
       "870          0           0           0           1           0         1   \n",
       "871          1           0           0           1           1         0   \n",
       "872          1           0           0           1           0         1   \n",
       "873          0           0           0           1           0         1   \n",
       "874          0           1           0           0           1         0   \n",
       "875          0           1           0           0           1         0   \n",
       "876          0           0           0           1           0         1   \n",
       "877          0           0           0           1           0         1   \n",
       "878          0           0           0           1           0         1   \n",
       "879          1           1           0           0           1         0   \n",
       "880          0           0           0           1           1         0   \n",
       "881          0           0           0           1           0         1   \n",
       "882          0           0           0           1           1         0   \n",
       "883          0           0           0           1           0         1   \n",
       "884          0           0           0           1           0         1   \n",
       "885          0           0           1           0           1         0   \n",
       "886          0           0           0           1           0         1   \n",
       "887          1           0           0           1           1         0   \n",
       "888          0           0           0           1           1         0   \n",
       "889          1           1           0           0           0         1   \n",
       "890          0           0           1           0           0         1   \n",
       "\n",
       "     Pclass_1  Pclass_2  Pclass_3  \n",
       "0           0         0         1  \n",
       "1           1         0         0  \n",
       "2           0         0         1  \n",
       "3           1         0         0  \n",
       "4           0         0         1  \n",
       "5           0         0         1  \n",
       "6           1         0         0  \n",
       "7           0         0         1  \n",
       "8           0         0         1  \n",
       "9           0         1         0  \n",
       "10          0         0         1  \n",
       "11          1         0         0  \n",
       "12          0         0         1  \n",
       "13          0         0         1  \n",
       "14          0         0         1  \n",
       "15          0         1         0  \n",
       "16          0         0         1  \n",
       "17          0         1         0  \n",
       "18          0         0         1  \n",
       "19          0         0         1  \n",
       "20          0         1         0  \n",
       "21          0         1         0  \n",
       "22          0         0         1  \n",
       "23          1         0         0  \n",
       "24          0         0         1  \n",
       "25          0         0         1  \n",
       "26          0         0         1  \n",
       "27          1         0         0  \n",
       "28          0         0         1  \n",
       "29          0         0         1  \n",
       "..        ...       ...       ...  \n",
       "861         0         1         0  \n",
       "862         1         0         0  \n",
       "863         0         0         1  \n",
       "864         0         1         0  \n",
       "865         0         1         0  \n",
       "866         0         1         0  \n",
       "867         1         0         0  \n",
       "868         0         0         1  \n",
       "869         0         0         1  \n",
       "870         0         0         1  \n",
       "871         1         0         0  \n",
       "872         1         0         0  \n",
       "873         0         0         1  \n",
       "874         0         1         0  \n",
       "875         0         0         1  \n",
       "876         0         0         1  \n",
       "877         0         0         1  \n",
       "878         0         0         1  \n",
       "879         1         0         0  \n",
       "880         0         1         0  \n",
       "881         0         0         1  \n",
       "882         0         0         1  \n",
       "883         0         1         0  \n",
       "884         0         0         1  \n",
       "885         0         0         1  \n",
       "886         0         1         0  \n",
       "887         1         0         0  \n",
       "888         0         0         1  \n",
       "889         1         0         0  \n",
       "890         0         0         1  \n",
       "\n",
       "[891 rows x 16 columns]"
      ]
     },
     "execution_count": 565,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 因为逻辑回归建模时，需要输入的特征都是数值型特征\n",
    "# 我们先对类目型的特征离散/因子化\n",
    "# 以Cabin为例，原本一个属性维度，因为其取值可以是['yes','no']，而将其平展开为'Cabin_yes','Cabin_no'两个属性\n",
    "# 原本Cabin取值为yes的，在此处的'Cabin_yes'下取值为1，在'Cabin_no'下取值为0\n",
    "# 原本Cabin取值为no的，在此处的'Cabin_yes'下取值为0，在'Cabin_no'下取值为1\n",
    "# 我们使用pandas的get_dummies来完成这个工作，并拼接在原来的data_train之上，如下所示\n",
    "dummies_Cabin = pd.get_dummies(data_train['Cabin'], prefix= 'Cabin')\n",
    "\n",
    "dummies_Embarked = pd.get_dummies(data_train['Embarked'], prefix= 'Embarked')\n",
    "\n",
    "dummies_Sex = pd.get_dummies(data_train['Sex'], prefix= 'Sex')\n",
    "\n",
    "dummies_Pclass = pd.get_dummies(data_train['Pclass'], prefix= 'Pclass')\n",
    "\n",
    "df = pd.concat([data_train, dummies_Cabin, dummies_Embarked, dummies_Sex, dummies_Pclass], axis=1)\n",
    "df.drop(['Pclass', 'Name', 'Sex', 'Ticket', 'Cabin', 'Embarked'], axis=1, inplace=True)\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<font color=red>我们还得做一些处理，仔细看看Age和Fare两个属性，乘客的数值幅度变化，也忒大了吧！！如果大家了解逻辑回归与梯度下降的话，会知道，各属性值之间scale差距太大，将对收敛速度造成几万点伤害值！甚至不收敛！ (╬▔皿▔)…所以我们先用scikit-learn里面的preprocessing模块对这俩货做一个scaling，所谓scaling，其实就是将一些变化幅度较大的特征化到[-1,1]之内。<font>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 566,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div style=\"max-height:1000px;max-width:1500px;overflow:auto;\">\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Survived</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Cabin_No</th>\n",
       "      <th>Cabin_Yes</th>\n",
       "      <th>Embarked_C</th>\n",
       "      <th>Embarked_Q</th>\n",
       "      <th>Embarked_S</th>\n",
       "      <th>Sex_female</th>\n",
       "      <th>Sex_male</th>\n",
       "      <th>Pclass_1</th>\n",
       "      <th>Pclass_2</th>\n",
       "      <th>Pclass_3</th>\n",
       "      <th>Age_scaled</th>\n",
       "      <th>Fare_scaled</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>22.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>7.2500</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.561417</td>\n",
       "      <td>-0.502445</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>38.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>71.2833</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.613177</td>\n",
       "      <td>0.786845</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>26.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7.9250</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.267768</td>\n",
       "      <td>-0.488854</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>35.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>53.1000</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.392941</td>\n",
       "      <td>0.420730</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>35.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0.392941</td>\n",
       "      <td>-0.486337</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>6</td>\n",
       "      <td>0</td>\n",
       "      <td>23.828953</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>8.4583</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.427149</td>\n",
       "      <td>-0.478116</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>7</td>\n",
       "      <td>0</td>\n",
       "      <td>54.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>51.8625</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1.787771</td>\n",
       "      <td>0.395814</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>8</td>\n",
       "      <td>0</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>21.0750</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-2.029659</td>\n",
       "      <td>-0.224083</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>9</td>\n",
       "      <td>1</td>\n",
       "      <td>27.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>11.1333</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.194356</td>\n",
       "      <td>-0.424256</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>10</td>\n",
       "      <td>1</td>\n",
       "      <td>14.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>30.0708</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>-1.148714</td>\n",
       "      <td>-0.042956</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>11</td>\n",
       "      <td>1</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>16.7000</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-1.882835</td>\n",
       "      <td>-0.312172</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>12</td>\n",
       "      <td>1</td>\n",
       "      <td>58.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>26.5500</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2.081420</td>\n",
       "      <td>-0.113846</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>13</td>\n",
       "      <td>0</td>\n",
       "      <td>20.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.708241</td>\n",
       "      <td>-0.486337</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>14</td>\n",
       "      <td>0</td>\n",
       "      <td>39.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>31.2750</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0.686589</td>\n",
       "      <td>-0.018709</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>15</td>\n",
       "      <td>0</td>\n",
       "      <td>14.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7.8542</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-1.148714</td>\n",
       "      <td>-0.490280</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>16</td>\n",
       "      <td>1</td>\n",
       "      <td>55.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>16.0000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1.861183</td>\n",
       "      <td>-0.326267</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>17</td>\n",
       "      <td>0</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>29.1250</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-2.029659</td>\n",
       "      <td>-0.061999</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>18</td>\n",
       "      <td>1</td>\n",
       "      <td>32.066493</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>13.0000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0.177586</td>\n",
       "      <td>-0.386671</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>19</td>\n",
       "      <td>0</td>\n",
       "      <td>31.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>18.0000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0.099292</td>\n",
       "      <td>-0.285997</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>20</td>\n",
       "      <td>1</td>\n",
       "      <td>29.518205</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7.2250</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.009489</td>\n",
       "      <td>-0.502949</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>21</td>\n",
       "      <td>0</td>\n",
       "      <td>35.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>26.0000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0.392941</td>\n",
       "      <td>-0.124920</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>22</td>\n",
       "      <td>1</td>\n",
       "      <td>34.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>13.0000</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0.319529</td>\n",
       "      <td>-0.386671</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>23</td>\n",
       "      <td>1</td>\n",
       "      <td>15.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>8.0292</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-1.075302</td>\n",
       "      <td>-0.486756</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>24</td>\n",
       "      <td>1</td>\n",
       "      <td>28.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>35.5000</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>-0.120944</td>\n",
       "      <td>0.066360</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>25</td>\n",
       "      <td>0</td>\n",
       "      <td>8.000000</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>21.0750</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-1.589186</td>\n",
       "      <td>-0.224083</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>26</td>\n",
       "      <td>1</td>\n",
       "      <td>38.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>31.3875</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0.613177</td>\n",
       "      <td>-0.016444</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>27</td>\n",
       "      <td>0</td>\n",
       "      <td>29.518205</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7.2250</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.009489</td>\n",
       "      <td>-0.502949</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>28</td>\n",
       "      <td>0</td>\n",
       "      <td>19.000000</td>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "      <td>263.0000</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>-0.781653</td>\n",
       "      <td>4.647001</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>29</td>\n",
       "      <td>1</td>\n",
       "      <td>22.380113</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7.8792</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.533512</td>\n",
       "      <td>-0.489776</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>30</td>\n",
       "      <td>0</td>\n",
       "      <td>27.947206</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7.8958</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.124820</td>\n",
       "      <td>-0.489442</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>861</th>\n",
       "      <td>862</td>\n",
       "      <td>0</td>\n",
       "      <td>21.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>11.5000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>-0.634829</td>\n",
       "      <td>-0.416873</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>862</th>\n",
       "      <td>863</td>\n",
       "      <td>1</td>\n",
       "      <td>48.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>25.9292</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1.347299</td>\n",
       "      <td>-0.126345</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>863</th>\n",
       "      <td>864</td>\n",
       "      <td>0</td>\n",
       "      <td>10.888325</td>\n",
       "      <td>8</td>\n",
       "      <td>2</td>\n",
       "      <td>69.5500</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-1.377148</td>\n",
       "      <td>0.751946</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>864</th>\n",
       "      <td>865</td>\n",
       "      <td>0</td>\n",
       "      <td>24.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>13.0000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>-0.414592</td>\n",
       "      <td>-0.386671</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>865</th>\n",
       "      <td>866</td>\n",
       "      <td>1</td>\n",
       "      <td>42.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>13.0000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0.906826</td>\n",
       "      <td>-0.386671</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>866</th>\n",
       "      <td>867</td>\n",
       "      <td>1</td>\n",
       "      <td>27.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>13.8583</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>-0.194356</td>\n",
       "      <td>-0.369389</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>867</th>\n",
       "      <td>868</td>\n",
       "      <td>0</td>\n",
       "      <td>31.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>50.4958</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.099292</td>\n",
       "      <td>0.368295</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>868</th>\n",
       "      <td>869</td>\n",
       "      <td>0</td>\n",
       "      <td>25.977889</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>9.5000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.269391</td>\n",
       "      <td>-0.457142</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>869</th>\n",
       "      <td>870</td>\n",
       "      <td>1</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>11.1333</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-1.882835</td>\n",
       "      <td>-0.424256</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>870</th>\n",
       "      <td>871</td>\n",
       "      <td>0</td>\n",
       "      <td>26.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7.8958</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.267768</td>\n",
       "      <td>-0.489442</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>871</th>\n",
       "      <td>872</td>\n",
       "      <td>1</td>\n",
       "      <td>47.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>52.5542</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1.273886</td>\n",
       "      <td>0.409741</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>872</th>\n",
       "      <td>873</td>\n",
       "      <td>0</td>\n",
       "      <td>33.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>5.0000</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.246117</td>\n",
       "      <td>-0.547748</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>873</th>\n",
       "      <td>874</td>\n",
       "      <td>0</td>\n",
       "      <td>47.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>9.0000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1.273886</td>\n",
       "      <td>-0.467209</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>874</th>\n",
       "      <td>875</td>\n",
       "      <td>1</td>\n",
       "      <td>28.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>24.0000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>-0.120944</td>\n",
       "      <td>-0.165189</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>875</th>\n",
       "      <td>876</td>\n",
       "      <td>1</td>\n",
       "      <td>15.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7.2250</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-1.075302</td>\n",
       "      <td>-0.502949</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>876</th>\n",
       "      <td>877</td>\n",
       "      <td>0</td>\n",
       "      <td>20.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>9.8458</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.708241</td>\n",
       "      <td>-0.450180</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>877</th>\n",
       "      <td>878</td>\n",
       "      <td>0</td>\n",
       "      <td>19.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7.8958</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.781653</td>\n",
       "      <td>-0.489442</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>878</th>\n",
       "      <td>879</td>\n",
       "      <td>0</td>\n",
       "      <td>27.947206</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7.8958</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.124820</td>\n",
       "      <td>-0.489442</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>879</th>\n",
       "      <td>880</td>\n",
       "      <td>1</td>\n",
       "      <td>56.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>83.1583</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1.934596</td>\n",
       "      <td>1.025945</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>880</th>\n",
       "      <td>881</td>\n",
       "      <td>1</td>\n",
       "      <td>25.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>26.0000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>-0.341180</td>\n",
       "      <td>-0.124920</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>881</th>\n",
       "      <td>882</td>\n",
       "      <td>0</td>\n",
       "      <td>33.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7.8958</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0.246117</td>\n",
       "      <td>-0.489442</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>882</th>\n",
       "      <td>883</td>\n",
       "      <td>0</td>\n",
       "      <td>22.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>10.5167</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.561417</td>\n",
       "      <td>-0.436671</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>883</th>\n",
       "      <td>884</td>\n",
       "      <td>0</td>\n",
       "      <td>28.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>10.5000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>-0.120944</td>\n",
       "      <td>-0.437007</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>884</th>\n",
       "      <td>885</td>\n",
       "      <td>0</td>\n",
       "      <td>25.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7.0500</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.341180</td>\n",
       "      <td>-0.506472</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>885</th>\n",
       "      <td>886</td>\n",
       "      <td>0</td>\n",
       "      <td>39.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>29.1250</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0.686589</td>\n",
       "      <td>-0.061999</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>886</th>\n",
       "      <td>887</td>\n",
       "      <td>0</td>\n",
       "      <td>27.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>13.0000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>-0.194356</td>\n",
       "      <td>-0.386671</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>887</th>\n",
       "      <td>888</td>\n",
       "      <td>1</td>\n",
       "      <td>19.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>30.0000</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>-0.781653</td>\n",
       "      <td>-0.044381</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>888</th>\n",
       "      <td>889</td>\n",
       "      <td>0</td>\n",
       "      <td>16.232379</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>23.4500</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.984830</td>\n",
       "      <td>-0.176263</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>889</th>\n",
       "      <td>890</td>\n",
       "      <td>1</td>\n",
       "      <td>26.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>30.0000</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>-0.267768</td>\n",
       "      <td>-0.044381</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>890</th>\n",
       "      <td>891</td>\n",
       "      <td>0</td>\n",
       "      <td>32.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7.7500</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0.172705</td>\n",
       "      <td>-0.492378</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>891 rows × 18 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     PassengerId  Survived        Age  SibSp  Parch      Fare  Cabin_No  \\\n",
       "0              1         0  22.000000      1      0    7.2500         1   \n",
       "1              2         1  38.000000      1      0   71.2833         0   \n",
       "2              3         1  26.000000      0      0    7.9250         1   \n",
       "3              4         1  35.000000      1      0   53.1000         0   \n",
       "4              5         0  35.000000      0      0    8.0500         1   \n",
       "5              6         0  23.828953      0      0    8.4583         1   \n",
       "6              7         0  54.000000      0      0   51.8625         0   \n",
       "7              8         0   2.000000      3      1   21.0750         1   \n",
       "8              9         1  27.000000      0      2   11.1333         1   \n",
       "9             10         1  14.000000      1      0   30.0708         1   \n",
       "10            11         1   4.000000      1      1   16.7000         0   \n",
       "11            12         1  58.000000      0      0   26.5500         0   \n",
       "12            13         0  20.000000      0      0    8.0500         1   \n",
       "13            14         0  39.000000      1      5   31.2750         1   \n",
       "14            15         0  14.000000      0      0    7.8542         1   \n",
       "15            16         1  55.000000      0      0   16.0000         1   \n",
       "16            17         0   2.000000      4      1   29.1250         1   \n",
       "17            18         1  32.066493      0      0   13.0000         1   \n",
       "18            19         0  31.000000      1      0   18.0000         1   \n",
       "19            20         1  29.518205      0      0    7.2250         1   \n",
       "20            21         0  35.000000      0      0   26.0000         1   \n",
       "21            22         1  34.000000      0      0   13.0000         0   \n",
       "22            23         1  15.000000      0      0    8.0292         1   \n",
       "23            24         1  28.000000      0      0   35.5000         0   \n",
       "24            25         0   8.000000      3      1   21.0750         1   \n",
       "25            26         1  38.000000      1      5   31.3875         1   \n",
       "26            27         0  29.518205      0      0    7.2250         1   \n",
       "27            28         0  19.000000      3      2  263.0000         0   \n",
       "28            29         1  22.380113      0      0    7.8792         1   \n",
       "29            30         0  27.947206      0      0    7.8958         1   \n",
       "..           ...       ...        ...    ...    ...       ...       ...   \n",
       "861          862         0  21.000000      1      0   11.5000         1   \n",
       "862          863         1  48.000000      0      0   25.9292         0   \n",
       "863          864         0  10.888325      8      2   69.5500         1   \n",
       "864          865         0  24.000000      0      0   13.0000         1   \n",
       "865          866         1  42.000000      0      0   13.0000         1   \n",
       "866          867         1  27.000000      1      0   13.8583         1   \n",
       "867          868         0  31.000000      0      0   50.4958         0   \n",
       "868          869         0  25.977889      0      0    9.5000         1   \n",
       "869          870         1   4.000000      1      1   11.1333         1   \n",
       "870          871         0  26.000000      0      0    7.8958         1   \n",
       "871          872         1  47.000000      1      1   52.5542         0   \n",
       "872          873         0  33.000000      0      0    5.0000         0   \n",
       "873          874         0  47.000000      0      0    9.0000         1   \n",
       "874          875         1  28.000000      1      0   24.0000         1   \n",
       "875          876         1  15.000000      0      0    7.2250         1   \n",
       "876          877         0  20.000000      0      0    9.8458         1   \n",
       "877          878         0  19.000000      0      0    7.8958         1   \n",
       "878          879         0  27.947206      0      0    7.8958         1   \n",
       "879          880         1  56.000000      0      1   83.1583         0   \n",
       "880          881         1  25.000000      0      1   26.0000         1   \n",
       "881          882         0  33.000000      0      0    7.8958         1   \n",
       "882          883         0  22.000000      0      0   10.5167         1   \n",
       "883          884         0  28.000000      0      0   10.5000         1   \n",
       "884          885         0  25.000000      0      0    7.0500         1   \n",
       "885          886         0  39.000000      0      5   29.1250         1   \n",
       "886          887         0  27.000000      0      0   13.0000         1   \n",
       "887          888         1  19.000000      0      0   30.0000         0   \n",
       "888          889         0  16.232379      1      2   23.4500         1   \n",
       "889          890         1  26.000000      0      0   30.0000         0   \n",
       "890          891         0  32.000000      0      0    7.7500         1   \n",
       "\n",
       "     Cabin_Yes  Embarked_C  Embarked_Q  Embarked_S  Sex_female  Sex_male  \\\n",
       "0            0           0           0           1           0         1   \n",
       "1            1           1           0           0           1         0   \n",
       "2            0           0           0           1           1         0   \n",
       "3            1           0           0           1           1         0   \n",
       "4            0           0           0           1           0         1   \n",
       "5            0           0           1           0           0         1   \n",
       "6            1           0           0           1           0         1   \n",
       "7            0           0           0           1           0         1   \n",
       "8            0           0           0           1           1         0   \n",
       "9            0           1           0           0           1         0   \n",
       "10           1           0           0           1           1         0   \n",
       "11           1           0           0           1           1         0   \n",
       "12           0           0           0           1           0         1   \n",
       "13           0           0           0           1           0         1   \n",
       "14           0           0           0           1           1         0   \n",
       "15           0           0           0           1           1         0   \n",
       "16           0           0           1           0           0         1   \n",
       "17           0           0           0           1           0         1   \n",
       "18           0           0           0           1           1         0   \n",
       "19           0           1           0           0           1         0   \n",
       "20           0           0           0           1           0         1   \n",
       "21           1           0           0           1           0         1   \n",
       "22           0           0           1           0           1         0   \n",
       "23           1           0           0           1           0         1   \n",
       "24           0           0           0           1           1         0   \n",
       "25           0           0           0           1           1         0   \n",
       "26           0           1           0           0           0         1   \n",
       "27           1           0           0           1           0         1   \n",
       "28           0           0           1           0           1         0   \n",
       "29           0           0           0           1           0         1   \n",
       "..         ...         ...         ...         ...         ...       ...   \n",
       "861          0           0           0           1           0         1   \n",
       "862          1           0           0           1           1         0   \n",
       "863          0           0           0           1           1         0   \n",
       "864          0           0           0           1           0         1   \n",
       "865          0           0           0           1           1         0   \n",
       "866          0           1           0           0           1         0   \n",
       "867          1           0           0           1           0         1   \n",
       "868          0           0           0           1           0         1   \n",
       "869          0           0           0           1           0         1   \n",
       "870          0           0           0           1           0         1   \n",
       "871          1           0           0           1           1         0   \n",
       "872          1           0           0           1           0         1   \n",
       "873          0           0           0           1           0         1   \n",
       "874          0           1           0           0           1         0   \n",
       "875          0           1           0           0           1         0   \n",
       "876          0           0           0           1           0         1   \n",
       "877          0           0           0           1           0         1   \n",
       "878          0           0           0           1           0         1   \n",
       "879          1           1           0           0           1         0   \n",
       "880          0           0           0           1           1         0   \n",
       "881          0           0           0           1           0         1   \n",
       "882          0           0           0           1           1         0   \n",
       "883          0           0           0           1           0         1   \n",
       "884          0           0           0           1           0         1   \n",
       "885          0           0           1           0           1         0   \n",
       "886          0           0           0           1           0         1   \n",
       "887          1           0           0           1           1         0   \n",
       "888          0           0           0           1           1         0   \n",
       "889          1           1           0           0           0         1   \n",
       "890          0           0           1           0           0         1   \n",
       "\n",
       "     Pclass_1  Pclass_2  Pclass_3  Age_scaled  Fare_scaled  \n",
       "0           0         0         1   -0.561417    -0.502445  \n",
       "1           1         0         0    0.613177     0.786845  \n",
       "2           0         0         1   -0.267768    -0.488854  \n",
       "3           1         0         0    0.392941     0.420730  \n",
       "4           0         0         1    0.392941    -0.486337  \n",
       "5           0         0         1   -0.427149    -0.478116  \n",
       "6           1         0         0    1.787771     0.395814  \n",
       "7           0         0         1   -2.029659    -0.224083  \n",
       "8           0         0         1   -0.194356    -0.424256  \n",
       "9           0         1         0   -1.148714    -0.042956  \n",
       "10          0         0         1   -1.882835    -0.312172  \n",
       "11          1         0         0    2.081420    -0.113846  \n",
       "12          0         0         1   -0.708241    -0.486337  \n",
       "13          0         0         1    0.686589    -0.018709  \n",
       "14          0         0         1   -1.148714    -0.490280  \n",
       "15          0         1         0    1.861183    -0.326267  \n",
       "16          0         0         1   -2.029659    -0.061999  \n",
       "17          0         1         0    0.177586    -0.386671  \n",
       "18          0         0         1    0.099292    -0.285997  \n",
       "19          0         0         1   -0.009489    -0.502949  \n",
       "20          0         1         0    0.392941    -0.124920  \n",
       "21          0         1         0    0.319529    -0.386671  \n",
       "22          0         0         1   -1.075302    -0.486756  \n",
       "23          1         0         0   -0.120944     0.066360  \n",
       "24          0         0         1   -1.589186    -0.224083  \n",
       "25          0         0         1    0.613177    -0.016444  \n",
       "26          0         0         1   -0.009489    -0.502949  \n",
       "27          1         0         0   -0.781653     4.647001  \n",
       "28          0         0         1   -0.533512    -0.489776  \n",
       "29          0         0         1   -0.124820    -0.489442  \n",
       "..        ...       ...       ...         ...          ...  \n",
       "861         0         1         0   -0.634829    -0.416873  \n",
       "862         1         0         0    1.347299    -0.126345  \n",
       "863         0         0         1   -1.377148     0.751946  \n",
       "864         0         1         0   -0.414592    -0.386671  \n",
       "865         0         1         0    0.906826    -0.386671  \n",
       "866         0         1         0   -0.194356    -0.369389  \n",
       "867         1         0         0    0.099292     0.368295  \n",
       "868         0         0         1   -0.269391    -0.457142  \n",
       "869         0         0         1   -1.882835    -0.424256  \n",
       "870         0         0         1   -0.267768    -0.489442  \n",
       "871         1         0         0    1.273886     0.409741  \n",
       "872         1         0         0    0.246117    -0.547748  \n",
       "873         0         0         1    1.273886    -0.467209  \n",
       "874         0         1         0   -0.120944    -0.165189  \n",
       "875         0         0         1   -1.075302    -0.502949  \n",
       "876         0         0         1   -0.708241    -0.450180  \n",
       "877         0         0         1   -0.781653    -0.489442  \n",
       "878         0         0         1   -0.124820    -0.489442  \n",
       "879         1         0         0    1.934596     1.025945  \n",
       "880         0         1         0   -0.341180    -0.124920  \n",
       "881         0         0         1    0.246117    -0.489442  \n",
       "882         0         0         1   -0.561417    -0.436671  \n",
       "883         0         1         0   -0.120944    -0.437007  \n",
       "884         0         0         1   -0.341180    -0.506472  \n",
       "885         0         0         1    0.686589    -0.061999  \n",
       "886         0         1         0   -0.194356    -0.386671  \n",
       "887         1         0         0   -0.781653    -0.044381  \n",
       "888         0         0         1   -0.984830    -0.176263  \n",
       "889         1         0         0   -0.267768    -0.044381  \n",
       "890         0         0         1    0.172705    -0.492378  \n",
       "\n",
       "[891 rows x 18 columns]"
      ]
     },
     "execution_count": 566,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 接下来我们要接着做一些数据预处理的工作，比如scaling，将一些变化幅度较大的特征化到[-1,1]之内\n",
    "# 这样可以加速logistic regression的收敛\n",
    "import sklearn.preprocessing as preprocessing\n",
    "scaler = preprocessing.StandardScaler()\n",
    "age_scale_param = scaler.fit(df['Age'])\n",
    "df['Age_scaled'] = scaler.fit_transform(df['Age'], age_scale_param)\n",
    "fare_scale_param = scaler.fit(df['Fare'])\n",
    "df['Fare_scaled'] = scaler.fit_transform(df['Fare'], fare_scale_param)\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<font color=red>我们把需要的feature字段取出来，转成numpy格式，使用scikit-learn中的LogisticRegression建模。<font>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 573,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "LogisticRegression(C=1.0, class_weight=None, dual=False, fit_intercept=True,\n",
       "          intercept_scaling=1, max_iter=100, multi_class='ovr',\n",
       "          penalty='l1', random_state=None, solver='liblinear', tol=1e-06,\n",
       "          verbose=0)"
      ]
     },
     "execution_count": 573,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 我们把需要的feature字段取出来，转成numpy格式，使用scikit-learn中的LogisticRegression建模\n",
    "from sklearn import linear_model\n",
    "\n",
    "train_df = df.filter(regex='Survived|Age_.*|SibSp|Parch|Fare_.*|Cabin_.*|Embarked_.*|Sex_.*|Pclass_.*')\n",
    "train_np = train_df.as_matrix()\n",
    "\n",
    "# y即Survival结果\n",
    "y = train_np[:, 0]\n",
    "\n",
    "# X即特征属性值\n",
    "X = train_np[:, 1:]\n",
    "\n",
    "# fit到RandomForestRegressor之中\n",
    "clf = linear_model.LogisticRegression(C=1.0, penalty='l1', tol=1e-6)\n",
    "clf.fit(X, y)\n",
    "    \n",
    "clf"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 574,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(891, 14)"
      ]
     },
     "execution_count": 574,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "接下来咱们对训练集和测试集做一样的操作"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 569,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div style=\"max-height:1000px;max-width:1500px;overflow:auto;\">\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Cabin_No</th>\n",
       "      <th>Cabin_Yes</th>\n",
       "      <th>Embarked_C</th>\n",
       "      <th>Embarked_Q</th>\n",
       "      <th>Embarked_S</th>\n",
       "      <th>Sex_female</th>\n",
       "      <th>Sex_male</th>\n",
       "      <th>Pclass_1</th>\n",
       "      <th>Pclass_2</th>\n",
       "      <th>Pclass_3</th>\n",
       "      <th>Age_scaled</th>\n",
       "      <th>Fare_scaled</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>892</td>\n",
       "      <td>34.500000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7.8292</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0.307495</td>\n",
       "      <td>-0.496637</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>893</td>\n",
       "      <td>47.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>7.0000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1.256225</td>\n",
       "      <td>-0.511497</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>894</td>\n",
       "      <td>62.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>9.6875</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>2.394702</td>\n",
       "      <td>-0.463335</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>895</td>\n",
       "      <td>27.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>8.6625</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.261743</td>\n",
       "      <td>-0.481704</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>896</td>\n",
       "      <td>22.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>12.2875</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.641235</td>\n",
       "      <td>-0.416740</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>897</td>\n",
       "      <td>14.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>9.2250</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-1.248423</td>\n",
       "      <td>-0.471623</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>898</td>\n",
       "      <td>30.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7.6292</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.034048</td>\n",
       "      <td>-0.500221</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>899</td>\n",
       "      <td>26.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>29.0000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>-0.337642</td>\n",
       "      <td>-0.117238</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>900</td>\n",
       "      <td>18.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7.2292</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.944829</td>\n",
       "      <td>-0.507390</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>901</td>\n",
       "      <td>21.000000</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>24.1500</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.717134</td>\n",
       "      <td>-0.204154</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>902</td>\n",
       "      <td>27.947206</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7.8958</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.189852</td>\n",
       "      <td>-0.495444</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>903</td>\n",
       "      <td>46.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>26.0000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1.180327</td>\n",
       "      <td>-0.171000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>904</td>\n",
       "      <td>23.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>82.2667</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>-0.565337</td>\n",
       "      <td>0.837349</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>905</td>\n",
       "      <td>63.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>26.0000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>2.470600</td>\n",
       "      <td>-0.171000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>906</td>\n",
       "      <td>47.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>61.1750</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1.256225</td>\n",
       "      <td>0.459367</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>907</td>\n",
       "      <td>24.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>27.7208</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>-0.489439</td>\n",
       "      <td>-0.140162</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>908</td>\n",
       "      <td>35.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>12.3500</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0.345444</td>\n",
       "      <td>-0.415620</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>909</td>\n",
       "      <td>21.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7.2250</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.717134</td>\n",
       "      <td>-0.507465</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>910</td>\n",
       "      <td>27.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>7.9250</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.261743</td>\n",
       "      <td>-0.494920</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>911</td>\n",
       "      <td>45.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7.2250</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1.104428</td>\n",
       "      <td>-0.507465</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>912</td>\n",
       "      <td>55.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>59.4000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1.863413</td>\n",
       "      <td>0.427557</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>913</td>\n",
       "      <td>9.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>3.1708</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-1.627915</td>\n",
       "      <td>-0.580120</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>914</td>\n",
       "      <td>52.314311</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>31.6833</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1.659573</td>\n",
       "      <td>-0.069151</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>915</td>\n",
       "      <td>21.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>61.3792</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>-0.717134</td>\n",
       "      <td>0.463026</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>916</td>\n",
       "      <td>48.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>262.3750</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1.332124</td>\n",
       "      <td>4.065049</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>917</td>\n",
       "      <td>50.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>14.5000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1.483921</td>\n",
       "      <td>-0.377090</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>918</td>\n",
       "      <td>22.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>61.9792</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>-0.641235</td>\n",
       "      <td>0.473779</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>919</td>\n",
       "      <td>22.500000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7.2250</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.603286</td>\n",
       "      <td>-0.507465</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>920</td>\n",
       "      <td>41.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>30.5000</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.800835</td>\n",
       "      <td>-0.090356</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>921</td>\n",
       "      <td>23.481602</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>21.6792</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.528784</td>\n",
       "      <td>-0.248433</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>388</th>\n",
       "      <td>1280</td>\n",
       "      <td>21.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7.7500</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.717134</td>\n",
       "      <td>-0.498056</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>389</th>\n",
       "      <td>1281</td>\n",
       "      <td>6.000000</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>21.0750</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-1.855610</td>\n",
       "      <td>-0.259261</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>390</th>\n",
       "      <td>1282</td>\n",
       "      <td>23.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>93.5000</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>-0.565337</td>\n",
       "      <td>1.038659</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>391</th>\n",
       "      <td>1283</td>\n",
       "      <td>51.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>39.4000</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1.559819</td>\n",
       "      <td>0.069140</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>392</th>\n",
       "      <td>1284</td>\n",
       "      <td>13.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>20.2500</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-1.324321</td>\n",
       "      <td>-0.274045</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>393</th>\n",
       "      <td>1285</td>\n",
       "      <td>47.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>10.5000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1.256225</td>\n",
       "      <td>-0.448774</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>394</th>\n",
       "      <td>1286</td>\n",
       "      <td>29.000000</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>22.0250</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.109946</td>\n",
       "      <td>-0.242236</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>395</th>\n",
       "      <td>1287</td>\n",
       "      <td>18.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>60.0000</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>-0.944829</td>\n",
       "      <td>0.438310</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>396</th>\n",
       "      <td>1288</td>\n",
       "      <td>24.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7.2500</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.489439</td>\n",
       "      <td>-0.507017</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>397</th>\n",
       "      <td>1289</td>\n",
       "      <td>48.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>79.2000</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1.332124</td>\n",
       "      <td>0.782391</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>398</th>\n",
       "      <td>1290</td>\n",
       "      <td>22.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7.7750</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.641235</td>\n",
       "      <td>-0.497608</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>399</th>\n",
       "      <td>1291</td>\n",
       "      <td>31.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7.7333</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0.041850</td>\n",
       "      <td>-0.498356</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>400</th>\n",
       "      <td>1292</td>\n",
       "      <td>30.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>164.8667</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>-0.034048</td>\n",
       "      <td>2.317614</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>401</th>\n",
       "      <td>1293</td>\n",
       "      <td>38.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>21.0000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0.573139</td>\n",
       "      <td>-0.260605</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>402</th>\n",
       "      <td>1294</td>\n",
       "      <td>22.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>59.4000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>-0.641235</td>\n",
       "      <td>0.427557</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>403</th>\n",
       "      <td>1295</td>\n",
       "      <td>17.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>47.1000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>-1.020728</td>\n",
       "      <td>0.207130</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>404</th>\n",
       "      <td>1296</td>\n",
       "      <td>43.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>27.7208</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.952632</td>\n",
       "      <td>-0.140162</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>405</th>\n",
       "      <td>1297</td>\n",
       "      <td>20.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>13.8625</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>-0.793032</td>\n",
       "      <td>-0.388515</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>406</th>\n",
       "      <td>1298</td>\n",
       "      <td>23.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>10.5000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>-0.565337</td>\n",
       "      <td>-0.448774</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>407</th>\n",
       "      <td>1299</td>\n",
       "      <td>50.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>211.5000</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1.483921</td>\n",
       "      <td>3.153324</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>408</th>\n",
       "      <td>1300</td>\n",
       "      <td>19.895581</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7.7208</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.800957</td>\n",
       "      <td>-0.498580</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>409</th>\n",
       "      <td>1301</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>13.7750</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-2.083305</td>\n",
       "      <td>-0.390083</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>410</th>\n",
       "      <td>1302</td>\n",
       "      <td>35.295824</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7.7500</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0.367897</td>\n",
       "      <td>-0.498056</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>411</th>\n",
       "      <td>1303</td>\n",
       "      <td>37.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>90.0000</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.497241</td>\n",
       "      <td>0.975936</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>412</th>\n",
       "      <td>1304</td>\n",
       "      <td>28.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7.7750</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.185845</td>\n",
       "      <td>-0.497608</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>413</th>\n",
       "      <td>1305</td>\n",
       "      <td>30.705727</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0.019516</td>\n",
       "      <td>-0.492680</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>414</th>\n",
       "      <td>1306</td>\n",
       "      <td>39.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>108.9000</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.649038</td>\n",
       "      <td>1.314641</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>415</th>\n",
       "      <td>1307</td>\n",
       "      <td>38.500000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7.2500</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0.611089</td>\n",
       "      <td>-0.507017</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>416</th>\n",
       "      <td>1308</td>\n",
       "      <td>30.705727</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0.019516</td>\n",
       "      <td>-0.492680</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>417</th>\n",
       "      <td>1309</td>\n",
       "      <td>25.729701</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>22.3583</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.358157</td>\n",
       "      <td>-0.236263</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>418 rows × 17 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     PassengerId        Age  SibSp  Parch      Fare  Cabin_No  Cabin_Yes  \\\n",
       "0            892  34.500000      0      0    7.8292         1          0   \n",
       "1            893  47.000000      1      0    7.0000         1          0   \n",
       "2            894  62.000000      0      0    9.6875         1          0   \n",
       "3            895  27.000000      0      0    8.6625         1          0   \n",
       "4            896  22.000000      1      1   12.2875         1          0   \n",
       "5            897  14.000000      0      0    9.2250         1          0   \n",
       "6            898  30.000000      0      0    7.6292         1          0   \n",
       "7            899  26.000000      1      1   29.0000         1          0   \n",
       "8            900  18.000000      0      0    7.2292         1          0   \n",
       "9            901  21.000000      2      0   24.1500         1          0   \n",
       "10           902  27.947206      0      0    7.8958         1          0   \n",
       "11           903  46.000000      0      0   26.0000         1          0   \n",
       "12           904  23.000000      1      0   82.2667         0          1   \n",
       "13           905  63.000000      1      0   26.0000         1          0   \n",
       "14           906  47.000000      1      0   61.1750         0          1   \n",
       "15           907  24.000000      1      0   27.7208         1          0   \n",
       "16           908  35.000000      0      0   12.3500         1          0   \n",
       "17           909  21.000000      0      0    7.2250         1          0   \n",
       "18           910  27.000000      1      0    7.9250         1          0   \n",
       "19           911  45.000000      0      0    7.2250         1          0   \n",
       "20           912  55.000000      1      0   59.4000         1          0   \n",
       "21           913   9.000000      0      1    3.1708         1          0   \n",
       "22           914  52.314311      0      0   31.6833         1          0   \n",
       "23           915  21.000000      0      1   61.3792         1          0   \n",
       "24           916  48.000000      1      3  262.3750         0          1   \n",
       "25           917  50.000000      1      0   14.5000         1          0   \n",
       "26           918  22.000000      0      1   61.9792         0          1   \n",
       "27           919  22.500000      0      0    7.2250         1          0   \n",
       "28           920  41.000000      0      0   30.5000         0          1   \n",
       "29           921  23.481602      2      0   21.6792         1          0   \n",
       "..           ...        ...    ...    ...       ...       ...        ...   \n",
       "388         1280  21.000000      0      0    7.7500         1          0   \n",
       "389         1281   6.000000      3      1   21.0750         1          0   \n",
       "390         1282  23.000000      0      0   93.5000         0          1   \n",
       "391         1283  51.000000      0      1   39.4000         0          1   \n",
       "392         1284  13.000000      0      2   20.2500         1          0   \n",
       "393         1285  47.000000      0      0   10.5000         1          0   \n",
       "394         1286  29.000000      3      1   22.0250         1          0   \n",
       "395         1287  18.000000      1      0   60.0000         0          1   \n",
       "396         1288  24.000000      0      0    7.2500         1          0   \n",
       "397         1289  48.000000      1      1   79.2000         0          1   \n",
       "398         1290  22.000000      0      0    7.7750         1          0   \n",
       "399         1291  31.000000      0      0    7.7333         1          0   \n",
       "400         1292  30.000000      0      0  164.8667         0          1   \n",
       "401         1293  38.000000      1      0   21.0000         1          0   \n",
       "402         1294  22.000000      0      1   59.4000         1          0   \n",
       "403         1295  17.000000      0      0   47.1000         1          0   \n",
       "404         1296  43.000000      1      0   27.7208         0          1   \n",
       "405         1297  20.000000      0      0   13.8625         0          1   \n",
       "406         1298  23.000000      1      0   10.5000         1          0   \n",
       "407         1299  50.000000      1      1  211.5000         0          1   \n",
       "408         1300  19.895581      0      0    7.7208         1          0   \n",
       "409         1301   3.000000      1      1   13.7750         1          0   \n",
       "410         1302  35.295824      0      0    7.7500         1          0   \n",
       "411         1303  37.000000      1      0   90.0000         0          1   \n",
       "412         1304  28.000000      0      0    7.7750         1          0   \n",
       "413         1305  30.705727      0      0    8.0500         1          0   \n",
       "414         1306  39.000000      0      0  108.9000         0          1   \n",
       "415         1307  38.500000      0      0    7.2500         1          0   \n",
       "416         1308  30.705727      0      0    8.0500         1          0   \n",
       "417         1309  25.729701      1      1   22.3583         1          0   \n",
       "\n",
       "     Embarked_C  Embarked_Q  Embarked_S  Sex_female  Sex_male  Pclass_1  \\\n",
       "0             0           1           0           0         1         0   \n",
       "1             0           0           1           1         0         0   \n",
       "2             0           1           0           0         1         0   \n",
       "3             0           0           1           0         1         0   \n",
       "4             0           0           1           1         0         0   \n",
       "5             0           0           1           0         1         0   \n",
       "6             0           1           0           1         0         0   \n",
       "7             0           0           1           0         1         0   \n",
       "8             1           0           0           1         0         0   \n",
       "9             0           0           1           0         1         0   \n",
       "10            0           0           1           0         1         0   \n",
       "11            0           0           1           0         1         1   \n",
       "12            0           0           1           1         0         1   \n",
       "13            0           0           1           0         1         0   \n",
       "14            0           0           1           1         0         1   \n",
       "15            1           0           0           1         0         0   \n",
       "16            0           1           0           0         1         0   \n",
       "17            1           0           0           0         1         0   \n",
       "18            0           0           1           1         0         0   \n",
       "19            1           0           0           1         0         0   \n",
       "20            1           0           0           0         1         1   \n",
       "21            0           0           1           0         1         0   \n",
       "22            0           0           1           1         0         1   \n",
       "23            1           0           0           0         1         1   \n",
       "24            1           0           0           1         0         1   \n",
       "25            0           0           1           0         1         0   \n",
       "26            1           0           0           1         0         1   \n",
       "27            1           0           0           0         1         0   \n",
       "28            0           0           1           0         1         1   \n",
       "29            1           0           0           0         1         0   \n",
       "..          ...         ...         ...         ...       ...       ...   \n",
       "388           0           1           0           0         1         0   \n",
       "389           0           0           1           0         1         0   \n",
       "390           0           0           1           0         1         1   \n",
       "391           0           0           1           1         0         1   \n",
       "392           0           0           1           0         1         0   \n",
       "393           0           0           1           0         1         0   \n",
       "394           0           0           1           0         1         0   \n",
       "395           0           0           1           1         0         1   \n",
       "396           0           1           0           0         1         0   \n",
       "397           1           0           0           1         0         1   \n",
       "398           0           0           1           0         1         0   \n",
       "399           0           1           0           0         1         0   \n",
       "400           0           0           1           1         0         1   \n",
       "401           0           0           1           0         1         0   \n",
       "402           1           0           0           1         0         1   \n",
       "403           0           0           1           0         1         1   \n",
       "404           1           0           0           0         1         1   \n",
       "405           1           0           0           0         1         0   \n",
       "406           0           0           1           0         1         0   \n",
       "407           1           0           0           0         1         1   \n",
       "408           0           1           0           1         0         0   \n",
       "409           0           0           1           1         0         0   \n",
       "410           0           1           0           1         0         0   \n",
       "411           0           1           0           1         0         1   \n",
       "412           0           0           1           1         0         0   \n",
       "413           0           0           1           0         1         0   \n",
       "414           1           0           0           1         0         1   \n",
       "415           0           0           1           0         1         0   \n",
       "416           0           0           1           0         1         0   \n",
       "417           1           0           0           0         1         0   \n",
       "\n",
       "     Pclass_2  Pclass_3  Age_scaled  Fare_scaled  \n",
       "0           0         1    0.307495    -0.496637  \n",
       "1           0         1    1.256225    -0.511497  \n",
       "2           1         0    2.394702    -0.463335  \n",
       "3           0         1   -0.261743    -0.481704  \n",
       "4           0         1   -0.641235    -0.416740  \n",
       "5           0         1   -1.248423    -0.471623  \n",
       "6           0         1   -0.034048    -0.500221  \n",
       "7           1         0   -0.337642    -0.117238  \n",
       "8           0         1   -0.944829    -0.507390  \n",
       "9           0         1   -0.717134    -0.204154  \n",
       "10          0         1   -0.189852    -0.495444  \n",
       "11          0         0    1.180327    -0.171000  \n",
       "12          0         0   -0.565337     0.837349  \n",
       "13          1         0    2.470600    -0.171000  \n",
       "14          0         0    1.256225     0.459367  \n",
       "15          1         0   -0.489439    -0.140162  \n",
       "16          1         0    0.345444    -0.415620  \n",
       "17          0         1   -0.717134    -0.507465  \n",
       "18          0         1   -0.261743    -0.494920  \n",
       "19          0         1    1.104428    -0.507465  \n",
       "20          0         0    1.863413     0.427557  \n",
       "21          0         1   -1.627915    -0.580120  \n",
       "22          0         0    1.659573    -0.069151  \n",
       "23          0         0   -0.717134     0.463026  \n",
       "24          0         0    1.332124     4.065049  \n",
       "25          0         1    1.483921    -0.377090  \n",
       "26          0         0   -0.641235     0.473779  \n",
       "27          0         1   -0.603286    -0.507465  \n",
       "28          0         0    0.800835    -0.090356  \n",
       "29          0         1   -0.528784    -0.248433  \n",
       "..        ...       ...         ...          ...  \n",
       "388         0         1   -0.717134    -0.498056  \n",
       "389         0         1   -1.855610    -0.259261  \n",
       "390         0         0   -0.565337     1.038659  \n",
       "391         0         0    1.559819     0.069140  \n",
       "392         0         1   -1.324321    -0.274045  \n",
       "393         1         0    1.256225    -0.448774  \n",
       "394         0         1   -0.109946    -0.242236  \n",
       "395         0         0   -0.944829     0.438310  \n",
       "396         0         1   -0.489439    -0.507017  \n",
       "397         0         0    1.332124     0.782391  \n",
       "398         0         1   -0.641235    -0.497608  \n",
       "399         0         1    0.041850    -0.498356  \n",
       "400         0         0   -0.034048     2.317614  \n",
       "401         1         0    0.573139    -0.260605  \n",
       "402         0         0   -0.641235     0.427557  \n",
       "403         0         0   -1.020728     0.207130  \n",
       "404         0         0    0.952632    -0.140162  \n",
       "405         1         0   -0.793032    -0.388515  \n",
       "406         1         0   -0.565337    -0.448774  \n",
       "407         0         0    1.483921     3.153324  \n",
       "408         0         1   -0.800957    -0.498580  \n",
       "409         0         1   -2.083305    -0.390083  \n",
       "410         0         1    0.367897    -0.498056  \n",
       "411         0         0    0.497241     0.975936  \n",
       "412         0         1   -0.185845    -0.497608  \n",
       "413         0         1    0.019516    -0.492680  \n",
       "414         0         0    0.649038     1.314641  \n",
       "415         0         1    0.611089    -0.507017  \n",
       "416         0         1    0.019516    -0.492680  \n",
       "417         0         1   -0.358157    -0.236263  \n",
       "\n",
       "[418 rows x 17 columns]"
      ]
     },
     "execution_count": 569,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data_test = pd.read_csv(\"test.csv\")\n",
    "data_test.loc[ (data_test.Fare.isnull()), 'Fare' ] = 0\n",
    "# 接着我们对test_data做和train_data中一致的特征变换\n",
    "# 首先用同样的RandomForestRegressor模型填上丢失的年龄\n",
    "tmp_df = data_test[['Age','Fare', 'Parch', 'SibSp', 'Pclass']]\n",
    "null_age = tmp_df[data_test.Age.isnull()].as_matrix()\n",
    "# 根据特征属性X预测年龄并补上\n",
    "X = null_age[:, 1:]\n",
    "predictedAges = rfr.predict(X)\n",
    "data_test.loc[ (data_test.Age.isnull()), 'Age' ] = predictedAges\n",
    "\n",
    "data_test = set_Cabin_type(data_test)\n",
    "dummies_Cabin = pd.get_dummies(data_test['Cabin'], prefix= 'Cabin')\n",
    "dummies_Embarked = pd.get_dummies(data_test['Embarked'], prefix= 'Embarked')\n",
    "dummies_Sex = pd.get_dummies(data_test['Sex'], prefix= 'Sex')\n",
    "dummies_Pclass = pd.get_dummies(data_test['Pclass'], prefix= 'Pclass')\n",
    "\n",
    "\n",
    "df_test = pd.concat([data_test, dummies_Cabin, dummies_Embarked, dummies_Sex, dummies_Pclass], axis=1)\n",
    "df_test.drop(['Pclass', 'Name', 'Sex', 'Ticket', 'Cabin', 'Embarked'], axis=1, inplace=True)\n",
    "df_test['Age_scaled'] = scaler.fit_transform(df_test['Age'], age_scale_param)\n",
    "df_test['Fare_scaled'] = scaler.fit_transform(df_test['Fare'], fare_scale_param)\n",
    "df_test"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 371,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "test = df_test.filter(regex='Age_.*|SibSp|Parch|Fare_.*|Cabin_.*|Embarked_.*|Sex_.*|Pclass_.*')\n",
    "predictions = clf.predict(test)\n",
    "result = pd.DataFrame({'PassengerId':data_test['PassengerId'].as_matrix(), 'Survived':predictions.astype(np.int32)})\n",
    "result.to_csv(\"logistic_regression_predictions.csv\", index=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 515,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div style=\"max-height:1000px;max-width:1500px;overflow:auto;\">\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Survived</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>892</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>893</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>894</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>895</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>896</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>897</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>898</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>899</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>900</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>901</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>902</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>903</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>904</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>905</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>906</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>907</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>908</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>909</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>910</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>911</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>912</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>913</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>914</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>915</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>916</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>917</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>918</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>919</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>920</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>921</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>388</th>\n",
       "      <td>1280</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>389</th>\n",
       "      <td>1281</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>390</th>\n",
       "      <td>1282</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>391</th>\n",
       "      <td>1283</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>392</th>\n",
       "      <td>1284</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>393</th>\n",
       "      <td>1285</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>394</th>\n",
       "      <td>1286</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>395</th>\n",
       "      <td>1287</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>396</th>\n",
       "      <td>1288</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>397</th>\n",
       "      <td>1289</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>398</th>\n",
       "      <td>1290</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>399</th>\n",
       "      <td>1291</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>400</th>\n",
       "      <td>1292</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>401</th>\n",
       "      <td>1293</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>402</th>\n",
       "      <td>1294</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>403</th>\n",
       "      <td>1295</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>404</th>\n",
       "      <td>1296</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>405</th>\n",
       "      <td>1297</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>406</th>\n",
       "      <td>1298</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>407</th>\n",
       "      <td>1299</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>408</th>\n",
       "      <td>1300</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>409</th>\n",
       "      <td>1301</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>410</th>\n",
       "      <td>1302</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>411</th>\n",
       "      <td>1303</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>412</th>\n",
       "      <td>1304</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>413</th>\n",
       "      <td>1305</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>414</th>\n",
       "      <td>1306</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>415</th>\n",
       "      <td>1307</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>416</th>\n",
       "      <td>1308</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>417</th>\n",
       "      <td>1309</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>418 rows × 2 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     PassengerId  Survived\n",
       "0            892         0\n",
       "1            893         0\n",
       "2            894         0\n",
       "3            895         0\n",
       "4            896         1\n",
       "5            897         0\n",
       "6            898         1\n",
       "7            899         0\n",
       "8            900         1\n",
       "9            901         0\n",
       "10           902         0\n",
       "11           903         0\n",
       "12           904         1\n",
       "13           905         0\n",
       "14           906         1\n",
       "15           907         1\n",
       "16           908         0\n",
       "17           909         0\n",
       "18           910         1\n",
       "19           911         1\n",
       "20           912         0\n",
       "21           913         0\n",
       "22           914         1\n",
       "23           915         0\n",
       "24           916         1\n",
       "25           917         0\n",
       "26           918         1\n",
       "27           919         0\n",
       "28           920         0\n",
       "29           921         0\n",
       "..           ...       ...\n",
       "388         1280         0\n",
       "389         1281         0\n",
       "390         1282         1\n",
       "391         1283         1\n",
       "392         1284         0\n",
       "393         1285         0\n",
       "394         1286         0\n",
       "395         1287         1\n",
       "396         1288         0\n",
       "397         1289         1\n",
       "398         1290         0\n",
       "399         1291         0\n",
       "400         1292         1\n",
       "401         1293         0\n",
       "402         1294         1\n",
       "403         1295         0\n",
       "404         1296         0\n",
       "405         1297         1\n",
       "406         1298         0\n",
       "407         1299         0\n",
       "408         1300         1\n",
       "409         1301         1\n",
       "410         1302         1\n",
       "411         1303         1\n",
       "412         1304         1\n",
       "413         1305         0\n",
       "414         1306         1\n",
       "415         1307         0\n",
       "416         1308         0\n",
       "417         1309         0\n",
       "\n",
       "[418 rows x 2 columns]"
      ]
     },
     "execution_count": 515,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.read_csv(\"logistic_regression_predictions.csv\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "<font color=red>0.76555，恩，结果还不错。毕竟，这只是我们简单分析过后出的一个baseline系统嘛</font>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 要判定一下当前模型所处状态(欠拟合or过拟合)\n",
    "\n",
    "<font color=red>有一个很可能发生的问题是，我们不断地做feature engineering，产生的特征越来越多，用这些特征去训练模型，会对我们的训练集拟合得越来越好，同时也可能在逐步丧失泛化能力，从而在待预测的数据上，表现不佳，也就是发生过拟合问题。<font><br>\n",
    "\n",
    "<font color=red>从另一个角度上说，如果模型在待预测的数据上表现不佳，除掉上面说的过拟合问题，也有可能是欠拟合问题，也就是说在训练集上，其实拟合的也不是那么好。<font><br>\n",
    "\n",
    "<font color=red>额，这个欠拟合和过拟合怎么解释呢。这么说吧：<font><br>\n",
    "\n",
    "1. <font color=red>过拟合就像是你班那个学数学比较刻板的同学，老师讲过的题目，一字不漏全记下来了，于是老师再出一样的题目，分分钟精确出结果。but数学考试，因为总是碰到新题目，所以成绩不咋地。<font>\n",
    "2. <font color=red>欠拟合就像是，咳咳，和博主level差不多的差生。连老师讲的练习题也记不住，于是连老师出一样题目复习的周测都做不好，考试更是可想而知了。<font>\n",
    "\n",
    "<font color=red>而在机器学习的问题上，对于过拟合和欠拟合两种情形。我们优化的方式是不同的。<font><br>\n",
    "\n",
    "<font color=red>对过拟合而言，通常以下策略对结果优化是有用的：<font><br>\n",
    "\n",
    "* <font color=red>做一下feature selection，挑出较好的feature的subset来做training\n",
    "* <font color=red>提供更多的数据，从而弥补原始数据的bias问题，学习到的model也会更准确\n",
    "\n",
    "<font color=red>而对于欠拟合而言，我们通常需要更多的feature，更复杂的模型来提高准确度。<font><br>\n",
    "\n",
    "<font color=red>著名的learning curve可以帮我们判定我们的模型现在所处的状态。我们以样本数为横坐标，训练和交叉验证集上的错误率作为纵坐标，两种状态分别如下两张图所示：过拟合(overfitting/high variace)，欠拟合(underfitting/high bias)<font><br>\n",
    "\n",
    "![](http://7xo0y8.com1.z0.glb.clouddn.com/2_titanic/high_variance.png?imageView/2/w/400/q/100)\n",
    "![](http://7xo0y8.com1.z0.glb.clouddn.com/2_titanic/high_bias.png?imageView/2/w/400/q/100)\n",
    "\n",
    "<font color=red>著名的learning curve可以帮我们判定我们的模型现在所处的状态。我们以样本数为横坐标，训练和交叉验证集上的错误率作为纵坐标，两种状态分别如下两张图所示：过拟合(overfitting/high variace)，欠拟合(underfitting/high bias)<font><br>\n",
    "\n",
    "<font color=red>我们也可以把错误率替换成准确率(得分)，得到另一种形式的learning curve(sklearn 里面是这么做的)。<font><br>\n",
    "\n",
    "<font color=red>回到我们的问题，我们用scikit-learn里面的learning_curve来帮我们分辨我们模型的状态。举个例子，这里我们一起画一下我们最先得到的baseline model的learning curve。<font><br>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 579,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/sklearn/cross_validation.py:1601: DeprecationWarning: check_cv will return indices instead of boolean masks from 0.17\n",
      "  def _score(estimator, X_test, y_test, scorer):\n",
      "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/sklearn/cross_validation.py:69: DeprecationWarning: The indices parameter is deprecated and will be removed (assumed True) in 0.17\n",
      "  def __iter__(self):\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "(0.80656968448540245, 0.018258876711338634)"
      ]
     },
     "execution_count": 579,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from sklearn.learning_curve import learning_curve\n",
    "\n",
    "# 用sklearn的learning_curve得到training_score和cv_score，使用matplotlib画出learning curve\n",
    "def plot_learning_curve(estimator, title, X, y, ylim=None, cv=None, n_jobs=1, \n",
    "                        train_sizes=np.linspace(.05, 1., 20), verbose=0, plot=True):\n",
    "    \"\"\"\n",
    "    画出data在某模型上的learning curve.\n",
    "    参数解释\n",
    "    ----------\n",
    "    estimator : 你用的分类器。\n",
    "    title : 表格的标题。\n",
    "    X : 输入的feature，numpy类型\n",
    "    y : 输入的target vector\n",
    "    ylim : tuple格式的(ymin, ymax), 设定图像中纵坐标的最低点和最高点\n",
    "    cv : 做cross-validation的时候，数据分成的份数，其中一份作为cv集，其余n-1份作为training(默认为3份)\n",
    "    n_jobs : 并行的的任务数(默认1)\n",
    "    \"\"\"\n",
    "    train_sizes, train_scores, test_scores = learning_curve(\n",
    "        estimator, X, y, cv=cv, n_jobs=n_jobs, train_sizes=train_sizes, verbose=verbose)\n",
    "    \n",
    "    train_scores_mean = np.mean(train_scores, axis=1)\n",
    "    train_scores_std = np.std(train_scores, axis=1)\n",
    "    test_scores_mean = np.mean(test_scores, axis=1)\n",
    "    test_scores_std = np.std(test_scores, axis=1)\n",
    "    \n",
    "    if plot:\n",
    "        plt.figure()\n",
    "        plt.title(title)\n",
    "        if ylim is not None:\n",
    "            plt.ylim(*ylim)\n",
    "        plt.xlabel(u\"训练样本数\")\n",
    "        plt.ylabel(u\"得分\")\n",
    "        plt.gca().invert_yaxis()\n",
    "        plt.grid()\n",
    "    \n",
    "        plt.fill_between(train_sizes, train_scores_mean - train_scores_std, train_scores_mean + train_scores_std, \n",
    "                         alpha=0.1, color=\"b\")\n",
    "        plt.fill_between(train_sizes, test_scores_mean - test_scores_std, test_scores_mean + test_scores_std, \n",
    "                         alpha=0.1, color=\"r\")\n",
    "        plt.plot(train_sizes, train_scores_mean, 'o-', color=\"b\", label=u\"训练集上得分\")\n",
    "        plt.plot(train_sizes, test_scores_mean, 'o-', color=\"r\", label=u\"交叉验证集上得分\")\n",
    "    \n",
    "        plt.legend(loc=\"best\")\n",
    "        \n",
    "        plt.draw()\n",
    "        plt.gca().invert_yaxis()\n",
    "        plt.show()\n",
    "    \n",
    "    midpoint = ((train_scores_mean[-1] + train_scores_std[-1]) + (test_scores_mean[-1] - test_scores_std[-1])) / 2\n",
    "    diff = (train_scores_mean[-1] + train_scores_std[-1]) - (test_scores_mean[-1] - test_scores_std[-1])\n",
    "    return midpoint, diff\n",
    "\n",
    "plot_learning_curve(clf, u\"学习曲线\", X, y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](http://7xo0y8.com1.z0.glb.clouddn.com/2_titanic/learning_curve.png?imageView/2/w/600/q/100)\n",
    "<font color=red>在实际数据上看，我们得到的learning curve没有理论推导的那么光滑哈，但是可以大致看出来，训练集和交叉验证集上的得分曲线走势还是符合预期的。<font><br>\n",
    "\n",
    "<font color=red>目前的曲线看来，我们的model并不处于overfitting的状态(overfitting的表现一般是训练集上得分高，而交叉验证集上要低很多，中间的gap比较大)。因此我们可以再做些feature engineering的工作，添加一些新产出的特征或者组合特征到模型中。<font><br>"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<font color=red>接下来，我们就该看看如何优化baseline系统了<br>\n",
    "我们还有些特征可以再挖掘挖掘<br><br>\n",
    "\n",
    "1. 比如说Name和Ticket两个属性被我们完整舍弃了(好吧，其实是一开始我们对于这种，每一条记录都是一个完全不同的值的属性，并没有很直接的处理方式)<br>\n",
    "2. 比如说，我们想想，年龄的拟合本身也未必是一件非常靠谱的事情<br>\n",
    "3. 另外，以我们的日常经验，小盆友和老人可能得到的照顾会多一些，这样看的话，年龄作为一个连续值，给一个固定的系数，似乎体现不出两头受照顾的实际情况，所以，说不定我们把年龄离散化，按区段分作类别属性会更合适一些<br>\n",
    "\n",
    "那怎么样才知道，哪些地方可以优化，哪些优化的方法是promising的呢？<br>\n",
    "是的<br><br>\n",
    "\n",
    "要做交叉验证(cross validation)!<br>\n",
    "要做交叉验证(cross validation)!<br>\n",
    "要做交叉验证(cross validation)!<br><br>\n",
    "\n",
    "重要的事情说3编！！！<br>\n",
    "因为test.csv里面并没有Survived这个字段(好吧，这是废话，这明明就是我们要预测的结果)，我们无法在这份数据上评定我们算法在该场景下的效果。。。<br>\n",
    "我们通常情况下，这么做cross validation：把train.csv分成两部分，一部分用于训练我们需要的模型，另外一部分数据上看我们预测算法的效果。<br>\n",
    "我们可以用scikit-learn的cross_validation来完成这个工作</font>"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<font color=red>在此之前，咱们可以看看现在得到的模型的系数，因为系数和它们最终的判定能力强弱是正相关的</font>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 493,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div style=\"max-height:1000px;max-width:1500px;overflow:auto;\">\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>coef</th>\n",
       "      <th>columns</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>[-0.344189431858]</td>\n",
       "      <td>SibSp</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>[-0.104924350555]</td>\n",
       "      <td>Parch</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>[0.0]</td>\n",
       "      <td>Cabin_No</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>[0.902071479485]</td>\n",
       "      <td>Cabin_Yes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>[0.0]</td>\n",
       "      <td>Embarked_C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>[0.0]</td>\n",
       "      <td>Embarked_Q</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>[-0.417226462259]</td>\n",
       "      <td>Embarked_S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>[1.95649520339]</td>\n",
       "      <td>Sex_female</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>[-0.677484871046]</td>\n",
       "      <td>Sex_male</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>[0.341226064445]</td>\n",
       "      <td>Pclass_1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>[0.0]</td>\n",
       "      <td>Pclass_2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>[-1.19410912948]</td>\n",
       "      <td>Pclass_3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>[-0.523774279397]</td>\n",
       "      <td>Age_scaled</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>[0.0844279740271]</td>\n",
       "      <td>Fare_scaled</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                 coef      columns\n",
       "0   [-0.344189431858]        SibSp\n",
       "1   [-0.104924350555]        Parch\n",
       "2               [0.0]     Cabin_No\n",
       "3    [0.902071479485]    Cabin_Yes\n",
       "4               [0.0]   Embarked_C\n",
       "5               [0.0]   Embarked_Q\n",
       "6   [-0.417226462259]   Embarked_S\n",
       "7     [1.95649520339]   Sex_female\n",
       "8   [-0.677484871046]     Sex_male\n",
       "9    [0.341226064445]     Pclass_1\n",
       "10              [0.0]     Pclass_2\n",
       "11   [-1.19410912948]     Pclass_3\n",
       "12  [-0.523774279397]   Age_scaled\n",
       "13  [0.0844279740271]  Fare_scaled"
      ]
     },
     "execution_count": 493,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.DataFrame({\"columns\":list(train_df.columns)[1:], \"coef\":list(clf.coef_.T)})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "上面的系数和最后的结果是一个正相关的关系<br>\n",
    "我们先看看那些权重绝对值非常大的feature，在我们的模型上：<br>\n",
    "\n",
    "* Sex属性，如果是female会极大提高最后获救的概率，而male会很大程度拉低这个概率。\n",
    "* Pclass属性，1等舱乘客最后获救的概率会上升，而乘客等级为3会极大地拉低这个概率。\n",
    "* 有Cabin值会很大程度拉升最后获救概率(这里似乎能看到了一点端倪，事实上从最上面的有无Cabin记录的Survived分布图上看出，即使有Cabin记录的乘客也有一部分遇难了，估计这个属性上我们挖掘还不够)\n",
    "* Age是一个负相关，意味着在我们的模型里，年龄越小，越有获救的优先权(还得回原数据看看这个是否合理）\n",
    "* 有一个登船港口S会很大程度拉低获救的概率，另外俩港口压根就没啥作用(这个实际上非常奇怪，因为我们从之前的统计图上并没有看到S港口的获救率非常低，所以也许可以考虑把登船港口这个feature去掉试试)。\n",
    "* 船票Fare有小幅度的正相关(并不意味着这个feature作用不大，有可能是我们细化的程度还不够，举个例子，说不定我们得对它离散化，再分至各个乘客等级上？)\n",
    "\n",
    "噢啦，观察完了，我们现在有一些想法了，但是怎么样才知道，哪些优化的方法是promising的呢？<br>\n",
    "\n",
    "恩，要靠交叉验证"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 560,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ 0.81564246  0.81005587  0.78651685  0.78651685  0.81355932]\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div style=\"max-height:1000px;max-width:1500px;overflow:auto;\">\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Survived</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Cabin_No</th>\n",
       "      <th>Cabin_Yes</th>\n",
       "      <th>Embarked_C</th>\n",
       "      <th>Embarked_Q</th>\n",
       "      <th>Embarked_S</th>\n",
       "      <th>Sex_female</th>\n",
       "      <th>Sex_male</th>\n",
       "      <th>Pclass_1</th>\n",
       "      <th>Pclass_2</th>\n",
       "      <th>Pclass_3</th>\n",
       "      <th>Age_scaled</th>\n",
       "      <th>Fare_scaled</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>298</th>\n",
       "      <td>299</td>\n",
       "      <td>1</td>\n",
       "      <td>41.574877</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>30.5000</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.875617</td>\n",
       "      <td>-0.034314</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>15</td>\n",
       "      <td>0</td>\n",
       "      <td>14.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7.8542</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-1.148714</td>\n",
       "      <td>-0.490280</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>803</th>\n",
       "      <td>804</td>\n",
       "      <td>1</td>\n",
       "      <td>0.420000</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>8.5167</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-2.145650</td>\n",
       "      <td>-0.476941</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>474</th>\n",
       "      <td>475</td>\n",
       "      <td>0</td>\n",
       "      <td>22.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>9.8375</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.561417</td>\n",
       "      <td>-0.450347</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>642</th>\n",
       "      <td>643</td>\n",
       "      <td>0</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "      <td>27.9000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-2.029659</td>\n",
       "      <td>-0.086664</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>55</th>\n",
       "      <td>56</td>\n",
       "      <td>1</td>\n",
       "      <td>44.064830</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>35.5000</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1.058409</td>\n",
       "      <td>0.066360</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>587</th>\n",
       "      <td>588</td>\n",
       "      <td>1</td>\n",
       "      <td>60.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>79.2000</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2.228244</td>\n",
       "      <td>0.946246</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>740</th>\n",
       "      <td>741</td>\n",
       "      <td>1</td>\n",
       "      <td>38.426632</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>30.0000</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.644497</td>\n",
       "      <td>-0.044381</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>839</th>\n",
       "      <td>840</td>\n",
       "      <td>1</td>\n",
       "      <td>50.903700</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>29.7000</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1.560465</td>\n",
       "      <td>-0.050421</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>301</th>\n",
       "      <td>302</td>\n",
       "      <td>1</td>\n",
       "      <td>23.332602</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>23.2500</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.463588</td>\n",
       "      <td>-0.180290</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>567</th>\n",
       "      <td>568</td>\n",
       "      <td>0</td>\n",
       "      <td>29.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>21.0750</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.047532</td>\n",
       "      <td>-0.224083</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>712</th>\n",
       "      <td>713</td>\n",
       "      <td>1</td>\n",
       "      <td>48.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>52.0000</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1.347299</td>\n",
       "      <td>0.398582</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>489</th>\n",
       "      <td>490</td>\n",
       "      <td>1</td>\n",
       "      <td>9.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>15.9000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-1.515774</td>\n",
       "      <td>-0.328280</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>312</th>\n",
       "      <td>313</td>\n",
       "      <td>0</td>\n",
       "      <td>26.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>26.0000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>-0.267768</td>\n",
       "      <td>-0.124920</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>483</th>\n",
       "      <td>484</td>\n",
       "      <td>1</td>\n",
       "      <td>63.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>9.5875</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>2.448480</td>\n",
       "      <td>-0.455380</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>505</th>\n",
       "      <td>506</td>\n",
       "      <td>0</td>\n",
       "      <td>18.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>108.9000</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>-0.855065</td>\n",
       "      <td>1.544246</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>251</th>\n",
       "      <td>252</td>\n",
       "      <td>0</td>\n",
       "      <td>29.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>10.4625</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.047532</td>\n",
       "      <td>-0.437762</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>279</th>\n",
       "      <td>280</td>\n",
       "      <td>1</td>\n",
       "      <td>35.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>20.2500</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0.392941</td>\n",
       "      <td>-0.240694</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>838</th>\n",
       "      <td>839</td>\n",
       "      <td>1</td>\n",
       "      <td>32.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>56.4958</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0.172705</td>\n",
       "      <td>0.489104</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>564</th>\n",
       "      <td>565</td>\n",
       "      <td>0</td>\n",
       "      <td>30.705727</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0.077689</td>\n",
       "      <td>-0.486337</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>390</th>\n",
       "      <td>391</td>\n",
       "      <td>1</td>\n",
       "      <td>36.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>120.0000</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.466353</td>\n",
       "      <td>1.767741</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>643</th>\n",
       "      <td>644</td>\n",
       "      <td>1</td>\n",
       "      <td>29.568898</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>56.4958</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.005768</td>\n",
       "      <td>0.489104</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>267</th>\n",
       "      <td>268</td>\n",
       "      <td>1</td>\n",
       "      <td>25.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>7.7750</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.341180</td>\n",
       "      <td>-0.491874</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>501</th>\n",
       "      <td>502</td>\n",
       "      <td>0</td>\n",
       "      <td>21.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7.7500</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.634829</td>\n",
       "      <td>-0.492378</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>113</th>\n",
       "      <td>114</td>\n",
       "      <td>0</td>\n",
       "      <td>20.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>9.8250</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.708241</td>\n",
       "      <td>-0.450598</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>654</th>\n",
       "      <td>655</td>\n",
       "      <td>0</td>\n",
       "      <td>18.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>6.7500</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.855065</td>\n",
       "      <td>-0.512513</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>293</th>\n",
       "      <td>294</td>\n",
       "      <td>0</td>\n",
       "      <td>24.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>8.8500</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.414592</td>\n",
       "      <td>-0.470230</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>503</th>\n",
       "      <td>504</td>\n",
       "      <td>0</td>\n",
       "      <td>37.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>9.5875</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0.539765</td>\n",
       "      <td>-0.455380</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>362</th>\n",
       "      <td>363</td>\n",
       "      <td>0</td>\n",
       "      <td>45.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>14.4542</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1.127062</td>\n",
       "      <td>-0.357391</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>570</th>\n",
       "      <td>571</td>\n",
       "      <td>1</td>\n",
       "      <td>62.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>10.5000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>2.375068</td>\n",
       "      <td>-0.437007</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>264</th>\n",
       "      <td>265</td>\n",
       "      <td>0</td>\n",
       "      <td>35.295824</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7.7500</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0.414658</td>\n",
       "      <td>-0.492378</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>762</th>\n",
       "      <td>763</td>\n",
       "      <td>1</td>\n",
       "      <td>20.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7.2292</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.708241</td>\n",
       "      <td>-0.502864</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>85</th>\n",
       "      <td>86</td>\n",
       "      <td>1</td>\n",
       "      <td>33.000000</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>15.8500</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0.246117</td>\n",
       "      <td>-0.329287</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>140</th>\n",
       "      <td>141</td>\n",
       "      <td>0</td>\n",
       "      <td>25.352908</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>15.2458</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.315273</td>\n",
       "      <td>-0.341452</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>402</th>\n",
       "      <td>403</td>\n",
       "      <td>0</td>\n",
       "      <td>21.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>9.8250</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.634829</td>\n",
       "      <td>-0.450598</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>447</th>\n",
       "      <td>448</td>\n",
       "      <td>1</td>\n",
       "      <td>34.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>26.5500</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.319529</td>\n",
       "      <td>-0.113846</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>204</th>\n",
       "      <td>205</td>\n",
       "      <td>1</td>\n",
       "      <td>18.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.855065</td>\n",
       "      <td>-0.486337</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>240</th>\n",
       "      <td>241</td>\n",
       "      <td>0</td>\n",
       "      <td>20.800154</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>14.4542</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.649500</td>\n",
       "      <td>-0.357391</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>283</th>\n",
       "      <td>284</td>\n",
       "      <td>1</td>\n",
       "      <td>19.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.781653</td>\n",
       "      <td>-0.486337</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>647</th>\n",
       "      <td>648</td>\n",
       "      <td>1</td>\n",
       "      <td>56.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>35.5000</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1.934596</td>\n",
       "      <td>0.066360</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>852</th>\n",
       "      <td>853</td>\n",
       "      <td>0</td>\n",
       "      <td>9.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>15.2458</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-1.515774</td>\n",
       "      <td>-0.341452</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>788</th>\n",
       "      <td>789</td>\n",
       "      <td>1</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>20.5750</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-2.103071</td>\n",
       "      <td>-0.234150</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>68</th>\n",
       "      <td>69</td>\n",
       "      <td>1</td>\n",
       "      <td>17.000000</td>\n",
       "      <td>4</td>\n",
       "      <td>2</td>\n",
       "      <td>7.9250</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.928477</td>\n",
       "      <td>-0.488854</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>271</th>\n",
       "      <td>272</td>\n",
       "      <td>1</td>\n",
       "      <td>25.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.341180</td>\n",
       "      <td>-0.648422</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>882</th>\n",
       "      <td>883</td>\n",
       "      <td>0</td>\n",
       "      <td>22.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>10.5167</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.561417</td>\n",
       "      <td>-0.436671</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>65</th>\n",
       "      <td>66</td>\n",
       "      <td>1</td>\n",
       "      <td>17.107664</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>15.2458</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.920573</td>\n",
       "      <td>-0.341452</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>680</th>\n",
       "      <td>681</td>\n",
       "      <td>0</td>\n",
       "      <td>23.476432</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>8.1375</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.453029</td>\n",
       "      <td>-0.484576</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>49</th>\n",
       "      <td>50</td>\n",
       "      <td>0</td>\n",
       "      <td>18.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>17.8000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.855065</td>\n",
       "      <td>-0.290024</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>338</th>\n",
       "      <td>339</td>\n",
       "      <td>1</td>\n",
       "      <td>45.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1.127062</td>\n",
       "      <td>-0.486337</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>261</th>\n",
       "      <td>262</td>\n",
       "      <td>1</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>4</td>\n",
       "      <td>2</td>\n",
       "      <td>31.3875</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-1.956247</td>\n",
       "      <td>-0.016444</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     PassengerId  Survived        Age  SibSp  Parch      Fare  Cabin_No  \\\n",
       "298          299         1  41.574877      0      0   30.5000         0   \n",
       "14            15         0  14.000000      0      0    7.8542         1   \n",
       "803          804         1   0.420000      0      1    8.5167         1   \n",
       "474          475         0  22.000000      0      0    9.8375         1   \n",
       "642          643         0   2.000000      3      2   27.9000         1   \n",
       "55            56         1  44.064830      0      0   35.5000         0   \n",
       "587          588         1  60.000000      1      1   79.2000         0   \n",
       "740          741         1  38.426632      0      0   30.0000         0   \n",
       "839          840         1  50.903700      0      0   29.7000         0   \n",
       "301          302         1  23.332602      2      0   23.2500         1   \n",
       "567          568         0  29.000000      0      4   21.0750         1   \n",
       "712          713         1  48.000000      1      0   52.0000         0   \n",
       "489          490         1   9.000000      1      1   15.9000         1   \n",
       "312          313         0  26.000000      1      1   26.0000         1   \n",
       "483          484         1  63.000000      0      0    9.5875         1   \n",
       "505          506         0  18.000000      1      0  108.9000         0   \n",
       "251          252         0  29.000000      1      1   10.4625         0   \n",
       "279          280         1  35.000000      1      1   20.2500         1   \n",
       "838          839         1  32.000000      0      0   56.4958         1   \n",
       "564          565         0  30.705727      0      0    8.0500         1   \n",
       "390          391         1  36.000000      1      2  120.0000         0   \n",
       "643          644         1  29.568898      0      0   56.4958         1   \n",
       "267          268         1  25.000000      1      0    7.7750         1   \n",
       "501          502         0  21.000000      0      0    7.7500         1   \n",
       "113          114         0  20.000000      1      0    9.8250         1   \n",
       "654          655         0  18.000000      0      0    6.7500         1   \n",
       "293          294         0  24.000000      0      0    8.8500         1   \n",
       "503          504         0  37.000000      0      0    9.5875         1   \n",
       "362          363         0  45.000000      0      1   14.4542         1   \n",
       "570          571         1  62.000000      0      0   10.5000         1   \n",
       "264          265         0  35.295824      0      0    7.7500         1   \n",
       "762          763         1  20.000000      0      0    7.2292         1   \n",
       "85            86         1  33.000000      3      0   15.8500         1   \n",
       "140          141         0  25.352908      0      2   15.2458         1   \n",
       "402          403         0  21.000000      1      0    9.8250         1   \n",
       "447          448         1  34.000000      0      0   26.5500         1   \n",
       "204          205         1  18.000000      0      0    8.0500         1   \n",
       "240          241         0  20.800154      1      0   14.4542         1   \n",
       "283          284         1  19.000000      0      0    8.0500         1   \n",
       "647          648         1  56.000000      0      0   35.5000         0   \n",
       "852          853         0   9.000000      1      1   15.2458         1   \n",
       "788          789         1   1.000000      1      2   20.5750         1   \n",
       "68            69         1  17.000000      4      2    7.9250         1   \n",
       "271          272         1  25.000000      0      0    0.0000         1   \n",
       "882          883         0  22.000000      0      0   10.5167         1   \n",
       "65            66         1  17.107664      1      1   15.2458         1   \n",
       "680          681         0  23.476432      0      0    8.1375         1   \n",
       "49            50         0  18.000000      1      0   17.8000         1   \n",
       "338          339         1  45.000000      0      0    8.0500         1   \n",
       "261          262         1   3.000000      4      2   31.3875         1   \n",
       "\n",
       "     Cabin_Yes  Embarked_C  Embarked_Q  Embarked_S  Sex_female  Sex_male  \\\n",
       "298          1           0           0           1           0         1   \n",
       "14           0           0           0           1           1         0   \n",
       "803          0           1           0           0           0         1   \n",
       "474          0           0           0           1           1         0   \n",
       "642          0           0           0           1           1         0   \n",
       "55           1           0           0           1           0         1   \n",
       "587          1           1           0           0           0         1   \n",
       "740          1           0           0           1           0         1   \n",
       "839          1           1           0           0           0         1   \n",
       "301          0           0           1           0           0         1   \n",
       "567          0           0           0           1           1         0   \n",
       "712          1           0           0           1           0         1   \n",
       "489          0           0           0           1           0         1   \n",
       "312          0           0           0           1           1         0   \n",
       "483          0           0           0           1           1         0   \n",
       "505          1           1           0           0           0         1   \n",
       "251          1           0           0           1           1         0   \n",
       "279          0           0           0           1           1         0   \n",
       "838          0           0           0           1           0         1   \n",
       "564          0           0           0           1           1         0   \n",
       "390          1           0           0           1           0         1   \n",
       "643          0           0           0           1           0         1   \n",
       "267          0           0           0           1           0         1   \n",
       "501          0           0           1           0           1         0   \n",
       "113          0           0           0           1           1         0   \n",
       "654          0           0           1           0           1         0   \n",
       "293          0           0           0           1           1         0   \n",
       "503          0           0           0           1           1         0   \n",
       "362          0           1           0           0           1         0   \n",
       "570          0           0           0           1           0         1   \n",
       "264          0           0           1           0           1         0   \n",
       "762          0           1           0           0           0         1   \n",
       "85           0           0           0           1           1         0   \n",
       "140          0           1           0           0           1         0   \n",
       "402          0           0           0           1           1         0   \n",
       "447          0           0           0           1           0         1   \n",
       "204          0           0           0           1           0         1   \n",
       "240          0           1           0           0           1         0   \n",
       "283          0           0           0           1           0         1   \n",
       "647          1           1           0           0           0         1   \n",
       "852          0           1           0           0           1         0   \n",
       "788          0           0           0           1           0         1   \n",
       "68           0           0           0           1           1         0   \n",
       "271          0           0           0           1           0         1   \n",
       "882          0           0           0           1           1         0   \n",
       "65           0           1           0           0           0         1   \n",
       "680          0           0           1           0           1         0   \n",
       "49           0           0           0           1           1         0   \n",
       "338          0           0           0           1           0         1   \n",
       "261          0           0           0           1           0         1   \n",
       "\n",
       "     Pclass_1  Pclass_2  Pclass_3  Age_scaled  Fare_scaled  \n",
       "298         1         0         0    0.875617    -0.034314  \n",
       "14          0         0         1   -1.148714    -0.490280  \n",
       "803         0         0         1   -2.145650    -0.476941  \n",
       "474         0         0         1   -0.561417    -0.450347  \n",
       "642         0         0         1   -2.029659    -0.086664  \n",
       "55          1         0         0    1.058409     0.066360  \n",
       "587         1         0         0    2.228244     0.946246  \n",
       "740         1         0         0    0.644497    -0.044381  \n",
       "839         1         0         0    1.560465    -0.050421  \n",
       "301         0         0         1   -0.463588    -0.180290  \n",
       "567         0         0         1   -0.047532    -0.224083  \n",
       "712         1         0         0    1.347299     0.398582  \n",
       "489         0         0         1   -1.515774    -0.328280  \n",
       "312         0         1         0   -0.267768    -0.124920  \n",
       "483         0         0         1    2.448480    -0.455380  \n",
       "505         1         0         0   -0.855065     1.544246  \n",
       "251         0         0         1   -0.047532    -0.437762  \n",
       "279         0         0         1    0.392941    -0.240694  \n",
       "838         0         0         1    0.172705     0.489104  \n",
       "564         0         0         1    0.077689    -0.486337  \n",
       "390         1         0         0    0.466353     1.767741  \n",
       "643         0         0         1   -0.005768     0.489104  \n",
       "267         0         0         1   -0.341180    -0.491874  \n",
       "501         0         0         1   -0.634829    -0.492378  \n",
       "113         0         0         1   -0.708241    -0.450598  \n",
       "654         0         0         1   -0.855065    -0.512513  \n",
       "293         0         0         1   -0.414592    -0.470230  \n",
       "503         0         0         1    0.539765    -0.455380  \n",
       "362         0         0         1    1.127062    -0.357391  \n",
       "570         0         1         0    2.375068    -0.437007  \n",
       "264         0         0         1    0.414658    -0.492378  \n",
       "762         0         0         1   -0.708241    -0.502864  \n",
       "85          0         0         1    0.246117    -0.329287  \n",
       "140         0         0         1   -0.315273    -0.341452  \n",
       "402         0         0         1   -0.634829    -0.450598  \n",
       "447         1         0         0    0.319529    -0.113846  \n",
       "204         0         0         1   -0.855065    -0.486337  \n",
       "240         0         0         1   -0.649500    -0.357391  \n",
       "283         0         0         1   -0.781653    -0.486337  \n",
       "647         1         0         0    1.934596     0.066360  \n",
       "852         0         0         1   -1.515774    -0.341452  \n",
       "788         0         0         1   -2.103071    -0.234150  \n",
       "68          0         0         1   -0.928477    -0.488854  \n",
       "271         0         0         1   -0.341180    -0.648422  \n",
       "882         0         0         1   -0.561417    -0.436671  \n",
       "65          0         0         1   -0.920573    -0.341452  \n",
       "680         0         0         1   -0.453029    -0.484576  \n",
       "49          0         0         1   -0.855065    -0.290024  \n",
       "338         0         0         1    1.127062    -0.486337  \n",
       "261         0         0         1   -1.956247    -0.016444  "
      ]
     },
     "execution_count": 560,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn import cross_validation\n",
    "\n",
    "# 简单看看打分情况\n",
    "clf = linear_model.LogisticRegression(C=1.0, penalty='l1', tol=1e-6)\n",
    "all_data = df.filter(regex='Survived|Age_.*|SibSp|Parch|Fare_.*|Cabin_.*|Embarked_.*|Sex_.*|Pclass_.*')\n",
    "X = all_data.as_matrix()[:,1:]\n",
    "y = all_data.as_matrix()[:,0]\n",
    "print cross_validation.cross_val_score(clf, X, y, cv=5)\n",
    "\n",
    "\n",
    "# 分割数据\n",
    "split_train, split_cv = cross_validation.train_test_split(\u001d",
    "df, test_size=0.3, random_state=0)\n",
    "train_df = split_train.filter(regex='Survived|Age_.*|SibSp|Parch|Fare_.*|Cabin_.*|Embarked_.*|Sex_.*|Pclass_.*')\n",
    "# 生成模型\n",
    "clf = linear_model.LogisticRegression(C=1.0, penalty='l1', tol=1e-6)\n",
    "clf.fit(train_df.as_matrix()[:,1:], train_df.as_matrix()[:,0])\n",
    "\n",
    "\n",
    "\n",
    "# 对cross validation数据进行预测\n",
    "\n",
    "cv_df = split_cv.filter(regex='Survived|Age_.*|SibSp|Parch|Fare_.*|Cabin_.*|Embarked_.*|Sex_.*|Pclass_.*')\n",
    "predictions = clf.predict(cv_df.as_matrix()[:,1:])\n",
    "split_cv[ predictions != cv_df.as_matrix()[:,0] ].drop()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 562,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div style=\"max-height:1000px;max-width:1500px;overflow:auto;\">\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Name</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Ticket</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Embarked</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>15</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Vestrom, Miss. Hulda Amanda Adolfina</td>\n",
       "      <td>female</td>\n",
       "      <td>14.00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>350406</td>\n",
       "      <td>7.8542</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>49</th>\n",
       "      <td>50</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Arnold-Franchi, Mrs. Josef (Josefine Franchi)</td>\n",
       "      <td>female</td>\n",
       "      <td>18.00</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>349237</td>\n",
       "      <td>17.8000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>55</th>\n",
       "      <td>56</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Woolner, Mr. Hugh</td>\n",
       "      <td>male</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>19947</td>\n",
       "      <td>35.5000</td>\n",
       "      <td>C52</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>65</th>\n",
       "      <td>66</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>Moubarek, Master. Gerios</td>\n",
       "      <td>male</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2661</td>\n",
       "      <td>15.2458</td>\n",
       "      <td>NaN</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>68</th>\n",
       "      <td>69</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>Andersson, Miss. Erna Alexandra</td>\n",
       "      <td>female</td>\n",
       "      <td>17.00</td>\n",
       "      <td>4</td>\n",
       "      <td>2</td>\n",
       "      <td>3101281</td>\n",
       "      <td>7.9250</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>85</th>\n",
       "      <td>86</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>Backstrom, Mrs. Karl Alfred (Maria Mathilda Gu...</td>\n",
       "      <td>female</td>\n",
       "      <td>33.00</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>3101278</td>\n",
       "      <td>15.8500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>113</th>\n",
       "      <td>114</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Jussila, Miss. Katriina</td>\n",
       "      <td>female</td>\n",
       "      <td>20.00</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>4136</td>\n",
       "      <td>9.8250</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>140</th>\n",
       "      <td>141</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Boulos, Mrs. Joseph (Sultana)</td>\n",
       "      <td>female</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>2678</td>\n",
       "      <td>15.2458</td>\n",
       "      <td>NaN</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>204</th>\n",
       "      <td>205</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>Cohen, Mr. Gurshon \"Gus\"</td>\n",
       "      <td>male</td>\n",
       "      <td>18.00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>A/5 3540</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>240</th>\n",
       "      <td>241</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Zabour, Miss. Thamine</td>\n",
       "      <td>female</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>2665</td>\n",
       "      <td>14.4542</td>\n",
       "      <td>NaN</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>251</th>\n",
       "      <td>252</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Strom, Mrs. Wilhelm (Elna Matilda Persson)</td>\n",
       "      <td>female</td>\n",
       "      <td>29.00</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>347054</td>\n",
       "      <td>10.4625</td>\n",
       "      <td>G6</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>261</th>\n",
       "      <td>262</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>Asplund, Master. Edvin Rojj Felix</td>\n",
       "      <td>male</td>\n",
       "      <td>3.00</td>\n",
       "      <td>4</td>\n",
       "      <td>2</td>\n",
       "      <td>347077</td>\n",
       "      <td>31.3875</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>264</th>\n",
       "      <td>265</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Henry, Miss. Delia</td>\n",
       "      <td>female</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>382649</td>\n",
       "      <td>7.7500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Q</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>267</th>\n",
       "      <td>268</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>Persson, Mr. Ernst Ulrik</td>\n",
       "      <td>male</td>\n",
       "      <td>25.00</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>347083</td>\n",
       "      <td>7.7750</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>271</th>\n",
       "      <td>272</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>Tornquist, Mr. William Henry</td>\n",
       "      <td>male</td>\n",
       "      <td>25.00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>LINE</td>\n",
       "      <td>0.0000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>279</th>\n",
       "      <td>280</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>Abbott, Mrs. Stanton (Rosa Hunt)</td>\n",
       "      <td>female</td>\n",
       "      <td>35.00</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>C.A. 2673</td>\n",
       "      <td>20.2500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>283</th>\n",
       "      <td>284</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>Dorking, Mr. Edward Arthur</td>\n",
       "      <td>male</td>\n",
       "      <td>19.00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>A/5. 10482</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>293</th>\n",
       "      <td>294</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Haas, Miss. Aloisia</td>\n",
       "      <td>female</td>\n",
       "      <td>24.00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>349236</td>\n",
       "      <td>8.8500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>298</th>\n",
       "      <td>299</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Saalfeld, Mr. Adolphe</td>\n",
       "      <td>male</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>19988</td>\n",
       "      <td>30.5000</td>\n",
       "      <td>C106</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>301</th>\n",
       "      <td>302</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>McCoy, Mr. Bernard</td>\n",
       "      <td>male</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>367226</td>\n",
       "      <td>23.2500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Q</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>312</th>\n",
       "      <td>313</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>Lahtinen, Mrs. William (Anna Sylfven)</td>\n",
       "      <td>female</td>\n",
       "      <td>26.00</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>250651</td>\n",
       "      <td>26.0000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>338</th>\n",
       "      <td>339</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>Dahl, Mr. Karl Edwart</td>\n",
       "      <td>male</td>\n",
       "      <td>45.00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7598</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>362</th>\n",
       "      <td>363</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Barbara, Mrs. (Catherine David)</td>\n",
       "      <td>female</td>\n",
       "      <td>45.00</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>2691</td>\n",
       "      <td>14.4542</td>\n",
       "      <td>NaN</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>390</th>\n",
       "      <td>391</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Carter, Mr. William Ernest</td>\n",
       "      <td>male</td>\n",
       "      <td>36.00</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>113760</td>\n",
       "      <td>120.0000</td>\n",
       "      <td>B96 B98</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>402</th>\n",
       "      <td>403</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Jussila, Miss. Mari Aina</td>\n",
       "      <td>female</td>\n",
       "      <td>21.00</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>4137</td>\n",
       "      <td>9.8250</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>447</th>\n",
       "      <td>448</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Seward, Mr. Frederic Kimber</td>\n",
       "      <td>male</td>\n",
       "      <td>34.00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>113794</td>\n",
       "      <td>26.5500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>474</th>\n",
       "      <td>475</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Strandberg, Miss. Ida Sofia</td>\n",
       "      <td>female</td>\n",
       "      <td>22.00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7553</td>\n",
       "      <td>9.8375</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>483</th>\n",
       "      <td>484</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>Turkula, Mrs. (Hedwig)</td>\n",
       "      <td>female</td>\n",
       "      <td>63.00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>4134</td>\n",
       "      <td>9.5875</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>489</th>\n",
       "      <td>490</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>Coutts, Master. Eden Leslie \"Neville\"</td>\n",
       "      <td>male</td>\n",
       "      <td>9.00</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>C.A. 37671</td>\n",
       "      <td>15.9000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>501</th>\n",
       "      <td>502</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Canavan, Miss. Mary</td>\n",
       "      <td>female</td>\n",
       "      <td>21.00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>364846</td>\n",
       "      <td>7.7500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Q</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>503</th>\n",
       "      <td>504</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Laitinen, Miss. Kristina Sofia</td>\n",
       "      <td>female</td>\n",
       "      <td>37.00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>4135</td>\n",
       "      <td>9.5875</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>505</th>\n",
       "      <td>506</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>Penasco y Castellana, Mr. Victor de Satode</td>\n",
       "      <td>male</td>\n",
       "      <td>18.00</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>PC 17758</td>\n",
       "      <td>108.9000</td>\n",
       "      <td>C65</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>564</th>\n",
       "      <td>565</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Meanwell, Miss. (Marion Ogden)</td>\n",
       "      <td>female</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>SOTON/O.Q. 392087</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>567</th>\n",
       "      <td>568</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Palsson, Mrs. Nils (Alma Cornelia Berglund)</td>\n",
       "      <td>female</td>\n",
       "      <td>29.00</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>349909</td>\n",
       "      <td>21.0750</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>570</th>\n",
       "      <td>571</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>Harris, Mr. George</td>\n",
       "      <td>male</td>\n",
       "      <td>62.00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>S.W./PP 752</td>\n",
       "      <td>10.5000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>587</th>\n",
       "      <td>588</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Frolicher-Stehli, Mr. Maxmillian</td>\n",
       "      <td>male</td>\n",
       "      <td>60.00</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>13567</td>\n",
       "      <td>79.2000</td>\n",
       "      <td>B41</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>642</th>\n",
       "      <td>643</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Skoog, Miss. Margit Elizabeth</td>\n",
       "      <td>female</td>\n",
       "      <td>2.00</td>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "      <td>347088</td>\n",
       "      <td>27.9000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>643</th>\n",
       "      <td>644</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>Foo, Mr. Choong</td>\n",
       "      <td>male</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1601</td>\n",
       "      <td>56.4958</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>647</th>\n",
       "      <td>648</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Simonius-Blumer, Col. Oberst Alfons</td>\n",
       "      <td>male</td>\n",
       "      <td>56.00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>13213</td>\n",
       "      <td>35.5000</td>\n",
       "      <td>A26</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>654</th>\n",
       "      <td>655</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Hegarty, Miss. Hanora \"Nora\"</td>\n",
       "      <td>female</td>\n",
       "      <td>18.00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>365226</td>\n",
       "      <td>6.7500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Q</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>680</th>\n",
       "      <td>681</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Peters, Miss. Katie</td>\n",
       "      <td>female</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>330935</td>\n",
       "      <td>8.1375</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Q</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>712</th>\n",
       "      <td>713</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Taylor, Mr. Elmer Zebley</td>\n",
       "      <td>male</td>\n",
       "      <td>48.00</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>19996</td>\n",
       "      <td>52.0000</td>\n",
       "      <td>C126</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>740</th>\n",
       "      <td>741</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Hawksford, Mr. Walter James</td>\n",
       "      <td>male</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>16988</td>\n",
       "      <td>30.0000</td>\n",
       "      <td>D45</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>762</th>\n",
       "      <td>763</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>Barah, Mr. Hanna Assi</td>\n",
       "      <td>male</td>\n",
       "      <td>20.00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2663</td>\n",
       "      <td>7.2292</td>\n",
       "      <td>NaN</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>788</th>\n",
       "      <td>789</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>Dean, Master. Bertram Vere</td>\n",
       "      <td>male</td>\n",
       "      <td>1.00</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>C.A. 2315</td>\n",
       "      <td>20.5750</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>803</th>\n",
       "      <td>804</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>Thomas, Master. Assad Alexander</td>\n",
       "      <td>male</td>\n",
       "      <td>0.42</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>2625</td>\n",
       "      <td>8.5167</td>\n",
       "      <td>NaN</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>838</th>\n",
       "      <td>839</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>Chip, Mr. Chang</td>\n",
       "      <td>male</td>\n",
       "      <td>32.00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1601</td>\n",
       "      <td>56.4958</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>839</th>\n",
       "      <td>840</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Marechal, Mr. Pierre</td>\n",
       "      <td>male</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>11774</td>\n",
       "      <td>29.7000</td>\n",
       "      <td>C47</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>852</th>\n",
       "      <td>853</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Boulos, Miss. Nourelain</td>\n",
       "      <td>female</td>\n",
       "      <td>9.00</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2678</td>\n",
       "      <td>15.2458</td>\n",
       "      <td>NaN</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>882</th>\n",
       "      <td>883</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Dahlberg, Miss. Gerda Ulrika</td>\n",
       "      <td>female</td>\n",
       "      <td>22.00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7552</td>\n",
       "      <td>10.5167</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     PassengerId  Survived  Pclass  \\\n",
       "14            15         0       3   \n",
       "49            50         0       3   \n",
       "55            56         1       1   \n",
       "65            66         1       3   \n",
       "68            69         1       3   \n",
       "85            86         1       3   \n",
       "113          114         0       3   \n",
       "140          141         0       3   \n",
       "204          205         1       3   \n",
       "240          241         0       3   \n",
       "251          252         0       3   \n",
       "261          262         1       3   \n",
       "264          265         0       3   \n",
       "267          268         1       3   \n",
       "271          272         1       3   \n",
       "279          280         1       3   \n",
       "283          284         1       3   \n",
       "293          294         0       3   \n",
       "298          299         1       1   \n",
       "301          302         1       3   \n",
       "312          313         0       2   \n",
       "338          339         1       3   \n",
       "362          363         0       3   \n",
       "390          391         1       1   \n",
       "402          403         0       3   \n",
       "447          448         1       1   \n",
       "474          475         0       3   \n",
       "483          484         1       3   \n",
       "489          490         1       3   \n",
       "501          502         0       3   \n",
       "503          504         0       3   \n",
       "505          506         0       1   \n",
       "564          565         0       3   \n",
       "567          568         0       3   \n",
       "570          571         1       2   \n",
       "587          588         1       1   \n",
       "642          643         0       3   \n",
       "643          644         1       3   \n",
       "647          648         1       1   \n",
       "654          655         0       3   \n",
       "680          681         0       3   \n",
       "712          713         1       1   \n",
       "740          741         1       1   \n",
       "762          763         1       3   \n",
       "788          789         1       3   \n",
       "803          804         1       3   \n",
       "838          839         1       3   \n",
       "839          840         1       1   \n",
       "852          853         0       3   \n",
       "882          883         0       3   \n",
       "\n",
       "                                                  Name     Sex    Age  SibSp  \\\n",
       "14                Vestrom, Miss. Hulda Amanda Adolfina  female  14.00      0   \n",
       "49       Arnold-Franchi, Mrs. Josef (Josefine Franchi)  female  18.00      1   \n",
       "55                                   Woolner, Mr. Hugh    male    NaN      0   \n",
       "65                            Moubarek, Master. Gerios    male    NaN      1   \n",
       "68                     Andersson, Miss. Erna Alexandra  female  17.00      4   \n",
       "85   Backstrom, Mrs. Karl Alfred (Maria Mathilda Gu...  female  33.00      3   \n",
       "113                            Jussila, Miss. Katriina  female  20.00      1   \n",
       "140                      Boulos, Mrs. Joseph (Sultana)  female    NaN      0   \n",
       "204                           Cohen, Mr. Gurshon \"Gus\"    male  18.00      0   \n",
       "240                              Zabour, Miss. Thamine  female    NaN      1   \n",
       "251         Strom, Mrs. Wilhelm (Elna Matilda Persson)  female  29.00      1   \n",
       "261                  Asplund, Master. Edvin Rojj Felix    male   3.00      4   \n",
       "264                                 Henry, Miss. Delia  female    NaN      0   \n",
       "267                           Persson, Mr. Ernst Ulrik    male  25.00      1   \n",
       "271                       Tornquist, Mr. William Henry    male  25.00      0   \n",
       "279                   Abbott, Mrs. Stanton (Rosa Hunt)  female  35.00      1   \n",
       "283                         Dorking, Mr. Edward Arthur    male  19.00      0   \n",
       "293                                Haas, Miss. Aloisia  female  24.00      0   \n",
       "298                              Saalfeld, Mr. Adolphe    male    NaN      0   \n",
       "301                                 McCoy, Mr. Bernard    male    NaN      2   \n",
       "312              Lahtinen, Mrs. William (Anna Sylfven)  female  26.00      1   \n",
       "338                              Dahl, Mr. Karl Edwart    male  45.00      0   \n",
       "362                    Barbara, Mrs. (Catherine David)  female  45.00      0   \n",
       "390                         Carter, Mr. William Ernest    male  36.00      1   \n",
       "402                           Jussila, Miss. Mari Aina  female  21.00      1   \n",
       "447                        Seward, Mr. Frederic Kimber    male  34.00      0   \n",
       "474                        Strandberg, Miss. Ida Sofia  female  22.00      0   \n",
       "483                             Turkula, Mrs. (Hedwig)  female  63.00      0   \n",
       "489              Coutts, Master. Eden Leslie \"Neville\"    male   9.00      1   \n",
       "501                                Canavan, Miss. Mary  female  21.00      0   \n",
       "503                     Laitinen, Miss. Kristina Sofia  female  37.00      0   \n",
       "505         Penasco y Castellana, Mr. Victor de Satode    male  18.00      1   \n",
       "564                     Meanwell, Miss. (Marion Ogden)  female    NaN      0   \n",
       "567        Palsson, Mrs. Nils (Alma Cornelia Berglund)  female  29.00      0   \n",
       "570                                 Harris, Mr. George    male  62.00      0   \n",
       "587                   Frolicher-Stehli, Mr. Maxmillian    male  60.00      1   \n",
       "642                      Skoog, Miss. Margit Elizabeth  female   2.00      3   \n",
       "643                                    Foo, Mr. Choong    male    NaN      0   \n",
       "647                Simonius-Blumer, Col. Oberst Alfons    male  56.00      0   \n",
       "654                       Hegarty, Miss. Hanora \"Nora\"  female  18.00      0   \n",
       "680                                Peters, Miss. Katie  female    NaN      0   \n",
       "712                           Taylor, Mr. Elmer Zebley    male  48.00      1   \n",
       "740                        Hawksford, Mr. Walter James    male    NaN      0   \n",
       "762                              Barah, Mr. Hanna Assi    male  20.00      0   \n",
       "788                         Dean, Master. Bertram Vere    male   1.00      1   \n",
       "803                    Thomas, Master. Assad Alexander    male   0.42      0   \n",
       "838                                    Chip, Mr. Chang    male  32.00      0   \n",
       "839                               Marechal, Mr. Pierre    male    NaN      0   \n",
       "852                            Boulos, Miss. Nourelain  female   9.00      1   \n",
       "882                       Dahlberg, Miss. Gerda Ulrika  female  22.00      0   \n",
       "\n",
       "     Parch             Ticket      Fare    Cabin Embarked  \n",
       "14       0             350406    7.8542      NaN        S  \n",
       "49       0             349237   17.8000      NaN        S  \n",
       "55       0              19947   35.5000      C52        S  \n",
       "65       1               2661   15.2458      NaN        C  \n",
       "68       2            3101281    7.9250      NaN        S  \n",
       "85       0            3101278   15.8500      NaN        S  \n",
       "113      0               4136    9.8250      NaN        S  \n",
       "140      2               2678   15.2458      NaN        C  \n",
       "204      0           A/5 3540    8.0500      NaN        S  \n",
       "240      0               2665   14.4542      NaN        C  \n",
       "251      1             347054   10.4625       G6        S  \n",
       "261      2             347077   31.3875      NaN        S  \n",
       "264      0             382649    7.7500      NaN        Q  \n",
       "267      0             347083    7.7750      NaN        S  \n",
       "271      0               LINE    0.0000      NaN        S  \n",
       "279      1          C.A. 2673   20.2500      NaN        S  \n",
       "283      0         A/5. 10482    8.0500      NaN        S  \n",
       "293      0             349236    8.8500      NaN        S  \n",
       "298      0              19988   30.5000     C106        S  \n",
       "301      0             367226   23.2500      NaN        Q  \n",
       "312      1             250651   26.0000      NaN        S  \n",
       "338      0               7598    8.0500      NaN        S  \n",
       "362      1               2691   14.4542      NaN        C  \n",
       "390      2             113760  120.0000  B96 B98        S  \n",
       "402      0               4137    9.8250      NaN        S  \n",
       "447      0             113794   26.5500      NaN        S  \n",
       "474      0               7553    9.8375      NaN        S  \n",
       "483      0               4134    9.5875      NaN        S  \n",
       "489      1         C.A. 37671   15.9000      NaN        S  \n",
       "501      0             364846    7.7500      NaN        Q  \n",
       "503      0               4135    9.5875      NaN        S  \n",
       "505      0           PC 17758  108.9000      C65        C  \n",
       "564      0  SOTON/O.Q. 392087    8.0500      NaN        S  \n",
       "567      4             349909   21.0750      NaN        S  \n",
       "570      0        S.W./PP 752   10.5000      NaN        S  \n",
       "587      1              13567   79.2000      B41        C  \n",
       "642      2             347088   27.9000      NaN        S  \n",
       "643      0               1601   56.4958      NaN        S  \n",
       "647      0              13213   35.5000      A26        C  \n",
       "654      0             365226    6.7500      NaN        Q  \n",
       "680      0             330935    8.1375      NaN        Q  \n",
       "712      0              19996   52.0000     C126        S  \n",
       "740      0              16988   30.0000      D45        S  \n",
       "762      0               2663    7.2292      NaN        C  \n",
       "788      2          C.A. 2315   20.5750      NaN        S  \n",
       "803      1               2625    8.5167      NaN        C  \n",
       "838      0               1601   56.4958      NaN        S  \n",
       "839      0              11774   29.7000      C47        C  \n",
       "852      1               2678   15.2458      NaN        C  \n",
       "882      0               7552   10.5167      NaN        S  "
      ]
     },
     "execution_count": 562,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 去除预测错误的case看原始dataframe数据\n",
    "#split_cv['PredictResult'] = predictions\n",
    "origin_data_train = pd.read_csv(\"Train.csv\")\n",
    "bad_cases = origin_data_train.loc[origin_data_train['PassengerId'].isin(split_cv[predictions != cv_df.as_matrix()[:,0]]['PassengerId'].values)]\n",
    "bad_cases"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "对比bad case，我们仔细看看我们预测错的样本，到底是哪些特征有问题，咱们处理得还不够细？<br>\n",
    "\n",
    "我们随便列一些可能可以做的优化操作：<br>\n",
    "\n",
    "* Age属性不使用现在的拟合方式，而是根据名称中的『Mr』『Mrs』『Miss』等的平均值进行填充。\n",
    "* Age不做成一个连续值属性，而是使用一个步长进行离散化，变成离散的类目feature。\n",
    "* Cabin再细化一些，对于有记录的Cabin属性，我们将其分为前面的字母部分(我猜是位置和船层之类的信息) 和 后面的数字部分(应该是房间号，有意思的事情是，如果你仔细看看原始数据，你会发现，这个值大的情况下，似乎获救的可能性高一些)。\n",
    "* Pclass和Sex俩太重要了，我们试着用它们去组出一个组合属性来试试，这也是另外一种程度的细化。\n",
    "* 单加一个Child字段，Age<=12的，设为1，其余为0(你去看看数据，确实小盆友优先程度很高啊)\n",
    "* 如果名字里面有『Mrs』，而Parch>1的，我们猜测她可能是一个母亲，应该获救的概率也会提高，因此可以多加一个Mother字段，此种情况下设为1，其余情况下设为0\n",
    "* 登船港口可以考虑先去掉试试(Q和C本来就没权重，S有点诡异)\n",
    "* 把堂兄弟/兄妹 和 Parch 还有自己 个数加在一起组一个Family_size字段(考虑到大家族可能对最后的结果有影响)\n",
    "* Name是一个我们一直没有触碰的属性，我们可以做一些简单的处理，比如说男性中带某些字眼的(‘Capt’, ‘Don’, ‘Major’, ‘Sir’)可以统一到一个Title，女性也一样。\n",
    "\n",
    "大家接着往下挖掘，可能还可以想到更多可以细挖的部分。我这里先列这些了，然后我们可以使用手头上的”train_df”和”cv_df”开始试验这些feature engineering的tricks是否有效了。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 440,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div style=\"max-height:1000px;max-width:1500px;overflow:auto;\">\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Name</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Ticket</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Embarked</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>449</th>\n",
       "      <td>450</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Peuchen, Major. Arthur Godfrey</td>\n",
       "      <td>male</td>\n",
       "      <td>52</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>113786</td>\n",
       "      <td>30.50</td>\n",
       "      <td>Yes</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>536</th>\n",
       "      <td>537</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>Butt, Major. Archibald Willingham</td>\n",
       "      <td>male</td>\n",
       "      <td>45</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>113050</td>\n",
       "      <td>26.55</td>\n",
       "      <td>Yes</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     PassengerId  Survived  Pclass                               Name   Sex  \\\n",
       "449          450         1       1     Peuchen, Major. Arthur Godfrey  male   \n",
       "536          537         0       1  Butt, Major. Archibald Willingham  male   \n",
       "\n",
       "     Age  SibSp  Parch  Ticket   Fare Cabin Embarked  \n",
       "449   52      0      0  113786  30.50   Yes        S  \n",
       "536   45      0      0  113050  26.55   Yes        S  "
      ]
     },
     "execution_count": 440,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data_train[data_train['Name'].str.contains(\"Major\")]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 546,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "LogisticRegression(C=1.0, class_weight=None, dual=False, fit_intercept=True,\n",
       "          intercept_scaling=1, max_iter=100, multi_class='ovr',\n",
       "          penalty='l1', random_state=None, solver='liblinear', tol=1e-06,\n",
       "          verbose=0)"
      ]
     },
     "execution_count": 546,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data_train = pd.read_csv(\"Train.csv\")\n",
    "data_train['Sex_Pclass'] = data_train.Sex + \"_\" + data_train.Pclass.map(str)\n",
    "\n",
    "from sklearn.ensemble import RandomForestRegressor\n",
    " \n",
    "### 使用 RandomForestClassifier 填补缺失的年龄属性\n",
    "def set_missing_ages(df):\n",
    "    \n",
    "    # 把已有的数值型特征取出来丢进Random Forest Regressor中\n",
    "    age_df = df[['Age','Fare', 'Parch', 'SibSp', 'Pclass']]\n",
    "\n",
    "    # 乘客分成已知年龄和未知年龄两部分\n",
    "    known_age = age_df[age_df.Age.notnull()].as_matrix()\n",
    "    unknown_age = age_df[age_df.Age.isnull()].as_matrix()\n",
    "\n",
    "    # y即目标年龄\n",
    "    y = known_age[:, 0]\n",
    "\n",
    "    # X即特征属性值\n",
    "    X = known_age[:, 1:]\n",
    "\n",
    "    # fit到RandomForestRegressor之中\n",
    "    rfr = RandomForestRegressor(random_state=0, n_estimators=2000, n_jobs=-1)\n",
    "    rfr.fit(X, y)\n",
    "    \n",
    "    # 用得到的模型进行未知年龄结果预测\n",
    "    predictedAges = rfr.predict(unknown_age[:, 1::])\n",
    "    \n",
    "    # 用得到的预测结果填补原缺失数据\n",
    "    df.loc[ (df.Age.isnull()), 'Age' ] = predictedAges \n",
    "    \n",
    "    return df, rfr\n",
    "\n",
    "def set_Cabin_type(df):\n",
    "    df.loc[ (df.Cabin.notnull()), 'Cabin' ] = \"Yes\"\n",
    "    df.loc[ (df.Cabin.isnull()), 'Cabin' ] = \"No\"\n",
    "    return df\n",
    "\n",
    "data_train, rfr = set_missing_ages(data_train)\n",
    "data_train = set_Cabin_type(data_train)\n",
    "\n",
    "dummies_Cabin = pd.get_dummies(data_train['Cabin'], prefix= 'Cabin')\n",
    "dummies_Embarked = pd.get_dummies(data_train['Embarked'], prefix= 'Embarked')\n",
    "dummies_Sex = pd.get_dummies(data_train['Sex'], prefix= 'Sex')\n",
    "dummies_Pclass = pd.get_dummies(data_train['Pclass'], prefix= 'Pclass')\n",
    "dummies_Sex_Pclass = pd.get_dummies(data_train['Sex_Pclass'], prefix= 'Sex_Pclass')\n",
    "\n",
    "\n",
    "df = pd.concat([data_train, dummies_Cabin, dummies_Embarked, dummies_Sex, dummies_Pclass, dummies_Sex_Pclass], axis=1)\n",
    "df.drop(['Pclass', 'Name', 'Sex', 'Ticket', 'Cabin', 'Embarked', 'Sex_Pclass'], axis=1, inplace=True)\n",
    "import sklearn.preprocessing as preprocessing\n",
    "scaler = preprocessing.StandardScaler()\n",
    "age_scale_param = scaler.fit(df['Age'])\n",
    "df['Age_scaled'] = scaler.fit_transform(df['Age'], age_scale_param)\n",
    "fare_scale_param = scaler.fit(df['Fare'])\n",
    "df['Fare_scaled'] = scaler.fit_transform(df['Fare'], fare_scale_param)\n",
    "\n",
    "from sklearn import linear_model\n",
    "\n",
    "train_df = df.filter(regex='Survived|Age_.*|SibSp|Parch|Fare_.*|Cabin_.*|Embarked_.*|Sex_.*|Pclass.*')\n",
    "train_np = train_df.as_matrix()\n",
    "\n",
    "# y即Survival结果\n",
    "y = train_np[:, 0]\n",
    "\n",
    "# X即特征属性值\n",
    "X = train_np[:, 1:]\n",
    "\n",
    "# fit到RandomForestRegressor之中\n",
    "clf = linear_model.LogisticRegression(C=1.0, penalty='l1', tol=1e-6)\n",
    "clf.fit(X, y)\n",
    "clf"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 543,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div style=\"max-height:1000px;max-width:1500px;overflow:auto;\">\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Cabin_No</th>\n",
       "      <th>Cabin_Yes</th>\n",
       "      <th>Embarked_C</th>\n",
       "      <th>Embarked_Q</th>\n",
       "      <th>Embarked_S</th>\n",
       "      <th>...</th>\n",
       "      <th>Pclass_2</th>\n",
       "      <th>Pclass_3</th>\n",
       "      <th>Sex_Pclass_female_1</th>\n",
       "      <th>Sex_Pclass_female_2</th>\n",
       "      <th>Sex_Pclass_female_3</th>\n",
       "      <th>Sex_Pclass_male_1</th>\n",
       "      <th>Sex_Pclass_male_2</th>\n",
       "      <th>Sex_Pclass_male_3</th>\n",
       "      <th>Age_scaled</th>\n",
       "      <th>Fare_scaled</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>892</td>\n",
       "      <td>34.500000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7.8292</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0.307495</td>\n",
       "      <td>-0.496637</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>893</td>\n",
       "      <td>47.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>7.0000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1.256225</td>\n",
       "      <td>-0.511497</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>894</td>\n",
       "      <td>62.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>9.6875</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>2.394702</td>\n",
       "      <td>-0.463335</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>895</td>\n",
       "      <td>27.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>8.6625</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.261743</td>\n",
       "      <td>-0.481704</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>896</td>\n",
       "      <td>22.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>12.2875</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>-0.641235</td>\n",
       "      <td>-0.416740</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>897</td>\n",
       "      <td>14.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>9.2250</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-1.248423</td>\n",
       "      <td>-0.471623</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>898</td>\n",
       "      <td>30.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7.6292</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>-0.034048</td>\n",
       "      <td>-0.500221</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>899</td>\n",
       "      <td>26.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>29.0000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>-0.337642</td>\n",
       "      <td>-0.117238</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>900</td>\n",
       "      <td>18.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7.2292</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>-0.944829</td>\n",
       "      <td>-0.507390</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>901</td>\n",
       "      <td>21.000000</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>24.1500</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.717134</td>\n",
       "      <td>-0.204154</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>902</td>\n",
       "      <td>27.947206</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7.8958</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.189852</td>\n",
       "      <td>-0.495444</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>903</td>\n",
       "      <td>46.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>26.0000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1.180327</td>\n",
       "      <td>-0.171000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>904</td>\n",
       "      <td>23.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>82.2667</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>-0.565337</td>\n",
       "      <td>0.837349</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>905</td>\n",
       "      <td>63.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>26.0000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>2.470600</td>\n",
       "      <td>-0.171000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>906</td>\n",
       "      <td>47.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>61.1750</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1.256225</td>\n",
       "      <td>0.459367</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>907</td>\n",
       "      <td>24.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>27.7208</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>-0.489439</td>\n",
       "      <td>-0.140162</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>908</td>\n",
       "      <td>35.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>12.3500</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0.345444</td>\n",
       "      <td>-0.415620</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>909</td>\n",
       "      <td>21.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7.2250</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.717134</td>\n",
       "      <td>-0.507465</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>910</td>\n",
       "      <td>27.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>7.9250</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>-0.261743</td>\n",
       "      <td>-0.494920</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>911</td>\n",
       "      <td>45.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7.2250</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1.104428</td>\n",
       "      <td>-0.507465</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>912</td>\n",
       "      <td>55.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>59.4000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1.863413</td>\n",
       "      <td>0.427557</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>913</td>\n",
       "      <td>9.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>3.1708</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-1.627915</td>\n",
       "      <td>-0.580120</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>914</td>\n",
       "      <td>52.314311</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>31.6833</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1.659573</td>\n",
       "      <td>-0.069151</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>915</td>\n",
       "      <td>21.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>61.3792</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>-0.717134</td>\n",
       "      <td>0.463026</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>916</td>\n",
       "      <td>48.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>262.3750</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1.332124</td>\n",
       "      <td>4.065049</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>917</td>\n",
       "      <td>50.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>14.5000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1.483921</td>\n",
       "      <td>-0.377090</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>918</td>\n",
       "      <td>22.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>61.9792</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>-0.641235</td>\n",
       "      <td>0.473779</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>919</td>\n",
       "      <td>22.500000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7.2250</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.603286</td>\n",
       "      <td>-0.507465</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>920</td>\n",
       "      <td>41.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>30.5000</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.800835</td>\n",
       "      <td>-0.090356</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>921</td>\n",
       "      <td>23.481602</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>21.6792</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.528784</td>\n",
       "      <td>-0.248433</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>388</th>\n",
       "      <td>1280</td>\n",
       "      <td>21.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7.7500</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.717134</td>\n",
       "      <td>-0.498056</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>389</th>\n",
       "      <td>1281</td>\n",
       "      <td>6.000000</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>21.0750</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-1.855610</td>\n",
       "      <td>-0.259261</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>390</th>\n",
       "      <td>1282</td>\n",
       "      <td>23.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>93.5000</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>-0.565337</td>\n",
       "      <td>1.038659</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>391</th>\n",
       "      <td>1283</td>\n",
       "      <td>51.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>39.4000</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1.559819</td>\n",
       "      <td>0.069140</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>392</th>\n",
       "      <td>1284</td>\n",
       "      <td>13.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>20.2500</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-1.324321</td>\n",
       "      <td>-0.274045</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>393</th>\n",
       "      <td>1285</td>\n",
       "      <td>47.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>10.5000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1.256225</td>\n",
       "      <td>-0.448774</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>394</th>\n",
       "      <td>1286</td>\n",
       "      <td>29.000000</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>22.0250</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.109946</td>\n",
       "      <td>-0.242236</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>395</th>\n",
       "      <td>1287</td>\n",
       "      <td>18.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>60.0000</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>-0.944829</td>\n",
       "      <td>0.438310</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>396</th>\n",
       "      <td>1288</td>\n",
       "      <td>24.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7.2500</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.489439</td>\n",
       "      <td>-0.507017</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>397</th>\n",
       "      <td>1289</td>\n",
       "      <td>48.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>79.2000</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1.332124</td>\n",
       "      <td>0.782391</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>398</th>\n",
       "      <td>1290</td>\n",
       "      <td>22.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7.7750</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.641235</td>\n",
       "      <td>-0.497608</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>399</th>\n",
       "      <td>1291</td>\n",
       "      <td>31.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7.7333</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0.041850</td>\n",
       "      <td>-0.498356</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>400</th>\n",
       "      <td>1292</td>\n",
       "      <td>30.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>164.8667</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>-0.034048</td>\n",
       "      <td>2.317614</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>401</th>\n",
       "      <td>1293</td>\n",
       "      <td>38.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>21.0000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0.573139</td>\n",
       "      <td>-0.260605</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>402</th>\n",
       "      <td>1294</td>\n",
       "      <td>22.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>59.4000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>-0.641235</td>\n",
       "      <td>0.427557</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>403</th>\n",
       "      <td>1295</td>\n",
       "      <td>17.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>47.1000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>-1.020728</td>\n",
       "      <td>0.207130</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>404</th>\n",
       "      <td>1296</td>\n",
       "      <td>43.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>27.7208</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.952632</td>\n",
       "      <td>-0.140162</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>405</th>\n",
       "      <td>1297</td>\n",
       "      <td>20.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>13.8625</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>-0.793032</td>\n",
       "      <td>-0.388515</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>406</th>\n",
       "      <td>1298</td>\n",
       "      <td>23.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>10.5000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>-0.565337</td>\n",
       "      <td>-0.448774</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>407</th>\n",
       "      <td>1299</td>\n",
       "      <td>50.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>211.5000</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1.483921</td>\n",
       "      <td>3.153324</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>408</th>\n",
       "      <td>1300</td>\n",
       "      <td>19.895581</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7.7208</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>-0.800957</td>\n",
       "      <td>-0.498580</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>409</th>\n",
       "      <td>1301</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>13.7750</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>-2.083305</td>\n",
       "      <td>-0.390083</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>410</th>\n",
       "      <td>1302</td>\n",
       "      <td>35.295824</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7.7500</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.367897</td>\n",
       "      <td>-0.498056</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>411</th>\n",
       "      <td>1303</td>\n",
       "      <td>37.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>90.0000</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.497241</td>\n",
       "      <td>0.975936</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>412</th>\n",
       "      <td>1304</td>\n",
       "      <td>28.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7.7750</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>-0.185845</td>\n",
       "      <td>-0.497608</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>413</th>\n",
       "      <td>1305</td>\n",
       "      <td>30.705727</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0.019516</td>\n",
       "      <td>-0.492680</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>414</th>\n",
       "      <td>1306</td>\n",
       "      <td>39.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>108.9000</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.649038</td>\n",
       "      <td>1.314641</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>415</th>\n",
       "      <td>1307</td>\n",
       "      <td>38.500000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7.2500</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0.611089</td>\n",
       "      <td>-0.507017</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>416</th>\n",
       "      <td>1308</td>\n",
       "      <td>30.705727</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0.019516</td>\n",
       "      <td>-0.492680</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>417</th>\n",
       "      <td>1309</td>\n",
       "      <td>25.729701</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>22.3583</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.358157</td>\n",
       "      <td>-0.236263</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>418 rows × 23 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     PassengerId        Age  SibSp  Parch      Fare  Cabin_No  Cabin_Yes  \\\n",
       "0            892  34.500000      0      0    7.8292         1          0   \n",
       "1            893  47.000000      1      0    7.0000         1          0   \n",
       "2            894  62.000000      0      0    9.6875         1          0   \n",
       "3            895  27.000000      0      0    8.6625         1          0   \n",
       "4            896  22.000000      1      1   12.2875         1          0   \n",
       "5            897  14.000000      0      0    9.2250         1          0   \n",
       "6            898  30.000000      0      0    7.6292         1          0   \n",
       "7            899  26.000000      1      1   29.0000         1          0   \n",
       "8            900  18.000000      0      0    7.2292         1          0   \n",
       "9            901  21.000000      2      0   24.1500         1          0   \n",
       "10           902  27.947206      0      0    7.8958         1          0   \n",
       "11           903  46.000000      0      0   26.0000         1          0   \n",
       "12           904  23.000000      1      0   82.2667         0          1   \n",
       "13           905  63.000000      1      0   26.0000         1          0   \n",
       "14           906  47.000000      1      0   61.1750         0          1   \n",
       "15           907  24.000000      1      0   27.7208         1          0   \n",
       "16           908  35.000000      0      0   12.3500         1          0   \n",
       "17           909  21.000000      0      0    7.2250         1          0   \n",
       "18           910  27.000000      1      0    7.9250         1          0   \n",
       "19           911  45.000000      0      0    7.2250         1          0   \n",
       "20           912  55.000000      1      0   59.4000         1          0   \n",
       "21           913   9.000000      0      1    3.1708         1          0   \n",
       "22           914  52.314311      0      0   31.6833         1          0   \n",
       "23           915  21.000000      0      1   61.3792         1          0   \n",
       "24           916  48.000000      1      3  262.3750         0          1   \n",
       "25           917  50.000000      1      0   14.5000         1          0   \n",
       "26           918  22.000000      0      1   61.9792         0          1   \n",
       "27           919  22.500000      0      0    7.2250         1          0   \n",
       "28           920  41.000000      0      0   30.5000         0          1   \n",
       "29           921  23.481602      2      0   21.6792         1          0   \n",
       "..           ...        ...    ...    ...       ...       ...        ...   \n",
       "388         1280  21.000000      0      0    7.7500         1          0   \n",
       "389         1281   6.000000      3      1   21.0750         1          0   \n",
       "390         1282  23.000000      0      0   93.5000         0          1   \n",
       "391         1283  51.000000      0      1   39.4000         0          1   \n",
       "392         1284  13.000000      0      2   20.2500         1          0   \n",
       "393         1285  47.000000      0      0   10.5000         1          0   \n",
       "394         1286  29.000000      3      1   22.0250         1          0   \n",
       "395         1287  18.000000      1      0   60.0000         0          1   \n",
       "396         1288  24.000000      0      0    7.2500         1          0   \n",
       "397         1289  48.000000      1      1   79.2000         0          1   \n",
       "398         1290  22.000000      0      0    7.7750         1          0   \n",
       "399         1291  31.000000      0      0    7.7333         1          0   \n",
       "400         1292  30.000000      0      0  164.8667         0          1   \n",
       "401         1293  38.000000      1      0   21.0000         1          0   \n",
       "402         1294  22.000000      0      1   59.4000         1          0   \n",
       "403         1295  17.000000      0      0   47.1000         1          0   \n",
       "404         1296  43.000000      1      0   27.7208         0          1   \n",
       "405         1297  20.000000      0      0   13.8625         0          1   \n",
       "406         1298  23.000000      1      0   10.5000         1          0   \n",
       "407         1299  50.000000      1      1  211.5000         0          1   \n",
       "408         1300  19.895581      0      0    7.7208         1          0   \n",
       "409         1301   3.000000      1      1   13.7750         1          0   \n",
       "410         1302  35.295824      0      0    7.7500         1          0   \n",
       "411         1303  37.000000      1      0   90.0000         0          1   \n",
       "412         1304  28.000000      0      0    7.7750         1          0   \n",
       "413         1305  30.705727      0      0    8.0500         1          0   \n",
       "414         1306  39.000000      0      0  108.9000         0          1   \n",
       "415         1307  38.500000      0      0    7.2500         1          0   \n",
       "416         1308  30.705727      0      0    8.0500         1          0   \n",
       "417         1309  25.729701      1      1   22.3583         1          0   \n",
       "\n",
       "     Embarked_C  Embarked_Q  Embarked_S     ...       Pclass_2  Pclass_3  \\\n",
       "0             0           1           0     ...              0         1   \n",
       "1             0           0           1     ...              0         1   \n",
       "2             0           1           0     ...              1         0   \n",
       "3             0           0           1     ...              0         1   \n",
       "4             0           0           1     ...              0         1   \n",
       "5             0           0           1     ...              0         1   \n",
       "6             0           1           0     ...              0         1   \n",
       "7             0           0           1     ...              1         0   \n",
       "8             1           0           0     ...              0         1   \n",
       "9             0           0           1     ...              0         1   \n",
       "10            0           0           1     ...              0         1   \n",
       "11            0           0           1     ...              0         0   \n",
       "12            0           0           1     ...              0         0   \n",
       "13            0           0           1     ...              1         0   \n",
       "14            0           0           1     ...              0         0   \n",
       "15            1           0           0     ...              1         0   \n",
       "16            0           1           0     ...              1         0   \n",
       "17            1           0           0     ...              0         1   \n",
       "18            0           0           1     ...              0         1   \n",
       "19            1           0           0     ...              0         1   \n",
       "20            1           0           0     ...              0         0   \n",
       "21            0           0           1     ...              0         1   \n",
       "22            0           0           1     ...              0         0   \n",
       "23            1           0           0     ...              0         0   \n",
       "24            1           0           0     ...              0         0   \n",
       "25            0           0           1     ...              0         1   \n",
       "26            1           0           0     ...              0         0   \n",
       "27            1           0           0     ...              0         1   \n",
       "28            0           0           1     ...              0         0   \n",
       "29            1           0           0     ...              0         1   \n",
       "..          ...         ...         ...     ...            ...       ...   \n",
       "388           0           1           0     ...              0         1   \n",
       "389           0           0           1     ...              0         1   \n",
       "390           0           0           1     ...              0         0   \n",
       "391           0           0           1     ...              0         0   \n",
       "392           0           0           1     ...              0         1   \n",
       "393           0           0           1     ...              1         0   \n",
       "394           0           0           1     ...              0         1   \n",
       "395           0           0           1     ...              0         0   \n",
       "396           0           1           0     ...              0         1   \n",
       "397           1           0           0     ...              0         0   \n",
       "398           0           0           1     ...              0         1   \n",
       "399           0           1           0     ...              0         1   \n",
       "400           0           0           1     ...              0         0   \n",
       "401           0           0           1     ...              1         0   \n",
       "402           1           0           0     ...              0         0   \n",
       "403           0           0           1     ...              0         0   \n",
       "404           1           0           0     ...              0         0   \n",
       "405           1           0           0     ...              1         0   \n",
       "406           0           0           1     ...              1         0   \n",
       "407           1           0           0     ...              0         0   \n",
       "408           0           1           0     ...              0         1   \n",
       "409           0           0           1     ...              0         1   \n",
       "410           0           1           0     ...              0         1   \n",
       "411           0           1           0     ...              0         0   \n",
       "412           0           0           1     ...              0         1   \n",
       "413           0           0           1     ...              0         1   \n",
       "414           1           0           0     ...              0         0   \n",
       "415           0           0           1     ...              0         1   \n",
       "416           0           0           1     ...              0         1   \n",
       "417           1           0           0     ...              0         1   \n",
       "\n",
       "     Sex_Pclass_female_1  Sex_Pclass_female_2  Sex_Pclass_female_3  \\\n",
       "0                      0                    0                    0   \n",
       "1                      0                    0                    1   \n",
       "2                      0                    0                    0   \n",
       "3                      0                    0                    0   \n",
       "4                      0                    0                    1   \n",
       "5                      0                    0                    0   \n",
       "6                      0                    0                    1   \n",
       "7                      0                    0                    0   \n",
       "8                      0                    0                    1   \n",
       "9                      0                    0                    0   \n",
       "10                     0                    0                    0   \n",
       "11                     0                    0                    0   \n",
       "12                     1                    0                    0   \n",
       "13                     0                    0                    0   \n",
       "14                     1                    0                    0   \n",
       "15                     0                    1                    0   \n",
       "16                     0                    0                    0   \n",
       "17                     0                    0                    0   \n",
       "18                     0                    0                    1   \n",
       "19                     0                    0                    1   \n",
       "20                     0                    0                    0   \n",
       "21                     0                    0                    0   \n",
       "22                     1                    0                    0   \n",
       "23                     0                    0                    0   \n",
       "24                     1                    0                    0   \n",
       "25                     0                    0                    0   \n",
       "26                     1                    0                    0   \n",
       "27                     0                    0                    0   \n",
       "28                     0                    0                    0   \n",
       "29                     0                    0                    0   \n",
       "..                   ...                  ...                  ...   \n",
       "388                    0                    0                    0   \n",
       "389                    0                    0                    0   \n",
       "390                    0                    0                    0   \n",
       "391                    1                    0                    0   \n",
       "392                    0                    0                    0   \n",
       "393                    0                    0                    0   \n",
       "394                    0                    0                    0   \n",
       "395                    1                    0                    0   \n",
       "396                    0                    0                    0   \n",
       "397                    1                    0                    0   \n",
       "398                    0                    0                    0   \n",
       "399                    0                    0                    0   \n",
       "400                    1                    0                    0   \n",
       "401                    0                    0                    0   \n",
       "402                    1                    0                    0   \n",
       "403                    0                    0                    0   \n",
       "404                    0                    0                    0   \n",
       "405                    0                    0                    0   \n",
       "406                    0                    0                    0   \n",
       "407                    0                    0                    0   \n",
       "408                    0                    0                    1   \n",
       "409                    0                    0                    1   \n",
       "410                    0                    0                    1   \n",
       "411                    1                    0                    0   \n",
       "412                    0                    0                    1   \n",
       "413                    0                    0                    0   \n",
       "414                    1                    0                    0   \n",
       "415                    0                    0                    0   \n",
       "416                    0                    0                    0   \n",
       "417                    0                    0                    0   \n",
       "\n",
       "     Sex_Pclass_male_1  Sex_Pclass_male_2  Sex_Pclass_male_3  Age_scaled  \\\n",
       "0                    0                  0                  1    0.307495   \n",
       "1                    0                  0                  0    1.256225   \n",
       "2                    0                  1                  0    2.394702   \n",
       "3                    0                  0                  1   -0.261743   \n",
       "4                    0                  0                  0   -0.641235   \n",
       "5                    0                  0                  1   -1.248423   \n",
       "6                    0                  0                  0   -0.034048   \n",
       "7                    0                  1                  0   -0.337642   \n",
       "8                    0                  0                  0   -0.944829   \n",
       "9                    0                  0                  1   -0.717134   \n",
       "10                   0                  0                  1   -0.189852   \n",
       "11                   1                  0                  0    1.180327   \n",
       "12                   0                  0                  0   -0.565337   \n",
       "13                   0                  1                  0    2.470600   \n",
       "14                   0                  0                  0    1.256225   \n",
       "15                   0                  0                  0   -0.489439   \n",
       "16                   0                  1                  0    0.345444   \n",
       "17                   0                  0                  1   -0.717134   \n",
       "18                   0                  0                  0   -0.261743   \n",
       "19                   0                  0                  0    1.104428   \n",
       "20                   1                  0                  0    1.863413   \n",
       "21                   0                  0                  1   -1.627915   \n",
       "22                   0                  0                  0    1.659573   \n",
       "23                   1                  0                  0   -0.717134   \n",
       "24                   0                  0                  0    1.332124   \n",
       "25                   0                  0                  1    1.483921   \n",
       "26                   0                  0                  0   -0.641235   \n",
       "27                   0                  0                  1   -0.603286   \n",
       "28                   1                  0                  0    0.800835   \n",
       "29                   0                  0                  1   -0.528784   \n",
       "..                 ...                ...                ...         ...   \n",
       "388                  0                  0                  1   -0.717134   \n",
       "389                  0                  0                  1   -1.855610   \n",
       "390                  1                  0                  0   -0.565337   \n",
       "391                  0                  0                  0    1.559819   \n",
       "392                  0                  0                  1   -1.324321   \n",
       "393                  0                  1                  0    1.256225   \n",
       "394                  0                  0                  1   -0.109946   \n",
       "395                  0                  0                  0   -0.944829   \n",
       "396                  0                  0                  1   -0.489439   \n",
       "397                  0                  0                  0    1.332124   \n",
       "398                  0                  0                  1   -0.641235   \n",
       "399                  0                  0                  1    0.041850   \n",
       "400                  0                  0                  0   -0.034048   \n",
       "401                  0                  1                  0    0.573139   \n",
       "402                  0                  0                  0   -0.641235   \n",
       "403                  1                  0                  0   -1.020728   \n",
       "404                  1                  0                  0    0.952632   \n",
       "405                  0                  1                  0   -0.793032   \n",
       "406                  0                  1                  0   -0.565337   \n",
       "407                  1                  0                  0    1.483921   \n",
       "408                  0                  0                  0   -0.800957   \n",
       "409                  0                  0                  0   -2.083305   \n",
       "410                  0                  0                  0    0.367897   \n",
       "411                  0                  0                  0    0.497241   \n",
       "412                  0                  0                  0   -0.185845   \n",
       "413                  0                  0                  1    0.019516   \n",
       "414                  0                  0                  0    0.649038   \n",
       "415                  0                  0                  1    0.611089   \n",
       "416                  0                  0                  1    0.019516   \n",
       "417                  0                  0                  1   -0.358157   \n",
       "\n",
       "     Fare_scaled  \n",
       "0      -0.496637  \n",
       "1      -0.511497  \n",
       "2      -0.463335  \n",
       "3      -0.481704  \n",
       "4      -0.416740  \n",
       "5      -0.471623  \n",
       "6      -0.500221  \n",
       "7      -0.117238  \n",
       "8      -0.507390  \n",
       "9      -0.204154  \n",
       "10     -0.495444  \n",
       "11     -0.171000  \n",
       "12      0.837349  \n",
       "13     -0.171000  \n",
       "14      0.459367  \n",
       "15     -0.140162  \n",
       "16     -0.415620  \n",
       "17     -0.507465  \n",
       "18     -0.494920  \n",
       "19     -0.507465  \n",
       "20      0.427557  \n",
       "21     -0.580120  \n",
       "22     -0.069151  \n",
       "23      0.463026  \n",
       "24      4.065049  \n",
       "25     -0.377090  \n",
       "26      0.473779  \n",
       "27     -0.507465  \n",
       "28     -0.090356  \n",
       "29     -0.248433  \n",
       "..           ...  \n",
       "388    -0.498056  \n",
       "389    -0.259261  \n",
       "390     1.038659  \n",
       "391     0.069140  \n",
       "392    -0.274045  \n",
       "393    -0.448774  \n",
       "394    -0.242236  \n",
       "395     0.438310  \n",
       "396    -0.507017  \n",
       "397     0.782391  \n",
       "398    -0.497608  \n",
       "399    -0.498356  \n",
       "400     2.317614  \n",
       "401    -0.260605  \n",
       "402     0.427557  \n",
       "403     0.207130  \n",
       "404    -0.140162  \n",
       "405    -0.388515  \n",
       "406    -0.448774  \n",
       "407     3.153324  \n",
       "408    -0.498580  \n",
       "409    -0.390083  \n",
       "410    -0.498056  \n",
       "411     0.975936  \n",
       "412    -0.497608  \n",
       "413    -0.492680  \n",
       "414     1.314641  \n",
       "415    -0.507017  \n",
       "416    -0.492680  \n",
       "417    -0.236263  \n",
       "\n",
       "[418 rows x 23 columns]"
      ]
     },
     "execution_count": 543,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data_test = pd.read_csv(\"test.csv\")\n",
    "data_test.loc[ (data_test.Fare.isnull()), 'Fare' ] = 0\n",
    "data_test['Sex_Pclass'] = data_test.Sex + \"_\" + data_test.Pclass.map(str)\n",
    "# 接着我们对test_data做和train_data中一致的特征变换\n",
    "# 首先用同样的RandomForestRegressor模型填上丢失的年龄\n",
    "tmp_df = data_test[['Age','Fare', 'Parch', 'SibSp', 'Pclass']]\n",
    "null_age = tmp_df[data_test.Age.isnull()].as_matrix()\n",
    "# 根据特征属性X预测年龄并补上\n",
    "X = null_age[:, 1:]\n",
    "predictedAges = rfr.predict(X)\n",
    "data_test.loc[ (data_test.Age.isnull()), 'Age' ] = predictedAges\n",
    "\n",
    "data_test = set_Cabin_type(data_test)\n",
    "dummies_Cabin = pd.get_dummies(data_test['Cabin'], prefix= 'Cabin')\n",
    "dummies_Embarked = pd.get_dummies(data_test['Embarked'], prefix= 'Embarked')\n",
    "dummies_Sex = pd.get_dummies(data_test['Sex'], prefix= 'Sex')\n",
    "dummies_Pclass = pd.get_dummies(data_test['Pclass'], prefix= 'Pclass')\n",
    "dummies_Sex_Pclass = pd.get_dummies(data_test['Sex_Pclass'], prefix= 'Sex_Pclass')\n",
    "\n",
    "\n",
    "df_test = pd.concat([data_test, dummies_Cabin, dummies_Embarked, dummies_Sex, dummies_Pclass, dummies_Sex_Pclass], axis=1)\n",
    "df_test.drop(['Pclass', 'Name', 'Sex', 'Ticket', 'Cabin', 'Embarked', 'Sex_Pclass'], axis=1, inplace=True)\n",
    "df_test['Age_scaled'] = scaler.fit_transform(df_test['Age'], age_scale_param)\n",
    "df_test['Fare_scaled'] = scaler.fit_transform(df_test['Fare'], fare_scale_param)\n",
    "df_test"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 545,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "test = df_test.filter(regex='Age_.*|SibSp|Parch|Fare_.*|Cabin_.*|Embarked_.*|Sex_.*|Pclass.*')\n",
    "predictions = clf.predict(test)\n",
    "result = pd.DataFrame({'PassengerId':data_test['PassengerId'].as_matrix(), 'Survived':predictions.astype(np.int32)})\n",
    "result.to_csv(\"logistic_regression_predictions2.csv\", index=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "<font color=red>一般做到后期，咱们要进行模型优化的方法就是模型融合啦<br>\n",
    "先解释解释啥叫模型融合哈，我们还是举几个例子直观理解一下好了。<br><br>\n",
    "\n",
    "大家都看过知识问答的综艺节目中，求助现场观众时候，让观众投票，最高的答案作为自己的答案的形式吧，每个人都有一个判定结果，最后我们相信答案在大多数人手里。<br>\n",
    "\n",
    "再通俗一点举个例子。你和你班某数学大神关系好，每次作业都『模仿』他的，于是绝大多数情况下，他做对了，你也对了。突然某一天大神脑子犯糊涂，手一抖，写错了一个数，于是…恩，你也只能跟着错了。 <br>\n",
    "我们再来看看另外一个场景，你和你班5个数学大神关系都很好，每次都把他们作业拿过来，对比一下，再『自己做』，那你想想，如果哪天某大神犯糊涂了，写错了，but另外四个写对了啊，那你肯定相信另外4人的是正确答案吧？<br>\n",
    "\n",
    "最简单的模型融合大概就是这么个意思，比如分类问题，当我们手头上有一堆在同一份数据集上训练得到的分类器(比如logistic regression，SVM，KNN，random forest，神经网络)，那我们让他们都分别去做判定，然后对结果做投票统计，取票数最多的结果为最后结果。<br>\n",
    "\n",
    "bingo，问题就这么完美的解决了。<br>\n",
    "\n",
    "模型融合可以比较好地缓解，训练过程中产生的过拟合问题，从而对于结果的准确度提升有一定的帮助。<br>\n",
    "\n",
    "话说回来，回到我们现在的问题。你看，我们现在只讲了logistic regression，如果我们还想用这个融合思想去提高我们的结果，我们该怎么做呢？<br>\n",
    "\n",
    "既然这个时候模型没得选，那咱们就在数据上动动手脚咯。大家想想，如果模型出现过拟合现在，一定是在我们的训练上出现拟合过度造成的对吧。<br>\n",
    "\n",
    "那我们干脆就不要用全部的训练集，每次取训练集的一个subset，做训练，这样，我们虽然用的是同一个机器学习算法，但是得到的模型却是不一样的；同时，因为我们没有任何一份子数据集是全的，因此即使出现过拟合，也是在子训练集上出现过拟合，而不是全体数据上，这样做一个融合，可能对最后的结果有一定的帮助。对，这就是常用的Bagging。<br>\n",
    "\n",
    "我们用scikit-learn里面的Bagging来完成上面的思路，过程非常简单。代码如下：<br><br><font>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.ensemble import BaggingRegressor\n",
    "\n",
    "train_df = df.filter(regex='Survived|Age_.*|SibSp|Parch|Fare_.*|Cabin_.*|Embarked_.*|Sex_.*|Pclass.*|Mother|Child|Family|Title')\n",
    "train_np = train_df.as_matrix()\n",
    "\n",
    "# y即Survival结果\n",
    "y = train_np[:, 0]\n",
    "\n",
    "# X即特征属性值\n",
    "X = train_np[:, 1:]\n",
    "\n",
    "# fit到BaggingRegressor之中\n",
    "clf = linear_model.LogisticRegression(C=1.0, penalty='l1', tol=1e-6)\n",
    "bagging_clf = BaggingRegressor(clf, n_estimators=10, max_samples=0.8, max_features=1.0, bootstrap=True, bootstrap_features=False, n_jobs=-1)\n",
    "bagging_clf.fit(X, y)\n",
    "\n",
    "test = df_test.filter(regex='Age_.*|SibSp|Parch|Fare_.*|Cabin_.*|Embarked_.*|Sex_.*|Pclass.*|Mother|Child|Family|Title')\n",
    "predictions = bagging_clf.predict(test)\n",
    "result = pd.DataFrame({'PassengerId':data_test['PassengerId'].as_matrix(), 'Survived':predictions.astype(np.int32)})\n",
    "result.to_csv(\"/Users/MLS/Downloads/logistic_regression_predictions2.csv\", index=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<font color=red>下面是咱们用别的分类器解决这个问题的代码：</font>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 581,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/Users/MLS/Downloads/train.csv 0\n",
      "(891,) (891, 12)\n",
      "Fitting 10 folds for each of 1 candidates, totalling 10 fits\n",
      "[CV]  ................................................................\n",
      "[CV] ....................................... , score=0.860140 -   0.4s\n",
      "[CV]  ................................................................\n",
      "[CV] ....................................... , score=0.832168 -   0.4s\n",
      "[CV]  ................................................................\n",
      "[CV] ....................................... , score=0.818182 -   0.4s\n",
      "[CV]  ................................................................\n",
      "[CV] ....................................... , score=0.839161 -   0.4s\n",
      "[CV]  ................................................................\n",
      "[CV] ....................................... , score=0.811189 -   0.5s\n",
      "[CV]  ................................................................\n",
      "[CV] ....................................... , score=0.874126 -   0.4s\n",
      "[CV]  ................................................................\n",
      "[CV] ....................................... , score=0.811189 -   0.4s\n",
      "[CV]  ................................................................\n",
      "[CV] ....................................... , score=0.783217 -   0.4s\n",
      "[CV]  ................................................................\n",
      "[CV] ....................................... , score=0.825175 -   0.4s\n",
      "[CV]  ................................................................\n",
      "[CV] ....................................... , score=0.839161 -   0.4s"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done   1 jobs       | elapsed:    0.4s\n",
      "[Parallel(n_jobs=1)]: Done  10 out of  10 | elapsed:    4.1s finished\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Best score: 0.829\n",
      "Pipeline(steps=[('clf', RandomForestClassifier(bootstrap=False, class_weight=None,\n",
      "            criterion='entropy', max_depth=5, max_features='auto',\n",
      "            max_leaf_nodes=None, min_samples_leaf=1, min_samples_split=1,\n",
      "            min_weight_fraction_leaf=0.0, n_estimators=500, n_jobs=1,\n",
      "            oob_score=False, random_state=0, verbose=0, warm_start=False))])\n",
      "Model with rank: 1\n",
      "Mean validation score: 0.829 (std: 0.025)\n",
      "Parameters: {}\n",
      "\n",
      "-----grid search end------------\n",
      "on all train set\n",
      "0.826038159371 [ 0.81144781  0.83501684  0.83164983]\n",
      "on test set\n",
      "0.782203389831 [ 0.76666667  0.78333333  0.79661017]\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.86      0.90      0.88       439\n",
      "        1.0       0.83      0.75      0.79       273\n",
      "\n",
      "avg / total       0.85      0.85      0.85       712\n",
      "\n",
      "test data\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.86      0.87      0.86       110\n",
      "        1.0       0.79      0.77      0.78        69\n",
      "\n",
      "avg / total       0.83      0.83      0.83       179\n",
      "\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "['/Users/MLS/Downloads/model-rf.pkl',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_01.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_02.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_03.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_04.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_05.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_06.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_07.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_08.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_09.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_10.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_11.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_12.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_13.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_14.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_15.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_16.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_17.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_18.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_19.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_20.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_21.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_22.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_23.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_24.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_25.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_26.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_27.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_28.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_29.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_30.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_31.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_32.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_33.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_34.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_35.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_36.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_37.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_38.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_39.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_40.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_41.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_42.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_43.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_44.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_45.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_46.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_47.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_48.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_49.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_50.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_51.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_52.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_53.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_54.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_55.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_56.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_57.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_58.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_59.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_60.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_61.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_62.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_63.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_64.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_65.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_66.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_67.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_68.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_69.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_70.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_71.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_72.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_73.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_74.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_75.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_76.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_77.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_78.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_79.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_80.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_81.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_82.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_83.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_84.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_85.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_86.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_87.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_88.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_89.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_90.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_91.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_92.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_93.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_94.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_95.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_96.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_97.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_98.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_99.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_100.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_101.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_102.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_103.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_104.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_105.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_106.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_107.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_108.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_109.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_110.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_111.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_112.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_113.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_114.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_115.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_116.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_117.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_118.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_119.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_120.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_121.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_122.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_123.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_124.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_125.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_126.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_127.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_128.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_129.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_130.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_131.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_132.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_133.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_134.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_135.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_136.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_137.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_138.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_139.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_140.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_141.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_142.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_143.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_144.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_145.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_146.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_147.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_148.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_149.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_150.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_151.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_152.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_153.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_154.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_155.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_156.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_157.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_158.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_159.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_160.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_161.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_162.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_163.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_164.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_165.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_166.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_167.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_168.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_169.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_170.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_171.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_172.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_173.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_174.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_175.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_176.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_177.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_178.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_179.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_180.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_181.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_182.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_183.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_184.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_185.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_186.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_187.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_188.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_189.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_190.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_191.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_192.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_193.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_194.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_195.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_196.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_197.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_198.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_199.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_200.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_201.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_202.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_203.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_204.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_205.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_206.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_207.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_208.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_209.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_210.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_211.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_212.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_213.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_214.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_215.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_216.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_217.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_218.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_219.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_220.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_221.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_222.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_223.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_224.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_225.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_226.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_227.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_228.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_229.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_230.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_231.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_232.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_233.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_234.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_235.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_236.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_237.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_238.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_239.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_240.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_241.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_242.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_243.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_244.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_245.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_246.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_247.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_248.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_249.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_250.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_251.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_252.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_253.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_254.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_255.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_256.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_257.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_258.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_259.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_260.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_261.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_262.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_263.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_264.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_265.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_266.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_267.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_268.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_269.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_270.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_271.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_272.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_273.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_274.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_275.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_276.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_277.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_278.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_279.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_280.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_281.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_282.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_283.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_284.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_285.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_286.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_287.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_288.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_289.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_290.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_291.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_292.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_293.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_294.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_295.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_296.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_297.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_298.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_299.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_300.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_301.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_302.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_303.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_304.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_305.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_306.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_307.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_308.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_309.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_310.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_311.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_312.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_313.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_314.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_315.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_316.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_317.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_318.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_319.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_320.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_321.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_322.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_323.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_324.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_325.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_326.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_327.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_328.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_329.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_330.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_331.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_332.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_333.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_334.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_335.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_336.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_337.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_338.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_339.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_340.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_341.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_342.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_343.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_344.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_345.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_346.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_347.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_348.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_349.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_350.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_351.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_352.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_353.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_354.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_355.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_356.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_357.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_358.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_359.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_360.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_361.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_362.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_363.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_364.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_365.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_366.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_367.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_368.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_369.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_370.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_371.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_372.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_373.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_374.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_375.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_376.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_377.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_378.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_379.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_380.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_381.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_382.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_383.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_384.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_385.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_386.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_387.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_388.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_389.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_390.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_391.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_392.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_393.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_394.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_395.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_396.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_397.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_398.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_399.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_400.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_401.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_402.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_403.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_404.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_405.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_406.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_407.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_408.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_409.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_410.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_411.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_412.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_413.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_414.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_415.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_416.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_417.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_418.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_419.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_420.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_421.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_422.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_423.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_424.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_425.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_426.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_427.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_428.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_429.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_430.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_431.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_432.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_433.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_434.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_435.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_436.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_437.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_438.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_439.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_440.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_441.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_442.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_443.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_444.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_445.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_446.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_447.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_448.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_449.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_450.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_451.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_452.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_453.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_454.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_455.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_456.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_457.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_458.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_459.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_460.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_461.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_462.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_463.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_464.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_465.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_466.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_467.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_468.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_469.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_470.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_471.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_472.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_473.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_474.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_475.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_476.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_477.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_478.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_479.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_480.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_481.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_482.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_483.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_484.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_485.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_486.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_487.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_488.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_489.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_490.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_491.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_492.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_493.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_494.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_495.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_496.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_497.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_498.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_499.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_500.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_501.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_502.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_503.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_504.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_505.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_506.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_507.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_508.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_509.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_510.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_511.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_512.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_513.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_514.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_515.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_516.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_517.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_518.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_519.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_520.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_521.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_522.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_523.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_524.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_525.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_526.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_527.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_528.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_529.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_530.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_531.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_532.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_533.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_534.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_535.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_536.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_537.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_538.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_539.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_540.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_541.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_542.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_543.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_544.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_545.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_546.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_547.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_548.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_549.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_550.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_551.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_552.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_553.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_554.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_555.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_556.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_557.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_558.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_559.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_560.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_561.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_562.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_563.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_564.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_565.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_566.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_567.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_568.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_569.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_570.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_571.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_572.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_573.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_574.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_575.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_576.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_577.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_578.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_579.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_580.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_581.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_582.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_583.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_584.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_585.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_586.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_587.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_588.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_589.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_590.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_591.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_592.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_593.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_594.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_595.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_596.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_597.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_598.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_599.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_600.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_601.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_602.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_603.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_604.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_605.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_606.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_607.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_608.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_609.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_610.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_611.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_612.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_613.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_614.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_615.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_616.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_617.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_618.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_619.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_620.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_621.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_622.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_623.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_624.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_625.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_626.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_627.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_628.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_629.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_630.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_631.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_632.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_633.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_634.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_635.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_636.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_637.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_638.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_639.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_640.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_641.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_642.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_643.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_644.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_645.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_646.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_647.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_648.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_649.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_650.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_651.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_652.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_653.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_654.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_655.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_656.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_657.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_658.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_659.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_660.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_661.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_662.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_663.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_664.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_665.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_666.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_667.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_668.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_669.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_670.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_671.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_672.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_673.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_674.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_675.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_676.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_677.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_678.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_679.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_680.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_681.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_682.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_683.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_684.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_685.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_686.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_687.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_688.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_689.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_690.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_691.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_692.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_693.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_694.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_695.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_696.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_697.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_698.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_699.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_700.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_701.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_702.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_703.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_704.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_705.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_706.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_707.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_708.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_709.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_710.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_711.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_712.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_713.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_714.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_715.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_716.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_717.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_718.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_719.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_720.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_721.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_722.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_723.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_724.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_725.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_726.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_727.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_728.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_729.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_730.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_731.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_732.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_733.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_734.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_735.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_736.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_737.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_738.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_739.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_740.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_741.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_742.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_743.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_744.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_745.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_746.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_747.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_748.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_749.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_750.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_751.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_752.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_753.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_754.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_755.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_756.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_757.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_758.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_759.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_760.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_761.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_762.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_763.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_764.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_765.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_766.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_767.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_768.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_769.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_770.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_771.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_772.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_773.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_774.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_775.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_776.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_777.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_778.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_779.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_780.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_781.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_782.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_783.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_784.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_785.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_786.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_787.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_788.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_789.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_790.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_791.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_792.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_793.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_794.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_795.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_796.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_797.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_798.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_799.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_800.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_801.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_802.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_803.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_804.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_805.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_806.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_807.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_808.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_809.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_810.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_811.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_812.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_813.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_814.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_815.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_816.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_817.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_818.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_819.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_820.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_821.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_822.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_823.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_824.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_825.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_826.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_827.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_828.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_829.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_830.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_831.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_832.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_833.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_834.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_835.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_836.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_837.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_838.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_839.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_840.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_841.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_842.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_843.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_844.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_845.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_846.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_847.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_848.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_849.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_850.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_851.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_852.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_853.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_854.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_855.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_856.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_857.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_858.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_859.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_860.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_861.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_862.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_863.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_864.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_865.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_866.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_867.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_868.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_869.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_870.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_871.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_872.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_873.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_874.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_875.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_876.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_877.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_878.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_879.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_880.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_881.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_882.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_883.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_884.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_885.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_886.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_887.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_888.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_889.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_890.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_891.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_892.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_893.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_894.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_895.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_896.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_897.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_898.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_899.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_900.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_901.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_902.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_903.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_904.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_905.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_906.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_907.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_908.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_909.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_910.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_911.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_912.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_913.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_914.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_915.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_916.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_917.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_918.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_919.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_920.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_921.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_922.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_923.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_924.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_925.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_926.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_927.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_928.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_929.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_930.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_931.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_932.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_933.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_934.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_935.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_936.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_937.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_938.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_939.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_940.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_941.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_942.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_943.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_944.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_945.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_946.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_947.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_948.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_949.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_950.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_951.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_952.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_953.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_954.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_955.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_956.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_957.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_958.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_959.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_960.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_961.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_962.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_963.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_964.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_965.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_966.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_967.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_968.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_969.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_970.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_971.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_972.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_973.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_974.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_975.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_976.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_977.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_978.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_979.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_980.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_981.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_982.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_983.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_984.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_985.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_986.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_987.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_988.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_989.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_990.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_991.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_992.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_993.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_994.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_995.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_996.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_997.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_998.npy',\n",
       " '/Users/MLS/Downloads/model-rf.pkl_999.npy',\n",
       " ...]"
      ]
     },
     "execution_count": 581,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "from pandas import  DataFrame\n",
    "from patsy import dmatrices\n",
    "import string\n",
    "from operator import itemgetter\n",
    "import json\n",
    "from sklearn.ensemble import RandomForestClassifier\n",
    "from sklearn.cross_validation import cross_val_score\n",
    "from sklearn.pipeline import Pipeline\n",
    "from sklearn.grid_search import GridSearchCV\n",
    "from sklearn.cross_validation import train_test_split,StratifiedShuffleSplit,StratifiedKFold\n",
    "from sklearn import preprocessing\n",
    "from sklearn.metrics import classification_report\n",
    "from sklearn.externals import joblib\n",
    "\n",
    "##Read configuration parameters\n",
    "\n",
    "train_file=\"train.csv\"\n",
    "MODEL_PATH=\"./\"\n",
    "test_file=\"test.csv\"\n",
    "SUBMISSION_PATH=\"./\"\n",
    "seed= 0\n",
    "\n",
    "print train_file,seed\n",
    "\n",
    "# 输出得分\n",
    "def report(grid_scores, n_top=3):\n",
    "    top_scores = sorted(grid_scores, key=itemgetter(1), reverse=True)[:n_top]\n",
    "    for i, score in enumerate(top_scores):\n",
    "        print(\"Model with rank: {0}\".format(i + 1))\n",
    "        print(\"Mean validation score: {0:.3f} (std: {1:.3f})\".format(\n",
    "              score.mean_validation_score,\n",
    "              np.std(score.cv_validation_scores)))\n",
    "        print(\"Parameters: {0}\".format(score.parameters))\n",
    "        print(\"\")\n",
    "\n",
    "#清理和处理数据\n",
    "def substrings_in_string(big_string, substrings):\n",
    "    for substring in substrings:\n",
    "        if string.find(big_string, substring) != -1:\n",
    "            return substring\n",
    "    print big_string\n",
    "    return np.nan\n",
    "\n",
    "le = preprocessing.LabelEncoder()\n",
    "enc=preprocessing.OneHotEncoder()\n",
    "\n",
    "def clean_and_munge_data(df):\n",
    "    #处理缺省值\n",
    "    df.Fare = df.Fare.map(lambda x: np.nan if x==0 else x)\n",
    "    #处理一下名字，生成Title字段\n",
    "    title_list=['Mrs', 'Mr', 'Master', 'Miss', 'Major', 'Rev',\n",
    "                'Dr', 'Ms', 'Mlle','Col', 'Capt', 'Mme', 'Countess',\n",
    "                'Don', 'Jonkheer']\n",
    "    df['Title']=df['Name'].map(lambda x: substrings_in_string(x, title_list))\n",
    "\n",
    "    #处理特殊的称呼，全处理成mr, mrs, miss, master\n",
    "    def replace_titles(x):\n",
    "        title=x['Title']\n",
    "        if title in ['Mr','Don', 'Major', 'Capt', 'Jonkheer', 'Rev', 'Col']:\n",
    "            return 'Mr'\n",
    "        elif title in ['Master']:\n",
    "            return 'Master'\n",
    "        elif title in ['Countess', 'Mme','Mrs']:\n",
    "            return 'Mrs'\n",
    "        elif title in ['Mlle', 'Ms','Miss']:\n",
    "            return 'Miss'\n",
    "        elif title =='Dr':\n",
    "            if x['Sex']=='Male':\n",
    "                return 'Mr'\n",
    "            else:\n",
    "                return 'Mrs'\n",
    "        elif title =='':\n",
    "            if x['Sex']=='Male':\n",
    "                return 'Master'\n",
    "            else:\n",
    "                return 'Miss'\n",
    "        else:\n",
    "            return title\n",
    "\n",
    "    df['Title']=df.apply(replace_titles, axis=1)\n",
    "\n",
    "    #看看家族是否够大，咳咳\n",
    "    df['Family_Size']=df['SibSp']+df['Parch']\n",
    "    df['Family']=df['SibSp']*df['Parch']\n",
    "\n",
    "\n",
    "    df.loc[ (df.Fare.isnull())&(df.Pclass==1),'Fare'] =np.median(df[df['Pclass'] == 1]['Fare'].dropna())\n",
    "    df.loc[ (df.Fare.isnull())&(df.Pclass==2),'Fare'] =np.median( df[df['Pclass'] == 2]['Fare'].dropna())\n",
    "    df.loc[ (df.Fare.isnull())&(df.Pclass==3),'Fare'] = np.median(df[df['Pclass'] == 3]['Fare'].dropna())\n",
    "\n",
    "    df['Gender'] = df['Sex'].map( {'female': 0, 'male': 1} ).astype(int)\n",
    "\n",
    "    df['AgeFill']=df['Age']\n",
    "    mean_ages = np.zeros(4)\n",
    "    mean_ages[0]=np.average(df[df['Title'] == 'Miss']['Age'].dropna())\n",
    "    mean_ages[1]=np.average(df[df['Title'] == 'Mrs']['Age'].dropna())\n",
    "    mean_ages[2]=np.average(df[df['Title'] == 'Mr']['Age'].dropna())\n",
    "    mean_ages[3]=np.average(df[df['Title'] == 'Master']['Age'].dropna())\n",
    "    df.loc[ (df.Age.isnull()) & (df.Title == 'Miss') ,'AgeFill'] = mean_ages[0]\n",
    "    df.loc[ (df.Age.isnull()) & (df.Title == 'Mrs') ,'AgeFill'] = mean_ages[1]\n",
    "    df.loc[ (df.Age.isnull()) & (df.Title == 'Mr') ,'AgeFill'] = mean_ages[2]\n",
    "    df.loc[ (df.Age.isnull()) & (df.Title == 'Master') ,'AgeFill'] = mean_ages[3]\n",
    "\n",
    "    df['AgeCat']=df['AgeFill']\n",
    "    df.loc[ (df.AgeFill<=10) ,'AgeCat'] = 'child'\n",
    "    df.loc[ (df.AgeFill>60),'AgeCat'] = 'aged'\n",
    "    df.loc[ (df.AgeFill>10) & (df.AgeFill <=30) ,'AgeCat'] = 'adult'\n",
    "    df.loc[ (df.AgeFill>30) & (df.AgeFill <=60) ,'AgeCat'] = 'senior'\n",
    "\n",
    "    df.Embarked = df.Embarked.fillna('S')\n",
    "\n",
    "\n",
    "    df.loc[ df.Cabin.isnull()==True,'Cabin'] = 0.5\n",
    "    df.loc[ df.Cabin.isnull()==False,'Cabin'] = 1.5\n",
    "\n",
    "    df['Fare_Per_Person']=df['Fare']/(df['Family_Size']+1)\n",
    "\n",
    "    #Age times class\n",
    "\n",
    "    df['AgeClass']=df['AgeFill']*df['Pclass']\n",
    "    df['ClassFare']=df['Pclass']*df['Fare_Per_Person']\n",
    "\n",
    "\n",
    "    df['HighLow']=df['Pclass']\n",
    "    df.loc[ (df.Fare_Per_Person<8) ,'HighLow'] = 'Low'\n",
    "    df.loc[ (df.Fare_Per_Person>=8) ,'HighLow'] = 'High'\n",
    "\n",
    "\n",
    "\n",
    "    le.fit(df['Sex'] )\n",
    "    x_sex=le.transform(df['Sex'])\n",
    "    df['Sex']=x_sex.astype(np.float)\n",
    "\n",
    "    le.fit( df['Ticket'])\n",
    "    x_Ticket=le.transform( df['Ticket'])\n",
    "    df['Ticket']=x_Ticket.astype(np.float)\n",
    "\n",
    "    le.fit(df['Title'])\n",
    "    x_title=le.transform(df['Title'])\n",
    "    df['Title'] =x_title.astype(np.float)\n",
    "\n",
    "    le.fit(df['HighLow'])\n",
    "    x_hl=le.transform(df['HighLow'])\n",
    "    df['HighLow']=x_hl.astype(np.float)\n",
    "\n",
    "\n",
    "    le.fit(df['AgeCat'])\n",
    "    x_age=le.transform(df['AgeCat'])\n",
    "    df['AgeCat'] =x_age.astype(np.float)\n",
    "\n",
    "    le.fit(df['Embarked'])\n",
    "    x_emb=le.transform(df['Embarked'])\n",
    "    df['Embarked']=x_emb.astype(np.float)\n",
    "\n",
    "    df = df.drop(['PassengerId','Name','Age','Cabin'], axis=1) #remove Name,Age and PassengerId\n",
    "\n",
    "\n",
    "    return df\n",
    "\n",
    "#读取数据\n",
    "traindf=pd.read_csv(train_file)\n",
    "##清洗数据\n",
    "df=clean_and_munge_data(traindf)\n",
    "########################################formula################################\n",
    " \n",
    "formula_ml='Survived~Pclass+C(Title)+Sex+C(AgeCat)+Fare_Per_Person+Fare+Family_Size' \n",
    "\n",
    "y_train, x_train = dmatrices(formula_ml, data=df, return_type='dataframe')\n",
    "y_train = np.asarray(y_train).ravel()\n",
    "print y_train.shape,x_train.shape\n",
    "\n",
    "##选择训练和测试集\n",
    "X_train, X_test, Y_train, Y_test = train_test_split(x_train, y_train, test_size=0.2,random_state=seed)\n",
    "#初始化分类器\n",
    "clf=RandomForestClassifier(n_estimators=500, criterion='entropy', max_depth=5, min_samples_split=1,\n",
    "  min_samples_leaf=1, max_features='auto',    bootstrap=False, oob_score=False, n_jobs=1, random_state=seed,\n",
    "  verbose=0)\n",
    "\n",
    "###grid search找到最好的参数\n",
    "param_grid = dict( )\n",
    "##创建分类pipeline\n",
    "pipeline=Pipeline([ ('clf',clf) ])\n",
    "grid_search = GridSearchCV(pipeline, param_grid=param_grid, verbose=3,scoring='accuracy',\\\n",
    "cv=StratifiedShuffleSplit(Y_train, n_iter=10, test_size=0.2, train_size=None, indices=None, \\\n",
    "random_state=seed, n_iterations=None)).fit(X_train, Y_train)\n",
    "# 对结果打分\n",
    "print(\"Best score: %0.3f\" % grid_search.best_score_)\n",
    "print(grid_search.best_estimator_)\n",
    "report(grid_search.grid_scores_)\n",
    " \n",
    "print('-----grid search end------------')\n",
    "print ('on all train set')\n",
    "scores = cross_val_score(grid_search.best_estimator_, x_train, y_train,cv=3,scoring='accuracy')\n",
    "print scores.mean(),scores\n",
    "print ('on test set')\n",
    "scores = cross_val_score(grid_search.best_estimator_, X_test, Y_test,cv=3,scoring='accuracy')\n",
    "print scores.mean(),scores\n",
    "\n",
    "# 对结果打分\n",
    "\n",
    "print(classification_report(Y_train, grid_search.best_estimator_.predict(X_train) ))\n",
    "print('test data')\n",
    "print(classification_report(Y_test, grid_search.best_estimator_.predict(X_test) ))\n",
    "\n",
    "model_file=MODEL_PATH+'model-rf.pkl'\n",
    "joblib.dump(grid_search.best_estimator_, model_file)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}