notebooks/understand_business_data.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Titanic"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The route of the Titanic\n",
    "![](../images/route.png)\n",
    "\n",
    "### The night of 14 April 1912\n",
    "![](../images/iceberg_titanic.jpg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <a href=\"https://en.wikipedia.org/wiki/David_Blair_(mariner)\">David Blair</a> (mariner)\n",
    "\n",
    "Blair’s daughter eventually donated the key to the International Sailors Society, and it was [auctioned off](http://news.bbc.co.uk/2/hi/uk_news/northern_ireland/7008300.stm) in 2007, fetching £90,000 (about $130,000)\n",
    "\n",
    "![](../images/key_to_binoculars.jpg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## [Olympic](https://en.wikipedia.org/wiki/RMS_Olympic), [Titanic](https://en.wikipedia.org/wiki/RMS_Titanic), [Britanic](https://en.wikipedia.org/wiki/HMHS_Britannic) (Gigantic)\n",
    "![](../images/olympic_titanic_gigantic.jpg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Hawke collision (20 September 1911)\n",
    "![](../images/olympic_hawke.jpg)\n",
    "![](../images/riddle_titanic.jpg)\n",
    "\n",
    "## [JP Morgan](https://www.youtube.com/watch?v=Z-Wd-hJw8nk)\n",
    "![](../images/jp_morgan.png)\n",
    "\n",
    "[Did the Titanic Really Sink or was it Olympic?](https://social.shorthand.com/TitanicMystery/jCPyIbzzPVc/did-the-titanic-really-sink-or-was-it-olympic)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There're about **1300 passangers** and **900 crew** in summary were around **2200** people and only [**20 lifeboats**](https://en.wikipedia.org/wiki/Lifeboats_of_the_RMS_Titanic) (1 lifeboat for 65 passengers).\n",
    "That means, that by definition at least a half of the people from Titanic will not be survived.\n",
    "\n",
    "![](../images/life_boats.jpg)\n",
    "![](../images/lifeboats.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](../images/sinking_animation.gif)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <a href=\"https://en.wikipedia.org/wiki/Titanic_Memorial_(Washington,_D.C.)\">Titanic Memorial</a>\n",
    "![](../images/man.jpg)\n",
    "![](../images/man_memorial.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "train (891, 12)\n",
      "test (418, 11)\n"
     ]
    }
   ],
   "source": [
    "df_train = pd.read_csv('../input/train.csv')\n",
    "df_test = pd.read_csv('../input/test.csv')\n",
    "\n",
    "print(\"train\", df_train.shape)\n",
    "print(\"test\", df_test.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Total passengers"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1309"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_train.shape[0] + df_test.shape[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 891 entries, 0 to 890\n",
      "Data columns (total 12 columns):\n",
      "PassengerId    891 non-null int64\n",
      "Survived       891 non-null int64\n",
      "Pclass         891 non-null int64\n",
      "Name           891 non-null object\n",
      "Sex            891 non-null object\n",
      "Age            714 non-null float64\n",
      "SibSp          891 non-null int64\n",
      "Parch          891 non-null int64\n",
      "Ticket         891 non-null object\n",
      "Fare           891 non-null float64\n",
      "Cabin          204 non-null object\n",
      "Embarked       889 non-null object\n",
      "dtypes: float64(2), int64(5), object(5)\n",
      "memory usage: 83.6+ KB\n"
     ]
    }
   ],
   "source": [
    "df_train.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Name</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Ticket</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Embarked</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Braund, Mr. Owen Harris</td>\n",
       "      <td>male</td>\n",
       "      <td>22.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>A/5 21171</td>\n",
       "      <td>7.2500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Cumings, Mrs. John Bradley (Florence Briggs Th...</td>\n",
       "      <td>female</td>\n",
       "      <td>38.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>PC 17599</td>\n",
       "      <td>71.2833</td>\n",
       "      <td>C85</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>Heikkinen, Miss. Laina</td>\n",
       "      <td>female</td>\n",
       "      <td>26.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>STON/O2. 3101282</td>\n",
       "      <td>7.9250</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Futrelle, Mrs. Jacques Heath (Lily May Peel)</td>\n",
       "      <td>female</td>\n",
       "      <td>35.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>113803</td>\n",
       "      <td>53.1000</td>\n",
       "      <td>C123</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Allen, Mr. William Henry</td>\n",
       "      <td>male</td>\n",
       "      <td>35.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>373450</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   PassengerId  Survived  Pclass  \\\n",
       "0            1         0       3   \n",
       "1            2         1       1   \n",
       "2            3         1       3   \n",
       "3            4         1       1   \n",
       "4            5         0       3   \n",
       "\n",
       "                                                Name     Sex   Age  SibSp  \\\n",
       "0                            Braund, Mr. Owen Harris    male  22.0      1   \n",
       "1  Cumings, Mrs. John Bradley (Florence Briggs Th...  female  38.0      1   \n",
       "2                             Heikkinen, Miss. Laina  female  26.0      0   \n",
       "3       Futrelle, Mrs. Jacques Heath (Lily May Peel)  female  35.0      1   \n",
       "4                           Allen, Mr. William Henry    male  35.0      0   \n",
       "\n",
       "   Parch            Ticket     Fare Cabin Embarked  \n",
       "0      0         A/5 21171   7.2500   NaN        S  \n",
       "1      0          PC 17599  71.2833   C85        C  \n",
       "2      0  STON/O2. 3101282   7.9250   NaN        S  \n",
       "3      0            113803  53.1000  C123        S  \n",
       "4      0            373450   8.0500   NaN        S  "
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_train.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Features\n",
    "\n",
    "- **PassengerId** - A numerical id assigned to each passenger\n",
    "- **Survived** - Whether the passenger survived (1), or didn't (0). **Target variable**.\n",
    "- **Pclass** - Ticket class (1st = Upper, 2nd = Middle, 3rd = Lower)\n",
    "- **Name** - The name of the passenger\n",
    "- **Sex** - The gender of the passenger (male or female)\n",
    "- **Age** - The age of the passenger\n",
    "- **SibSp** - # of siblings (brother, sister, stepbrother, stepsister) / spouses (husband, wife [mistresses and fiancés were ignored]) aboard the Titanic\t\n",
    "- **Parch** - # of parents (mother, father) / children (daughter, son, stepdaughter, stepson) aboard the Titanic. *Note*: Some children travelled only with a nanny, therefore parch=0 for them..\n",
    "- **Ticket** - The ticket number of the passenger\n",
    "- **Fare** - How much the passenger paid for the ticker\n",
    "- **Cabin** - Cabin number\n",
    "- **Embarked** - Where the passenger boarded the Titanic Port of Embarkation (C = Cherbourg, Q = Queenstown, S = Southampton)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Pclass & Sex => Survived"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Pclass  Sex     Survived\n",
      "1       female  1           0.968085\n",
      "                0           0.031915\n",
      "        male    0           0.631148\n",
      "                1           0.368852\n",
      "2       female  1           0.921053\n",
      "                0           0.078947\n",
      "        male    0           0.842593\n",
      "                1           0.157407\n",
      "3       female  0           0.500000\n",
      "                1           0.500000\n",
      "        male    0           0.864553\n",
      "                1           0.135447\n",
      "Name: Survived, dtype: float64\n"
     ]
    }
   ],
   "source": [
    "print(df_train.groupby([\"Pclass\", \"Sex\"])[\"Survived\"].value_counts(normalize=True))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### [Isidor](https://en.wikipedia.org/wiki/Isidor_Straus) and [Ida](https://en.wikipedia.org/wiki/Ida_Straus) Straus\n",
    "who refused to board a lifeboat while there were younger people still waiting to board\n",
    "\n",
    "![](../images/Rosalie_Ida_Blun_Straus.jpg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Train vs Test"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train: males\n",
      "              Age        Fare      Pclass       SibSp       Parch\n",
      "count  453.000000  577.000000  577.000000  577.000000  577.000000\n",
      "mean    30.726645   25.523893    2.389948    0.429809    0.235702\n",
      "std     14.678201   43.138263    0.813580    1.061811    0.612294\n",
      "min      0.420000    0.000000    1.000000    0.000000    0.000000\n",
      "25%     21.000000    7.895800    2.000000    0.000000    0.000000\n",
      "50%     29.000000   10.500000    3.000000    0.000000    0.000000\n",
      "75%     39.000000   26.550000    3.000000    0.000000    0.000000\n",
      "max     80.000000  512.329200    3.000000    8.000000    5.000000\n",
      "Test: males\n",
      "              Age        Fare      Pclass       SibSp       Parch\n",
      "count  205.000000  265.000000  266.000000  266.000000  266.000000\n",
      "mean    30.272732   27.527877    2.334586    0.379699    0.274436\n",
      "std     13.389528   41.079423    0.808497    0.843735    0.883745\n",
      "min      0.330000    0.000000    1.000000    0.000000    0.000000\n",
      "25%     22.000000    7.854200    2.000000    0.000000    0.000000\n",
      "50%     27.000000   13.000000    3.000000    0.000000    0.000000\n",
      "75%     40.000000   26.550000    3.000000    1.000000    0.000000\n",
      "max     67.000000  262.375000    3.000000    8.000000    9.000000\n",
      "Train: females\n",
      "              Age        Fare      Pclass       SibSp       Parch\n",
      "count  261.000000  314.000000  314.000000  314.000000  314.000000\n",
      "mean    27.915709   44.479818    2.159236    0.694268    0.649682\n",
      "std     14.110146   57.997698    0.857290    1.156520    1.022846\n",
      "min      0.750000    6.750000    1.000000    0.000000    0.000000\n",
      "25%     18.000000   12.071875    1.000000    0.000000    0.000000\n",
      "50%     27.000000   23.000000    2.000000    0.000000    0.000000\n",
      "75%     37.000000   55.000000    3.000000    1.000000    1.000000\n",
      "max     63.000000  512.329200    3.000000    8.000000    6.000000\n",
      "Test: females\n",
      "              Age        Fare      Pclass       SibSp       Parch\n",
      "count  127.000000  152.000000  152.000000  152.000000  152.000000\n",
      "mean    30.272362   49.747699    2.144737    0.565789    0.598684\n",
      "std     15.428613   73.108716    0.887051    0.974313    1.105434\n",
      "min      0.170000    6.950000    1.000000    0.000000    0.000000\n",
      "25%     20.500000    8.626050    1.000000    0.000000    0.000000\n",
      "50%     27.000000   21.512500    2.000000    0.000000    0.000000\n",
      "75%     38.500000   55.441700    3.000000    1.000000    1.000000\n",
      "max     76.000000  512.329200    3.000000    8.000000    9.000000\n"
     ]
    }
   ],
   "source": [
    "describe_fields = [\"Age\", \"Fare\", \"Pclass\", \"SibSp\", \"Parch\"]\n",
    "\n",
    "print(\"Train: males\")\n",
    "print(df_train[df_train.Sex == \"male\"][describe_fields].describe())\n",
    "\n",
    "print(\"Test: males\")\n",
    "print(df_test[df_test.Sex == \"male\"][describe_fields].describe())\n",
    "\n",
    "print(\"Train: females\")\n",
    "print(df_train[df_train.Sex == \"female\"][describe_fields].describe())\n",
    "\n",
    "print(\"Test: females\")\n",
    "print(df_test[df_test.Sex == \"female\"][describe_fields].describe())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}