exercises/exercise9/09-exercise-pids2023.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "dc08b1ac",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-525c2fd5f3e93d2b",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "# Exercise sheet 9\n",
    "**Hello everyone!**\n",
    "\n",
    "**Points: 15**\n",
    "\n",
    "Please let us know if you have questions or problems! <br>\n",
    "Contact us during the exercise session, on [Piazza](https://piazza.com/class/leibdr4lk8n3w4), or [via email](https://sada.dmi.unibas.ch/en/teaching/pids23).\n",
    "\n",
    "Please submit this exercise sheet on **GRADESCOPE**.\n",
    "Naming conventions:\n",
    "* Name of notebook: \"09-exercise-pids2023.ipynb\"\n",
    "\n",
    "Your notebook will be automatically graded using gradescope. To get graded, you need to upload the file *09-exercise-pids2023.ipynb* to gradescope. If you are using JupyterHub, make sure you have saved the current notebook, then you can download this file by following: *File*->*Download* and save the file on you computer.\n",
    "\n",
    "The sheet is released before the exercise session on Tuesday. You have until the next exercise session on Tuesday to submit this sheet. Most of the questions of the exercise sheet can be handle during the session.\n",
    "\n",
    "**Handout date**: 02.05.2023 <br>\n",
    "**Submission date**: 09.05.2023 <br>\n",
    "\n",
    "### Please name the variables as suggested! Be careful with the type! Otherwise the grading system will fail you!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "886c9dcf",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-41524afeb7ebe206",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import seaborn as sns\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "from scipy import stats\n",
    "from scipy.stats import norm\n",
    "from nose.tools import assert_is_instance, assert_equal, assert_almost_equal, assert_true"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "784719ac",
   "metadata": {},
   "outputs": [],
   "source": [
    "import otter\n",
    "grader = otter.Notebook()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a85dd7ef",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-4637965bfae8d65f",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "# More uncertainty and confidence intervals!\n",
    "Throughout this notebook, we will steadily increase our understanding of confidence intervals. While there are more mathematical notations than usual, they are included solely to enhance the clarity of the exercises. You won't need to run difficult computations other than empirical mean and empirical standard deviation.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "03e5658a",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-53460d34058f3491",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "# Question 1: Titanic dataset\n",
    "We are working with a dataset containing information about passengers of the sadly famous Titanic. We will see how we can use tools from the lecture to get a sense of what factors are important to increase our probability of survival during a shipwreck.<br>\n",
    "\n",
    "In the first part, we will load the dataset, and plot a few values to see which parameters might be important. In a second part, we will focus on one particular feature and see if it is trustworthy."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e15a2faf",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-626919b1facd1abd",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "### 1a (1 point): \n",
    "We first load the dataset. If you are working locally, you should be able to load the dataset from the csv file named 'train.csv', in the same location as this current file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "d410f78d",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-de32fce7c84d1792",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "data_source = \"https://raw.githubusercontent.com/paulhendricks/titanic/master/inst/data-raw/train.csv\"\n",
    "data = ...\n",
    "data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6d527850",
   "metadata": {
    "deletable": false,
    "editable": false
   },
   "outputs": [],
   "source": [
    "grader.check(\"Question 1a\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6a189967",
   "metadata": {},
   "source": [
    "### 1b (1 point):\n",
    "Create a variable ***data_class_survived*** and ***data_class_not_survived*** that contains only the passengers that survived and the passagengers that didn't survive, respectively. Here, we will only keep the *Pclass* and *Survived* columns from the dataset. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "b6f83d0d",
   "metadata": {},
   "outputs": [],
   "source": [
    "data_class = data[['Survived','Pclass']]\n",
    "data_class_survived = ...\n",
    "data_class_not_survived = ..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8d4d48f5",
   "metadata": {
    "deletable": false,
    "editable": false
   },
   "outputs": [],
   "source": [
    "grader.check(\"Question 1b\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c2944e0d",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-b28895179eafb8c6",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "### 1c (1 point): \n",
    "Make a normalized histogram showing the class (column 'Pclass') of the passengers that survived.<br>\n",
    "On the same plot, display the same histogram for the passengers that did not survive.<br>\n",
    "Hint: you can use plt.hist and the density parameter."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "497dfe05",
   "metadata": {
    "deletable": false,
    "editable": false
   },
   "source": [
    "<!-- BEGIN QUESTION -->\n",
    "\n",
    "<!--\n",
    "BEGIN QUESTION\n",
    "name: Question 1c        # (required) the path to a requirements.txt file\n",
    "manual: true     # whether this is a manually-graded question\n",
    "points: 1      # how many points this question is worth; defaults to 1 internally\n",
    "check_cell: false  # whether to include a check cell after this question (for autograded questions only)\n",
    "-->"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "bce1193e",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-ede3d355a0957fbe",
     "locked": false,
     "schema_version": 3,
     "solution": true,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "..."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "35867cea",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-e44d240b6d693930",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "<!-- END QUESTION -->\n",
    "\n",
    "\n",
    "\n",
    "### 1d (1 point):\n",
    "Similar to 1c), display the 'Fare' of passengers survived in a histogram.<br>\n",
    "On the same plot, display the same histogram for the passengers that did not survive. <br>\n",
    "In order to do so, create data frames for the information in column 'Fare' as you did in 1b.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4147fb40",
   "metadata": {
    "deletable": false,
    "editable": false
   },
   "source": [
    "<!-- BEGIN QUESTION -->\n",
    "\n",
    "<!--\n",
    "BEGIN QUESTION\n",
    "name: Question 1d        # (required) the path to a requirements.txt file\n",
    "manual: true     # whether this is a manually-graded question\n",
    "points: 1      # how many points this question is worth; defaults to 1 internally\n",
    "check_cell: false  # whether to include a check cell after this question (for autograded questions only)\n",
    "-->"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "c4db4800",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-f95caf2cb1886049",
     "locked": false,
     "schema_version": 3,
     "solution": true,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "..."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2d16fd10",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-f5f3401d73531224",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "<!-- END QUESTION -->\n",
    "\n",
    "\n",
    "\n",
    "### 1e (2 points):\n",
    "Create variables ***data_age_survived*** and ***data_age_not_survived*** to extract passengers that survived and passengers that did not.<br>\n",
    "Among the passengers that survived, display ages of the passengers in a histogram. Make sure that the histogram is *normalized*<br>\n",
    "On the same plot, display the same normalized histogram for the passengers that didn't survive.<br>\n",
    "Hint: you can use plt.hist."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "b2c66ad0",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-0da4e66b7921cec4",
     "locked": false,
     "schema_version": 3,
     "solution": true,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "...\n",
    "data_age_survived = ...\n",
    "data_age_not_survived = ...\n",
    "\n",
    "..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8ba1bc48",
   "metadata": {
    "deletable": false,
    "editable": false
   },
   "outputs": [],
   "source": [
    "grader.check(\"Question 1e\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f300b4ec",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-1614b841c240181a",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "### 1f (2 points): \n",
    "Display on the same plot, the column 'Survived' for the female and male passengers. <br>\n",
    "Hint: you can use plt.hist.<br>\n",
    "Among the passengers, compute the proportion of survivors that are women and store it in ***surv_female***.<br>\n",
    "Among the passengers, compute the proportion of survivors that are men and store it in ***surv_male***."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "6c1edb84",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-a66ee7dc2a8b5ed1",
     "locked": false,
     "schema_version": 3,
     "solution": true,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "surv_female = ...\n",
    "surv_male = ...\n",
    "..."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b1d1f952",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-b8d8461a1b1e2d4c",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "## Will your sex increase or decrease your probability of survival?\n",
    "\n",
    "Let $(X_i)_{i\\leq N_X} \\overset{i.i.d.}{\\sim} \\mathrm{Ber}(p_X)$ denote $N_X$ Bernoulli random variables that model the outcome of a female passenger on the Titanic. <br>\n",
    "Let $(Y_i)_{i\\leq N_Y} \\overset{i.i.d.}{\\sim} \\mathrm{Ber}(p_Y)$ denote $N_Y$ Bernoulli random variables that model the outcome of a male passenger on the Titanic.<br>\n",
    "\n",
    "We aim at estimating the probailty of survival $p_X$ and $p_Y$."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5365f120",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-dc490e5fa8151ae5",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "Ideally we would like to get the values $p_X = \\mathbb{E}[X_i]$ and $p_Y = \\mathbb{E}[Y_i]$. Unfortunately, we only have access to the $N_X$ realizations $(X_i)$ and $N_Y$ realization $(Y_i)$. <br>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "26998b58",
   "metadata": {},
   "source": [
    "### 1g (1 point):\n",
    "Use the law of large numbers to create an estimate $\\hat p_X$ of $p_X$ and an estimate $\\hat p_Y$ of $p_Y$. Name these estimators ***pX_hat*** and ***pY_hat***."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "19f55afa",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-f870da6474590bea",
     "locked": false,
     "schema_version": 3,
     "solution": true,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "pX_hat = ...\n",
    "pY_hat = ...\n",
    "print(pX_hat)\n",
    "print(pY_hat)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c0a3162f",
   "metadata": {
    "deletable": false,
    "editable": false
   },
   "outputs": [],
   "source": [
    "grader.check(\"Question 1g\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7bee6a78",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-c0d8be22c9907c69",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "Let's do some math.<br>\n",
    "We have some estimate for $p_X$ and $p_Y$, but we would like to know if we can trust them. <br>\n",
    "\n",
    "Let's first focus on $p_X$. <br>\n",
    "We want to compute the probability $p_{trust X}$ that we can trust our estimate for $p_X$:\n",
    "$$ p_{trust X} = \\mathbb{P}\\left( p_X-0.05 \\leq \\hat X \\leq p_X + 0.05 \\right). $$\n",
    "\n",
    "Based the central limit theorem, we'll make the assumption that $\\hat p_X \\sim \\mathcal{N}(p_X, \\sigma^2/N_X)$.<br>\n",
    "Then, following slides of week 8, we have\n",
    "$$ p_{trust X} = \\mathbb{P}\\left( Z \\leq 0.05\\sqrt{ \\frac{N_X}{p_X(1-p_X)}} \\right)-\\mathbb{P}\\left( Z \\leq -0.05\\sqrt{ \\frac{N_X}{p_X(1-p_X)}} \\right), $$\n",
    "where $Z\\sim \\mathcal{N}(0,1)$. <br>\n",
    "As mentioned during the lecture, we don't know $p_X(1-p_X)$. Let's replace this term by $\\hat p_X(1-\\hat p_X)$.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6478451a",
   "metadata": {},
   "source": [
    "### 1h (2 points):\n",
    "\n",
    "Compute the value \n",
    "$$ p1 = \\mathbb{P}\\left( Z \\leq 0.05\\sqrt{ \\frac{N_X}{\\hat p_X(1-\\hat p_X)}} \\right)$$\n",
    "and \n",
    "$$ p2 = \\mathbb{P}\\left( Z \\leq -0.05\\sqrt{ \\frac{N_X}{\\hat p_X(1-\\hat p_X)}} \\right).$$\n",
    "Store these values in variables named **p1** and **p2**.<br>\n",
    "Hint: you can have access to the cumulative distribvutuion function (cdf) of a Gaussian random variable by using the function *norm* (for normal) from the library *scipy.stats* (already imported)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "4e6e08bd",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-11532a187e7a16cc",
     "locked": false,
     "schema_version": 3,
     "solution": true,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "...\n",
    "p1 = ...\n",
    "p2 = ...\n",
    "print(p1, p2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5da1ecc7",
   "metadata": {
    "deletable": false,
    "editable": false
   },
   "outputs": [],
   "source": [
    "grader.check(\"Question 1h\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c5a139a9",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-97dc9411b2c42168",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "### 1i (1 point): \n",
    "Based on the previous assumptions, what is the probability that $\\hat p_X \\in [p_X - 0.05, p_X+0.05]$? Store your answer in a variable named ***proba_trust_X***."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "7b7d7f1b",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-c5f50be87fabc3a0",
     "locked": false,
     "schema_version": 3,
     "solution": true,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "proba_trust_X = ...\n",
    "print(proba_trust_X)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "95bff4cf",
   "metadata": {
    "deletable": false,
    "editable": false
   },
   "outputs": [],
   "source": [
    "grader.check(\"Question 1i\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4767c32e",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-1559a3e0780d2fad",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "### 1j (1 point): \n",
    "Using a similar approach, compute the probability that $\\hat p_Y \\in [p_Y - 0.05, p_Y+0.05]$. Store your answers in variables named ***p1_j*** and ***p2_j*** and ***proba_trust_Y***. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "f9bd911f",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-b6aeee80244a9dd3",
     "locked": false,
     "schema_version": 3,
     "solution": true,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "...\n",
    "p1_j = ...\n",
    "p2_j = ...\n",
    "proba_trust_Y = ...\n",
    "print(proba_trust_Y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "bc12f1b2",
   "metadata": {
    "deletable": false,
    "editable": false
   },
   "outputs": [],
   "source": [
    "grader.check(\"Question 1j\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0fc0496e",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-d6958f47450d57fb",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "# What about the children?\n",
    "Should we remove the children in the previous analysis about the sex? If we do so, the number of women drop from 314 to 259. And we have seen in the lecture that the lower the number of sample, the worse is the confidence interval. \n",
    "<br>\n",
    "But anyway, let's finish by checking that children have a very high probability of survival."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5428d74f",
   "metadata": {},
   "source": [
    "### 1k (2 points): \n",
    "Compute the probability of survival of a child (age<18) and store it in ***proba_surv_child***."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "0bf39538",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-f85e1d8e3c4f0049",
     "locked": false,
     "schema_version": 3,
     "solution": true,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "proba_surv_child = ...\n",
    "...\n",
    "print(proba_surv_child)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f2daaddb",
   "metadata": {
    "deletable": false,
    "editable": false
   },
   "outputs": [],
   "source": [
    "grader.check(\"Question 1k\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "024381e5",
   "metadata": {
    "deletable": false,
    "editable": false
   },
   "source": [
    "---\n",
    "\n",
    "To double-check your work, the cell below will rerun all of the autograder tests."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e7eba4ab",
   "metadata": {
    "deletable": false,
    "editable": false
   },
   "outputs": [],
   "source": [
    "grader.check_all()"
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Create Assignment",
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}