Homework/STA130_HW_4.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "42ab3d54",
   "metadata": {},
   "source": [
    "# Coding Homework 4: [Your Name]\n",
    "\n",
    "Go through this notebook, following the instructions! \n",
    "\n",
    "- TAs will mark this assignment by checking ***MarkUs*** autotests; then, manually confirming the correctness of plot in `Q2` and `Q13` (based on correct code usage of `replace=False`, etc.); then, manually reviewing the written response to question `Q6`. TAs may or may not spot check the presence of plots for `Q8` and `Q9` and written answers for `Q8`.\n",
    "     - The autotests for questions `Q8` and `Q9` (and `Q6`) \"automatically fail\" during automated testing with ***MarkUs*** in order to expose example answers for student consideration and review.  These failed ***MarkUs*** \"autotests\" are not counted against the student. \n",
    "\n",
    "\n",
    "> You can add new cells if you need (with the \"+\" button above); but, deleting or reordering cells could very likely cause your notebook to fail ***MarkUs*** autotesting (and you'd have to start over and re-enter your answers into a completely fresh version of the notebook to get things working again...).\n",
    ">\n",
    "> - ***MarkUs*** autotesting works by running your notebook from top to bottom, and if there's an error when doing this (such as a variable is getting called that hasn't yet been defined, etc.) it can cause automated tests to fail; so, make sure your notebook runs from top to bottom without error once you're done (which you can do by restarting the \"Restart & Run All\" from the \"Kernel\" menu).\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c6b08913",
   "metadata": {},
   "source": [
    "## Driving on the \"Right\" Side of the Road\n",
    "Let's consider how cars drive on the right or left side of the road in different countries. World Standards' [list of left driving countries](https://www.worldstandards.eu/cars/list-of-left-driving-countries/) shows that 86 of all 270 countries in the world drive on the left side of the road."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "07099e61",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Run the following code to get the data\n",
    "import pandas as pd # used immediately\n",
    "import numpy as np # to be used eventually!\n",
    "\n",
    "roaddata = pd.DataFrame({\"road_side\": [\"left\"] * 86 + [\"right\"] * (270 - 86)})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "26ecf996",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Have a look at the data!\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4debf2c3",
   "metadata": {},
   "source": [
    "### Q0: Are the observations (rows) in `roaddata` the entire population or a sample from a population?\n",
    "A. The entire population    \n",
    "B. A sample from the population"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d512c280",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Q0: your answer will be tested!\n",
    "Q0 = None # Assign either 'A' or 'B' to `Q0` instead of `None`\n",
    "# E.g., Q0 = 'A'"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "255ce211",
   "metadata": {},
   "source": [
    "### Q1: Select a random sample of 100 *unique* countries from `roaddata` and call it `roaddata_sample`.\n",
    "\n",
    "#### The  `roaddata_sample` object must be a `pd.DataFrame` subset of the contents of `roaddata`.\n",
    "\n",
    "> - Hint 1: Use `roaddata.sample()`; or, [google \"python pandas sample\"](https://letmegooglethat.com/?q=python+pandas+sample) and you'll want that first `pd.sample()` link that comes up.\n",
    ">\n",
    ">\n",
    "> - Hint 2: Do not use `replace=True` because this means after one of the countries is \"randomly sampled\", it is put back (\"replaced\" with `replace=True`) and can be drawn again (twice or even more!) but this is not what we want for this question! "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a29cca1d",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Q1: Your answer will be tested based on the created `roaddata_sample` `pd.DataFrame` object itself:\n",
    "#     you don't need to assign anything to a `Q1` object.\n",
    "np.random.seed(130) # Do NOT change this line: it sets the \"random number generation seed\"\n",
    "# which makes the \"pseudorandomness\" gererated in the code the same every time; which, \n",
    "# makes the code reproducible which ensures that our testing code works properly every time\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cb8f68da",
   "metadata": {},
   "source": [
    "### Q2: Using `roaddata_sample` (as opposed to `roaddata`) create 2000 bootstrap samples and visualize the distribution of proportions of countries where cars drive on the left side of the road observed across these 2000 bootstrap samples. \n",
    "\n",
    "#### Which of the following best describes how the `plotly` histogram of the bootstrap (sampling) distribution of the proportion of countries that drive on the left side compares to the proportion observed in the original sample?\n",
    "\n",
    "A. The proportion observed in the orginal sample is on the left side of the bootstrap (sampling) distribution of proportions, which makes sense since the sample size is less than the population size  \n",
    "\n",
    "B. The proportion observed in the orginal sample is pretty much in the middle of the bootstrap (sampling) distribution of proportions, which makes sense since bootstrap data is resampled versions of the original sample  \n",
    "\n",
    "C. The proportion observed in the orginal sample is on the right side of the bootstrap (sampling) distribution of proportions, which makes sense since there are more bootstrap samples than the orignal sample size  \n",
    "\n",
    "D. None of the above  \n",
    "\n",
    "> - Hint 1: **A bootstrap sample is a resampling of another sample with replacement based on that original sample; so, bootstrapping means creating many bootstrap samples from an original sample that (a) have the same sample size as the original sample and (b) where each observation in the sample can be selected regardless of whether or not it was previously selected (which is known as \"sampling with replacement\" and in `python` is specified in code using `replace=True`)**.\n",
    ">   - If `replace=False` then the resamples (of the same size as the original sample) would just recreate the original sample!\n",
    ">\n",
    ">\n",
    "> - Hint 2: To calculate a proportion you can use coercion to change 'left' and 'right' into `0`'s and `1`'s as you create the average `(roaddata_sample['road_side'] == 'left').mean()` (which is a proportion). \n",
    ">\n",
    ">\n",
    "> - Hint 3: You can mark the proportion in the original sample on your histogram with `fig.add_vline(x=(roaddata_sample['road_side'] == 'left').mean())`.\n",
    ">\n",
    ">\n",
    "> - Hint 4: The population is `roaddata`, whereas `roaddata_sample` is a sample from this population... <br>**Bootstrapping samples from samples... samples from populations are just samples...**\n",
    ">\n",
    ">\n",
    "> - Hint 5: In the multiple choice options above, the reason \"(sampling)\" is emphasized in \"bootstrap (sampling) distribution of proportions\" is because a \"sampling distribution\" refers specifically to the notion of a \"distribution of a statistic\" (here a proportion) created from a sample (which is distinct from the distribution of the population or the sample itself). A \"sampling distribution\" can be derived theoretically in some cases; or, simulated by resampling from the population (if it's available) over and over (and recalculating the statistic in each sample and visualing the distribution of these \"sampled\" statistics). What we're doing in this problem is creating what should should be referred to as a \"bootstrap sampling distribution of proportions\", wherein we presume the sample we have is sufficiently representative of the population to allow us to proceed by using it as such: the sample then \"replaces\" the population with us pretending that it is a suitable stand in placeholder for the population. \n",
    ">   - If you had the whole population then you'd know everything about it exactly without concerning yourself with sampling *per se*; so, for situations we're you don't have the population you can emulate sampling from the population by instead bootstrap sampling from the sample that you do have as if it was the population itself...\n",
    ">\n",
    ">\n",
    "> - Hint 6: `(roaddata_sample['road_side']=='left').mean()` is different than `(roaddata_sample=='left').mean()`...\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1b4624b8",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Complete the code here and add scratch work cells above if needed\n",
    "\n",
    "np.random.seed(130) # Use this line to create reproducible pseudorandomness for your bootstrapping\n",
    "\n",
    "number_of_bootstrap_samples = 2000 \n",
    "# -> use one of these \n",
    "#bootstrapped_proportions_v1 = []\n",
    "#bootstrapped_proportions_v2 = np.array([0.0]*number_of_bootstrap_samples, dtype='float')\n",
    "for i in range(number_of_bootstrap_samples):\n",
    "    pass # complete\n",
    "    # -> corresponding to one of these\n",
    "    # bootstrapped_proportions_v1 += [new_bootstrap_proportion] # Note `.values[0]` in hints 2 and 3...\n",
    "    # bootstrapped_proportions_v2[i] = new_bootstrap_proportion"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3e0391d2",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Add a figure of `bootstrapped_proportions`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6f897cf5",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Q2: your answer will be tested!\n",
    "Q2 = None # Assign either 'A', 'B', 'C', or 'D' to `Q2` instead of `None`\n",
    "# E.g., Q2 = 'A'"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "16c8ec77",
   "metadata": {},
   "source": [
    "### Q3: The proportion of countries where cars drive on the left size of the road in the original `roaddata_sample` sample is $0.3$; whereas, the overall mean of the bootstrapped proportions calculated using the solution code (given in the ***MarkUs*** hints for missed Q2 or Q3) is about $0.298155$; whereas, the actual proportion of all countries in `roaddata` where cars drive on the left side of the road is $0.31851851851851853$...\n",
    "\n",
    "\n",
    "#### Which of the following best explains why these numbers are different?\n",
    "\n",
    "A. The solution code is wrong since it uses pre-converted numeric values and a pre-allocated `np.array` that it indexes values into (as opposed to extending a list through concatenation) which is why it doesn't match the other numbers.  \n",
    "\n",
    "\n",
    "B. The larger the sample size, the closer the mean of the sample would be expected to be to the population mean (since it uses more values from the population); whereas, the overall average of a bootstrap sampling distribution would be expected to be more similar to the mean of the sample being bootstrapped (since it reflects the \"bias\" of that sample by only using values from that sample) and it should be increasingly similar as more and more bootstrap samples are averaged (and the bootstrap sampling distribution more and more accurately centers around the mean of the original sample). \n",
    "\n",
    "\n",
    "C. The overall average of a bootstrap sampling distribution will be closer to the original sample mean when there are fewer bootstrap samples, and closer to the population mean when there are more bootstrap samples.\n",
    "\n",
    "\n",
    "D. None of these are similar because none of them are related to each other.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6614af5e",
   "metadata": {},
   "outputs": [],
   "source": [
    "# cell for some scratch work if needed\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8ccd84c6",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Q3: your answer will be tested!\n",
    "Q3 = None # Assign either 'A', 'B', 'C', or 'D' to `Q3` instead of `None`\n",
    "# E.g., Q3 = 'A'"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0edd86a3",
   "metadata": {},
   "source": [
    "### Q4: Calculate a 90% bootstrap confidence interval for the proportion of countries where cars drive on the left side of the road based on the bootstrap sampling distribution you generated in Q2.\n",
    "\n",
    "#### Give your interval limits as continuous numeric values rounded to two decimal positions.\n",
    "\n",
    "> - Hint 1: Use the `np.quantile` or `np.percentile` functions (which are slightly different... how?).\n",
    "> - Hint 2: **A 95% bootstrap confidence interval is based on the 0.025 and 97.5 percentiles of a bootstrapped sampling distribution...**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "51619289",
   "metadata": {},
   "outputs": [],
   "source": [
    "# cell for scratch work if needed\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3c8a6446",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Q4: your answer will be tested!\n",
    "your_2000_bootstrapped_proportions = None # Assign your list/numpy array variable here instead of `None`\n",
    "your_90percent_confidence_interval_lower_bound = None # use np.round(..., 2)\n",
    "your_90percent_confidence_interval_upper_bound = None # use np.round(..., 2)\n",
    "Q4 = (your_2000_bootstrapped_proportions, \n",
    "      (your_90percent_confidence_interval_lower_bound, your_90percent_confidence_interval_upper_bound)) \n",
    "# Assign your 1000 bootstrapped medians and `tuple` \"interval\" of two-decimal rounded `float`s \n",
    "# as another `tuple` to Q10; e.g., ([bootstrapped_prop_1, ..., bootstrapped_prop_2000], (0.12, 0.34))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5dab67ae",
   "metadata": {},
   "source": [
    "### Q5: Suppose for the sake of this question that you got a bootstrap confidence interval of (27%, 44%). Indicate whether or not each of the following statements is a correct interpretation of the confidence interval.\n",
    "\n",
    "A. We are 90% confident that between 27% and 44% of countries in our `roaddata_sample` sample are countries where cars drive on the left side of the road.   \n",
    "\n",
    "\n",
    "B. There is a 90% chance that between 27% and 44% of all countries in the population drive on the left side of the road; or, equivalently meticulously pedantically restated: at any given time there's a 90% chance that the proportions of countries in the world where cars drive on the left side of the road will be between 27% and 44%.\n",
    "\n",
    "\n",
    "C. If we consider many random samples of 100 countries in the world (such as `roaddata_sample_1`, `roaddata_sample_2`, etc.) and we calculate 90% bootstrap confidence intervals for each sample, approximately 90% of these bootstrap confidence intervals will include the true proportion of countries in the population where cars drive on the left side of the road.   \n",
    "\n",
    "\n",
    "D. The chance that we drew a sample which results in the constructed (27%, 44%) confidence interval and bounds the true proportion of countries in the population where cars drive on the left side is about 90%. \n",
    "\n",
    "\n",
    "E. We are 90% confident that between 27% and 44% of countries in the world are countries where cars drive on the left side of the road.   \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "658f8caf",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Q5: your answer will be tested!\n",
    "Q5a = None # Assign either True or False to each of these\n",
    "Q5b = None\n",
    "Q5c = None\n",
    "Q5d = None\n",
    "Q5e = None\n",
    "Q5 = (Q5a, Q5b, Q5c, Q5d, Q5e)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7648cf60",
   "metadata": {},
   "source": [
    "### Q6: If we want to be more confident that we've captured the actual true proportion of all countries where cars drive on the left side of the road, should we use a wider or narrower confidence interval? If we want to have a more precise, understandable, and interpretable estimate, should we use a wider or narrower confidence interval? Discuss the relevant tradeoffs in explaining your answers.\n",
    "\n",
    "#### Write three or so sentences to answer this question in markdown cell below.\n",
    "- Your TA will manually review your written response."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e48e0ca3",
   "metadata": {},
   "source": [
    "> Answer here..."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "22a1fed5",
   "metadata": {},
   "source": [
    "## Auto Claims\n",
    "The data set `auto_claims.csv` includes claims paid (in USD) by an auto insurance company in a specific year to a sample of auto insurance claimants greater than 60 years of age. In other words, this represents a \"sample\" (the \"original sample\") of car insurance claims paid by the auto insurance company in that year."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "16d73cc4",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Load the `auto_claims.csv` dataset and have a quick look at it.\n",
    "# MarkUs expect you to load the file as 'auto_claims.csv' without any directory pathing\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "34aa2405",
   "metadata": {},
   "source": [
    "### Q7: What is the sample size, median, and standard deviation of paid claims (in the `PAID` column) in `auto_claims.csv`?\n",
    "\n",
    "> Hint: use the `.describe()` method of a `pd.DataFrame` object.\n",
    "#### Give your answers rounded to integer values with `.round()`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fa6de84f",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "64477b7e",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Q7: Your answer will be tested!\n",
    "paid_sample_size = None\n",
    "paid_median = None\n",
    "paid_std = None\n",
    "Q7 = (paid_sample_size, paid_median, paid_std) # Assign a `tuple` of three `int`s to Q7\n",
    "# E.g., `Q7 = (1,2,3)`"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f30819a7",
   "metadata": {},
   "source": [
    "### Q8: Produce `plotly` histogram and boxplot visualizations of paid claims (in the `PAID` column) and comment on the shape, centre, spread, and outliers of this distribution.\n",
    "\n",
    "#### Create the plots and write your answer in the code cell and markdown cells (respectively) below.\n",
    "\n",
    "- You can compare your figure and response to the answer given in the ***MarkUs*** output: the TAs may or may not spot check the work in `Q9` "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f8c48efa",
   "metadata": {},
   "outputs": [],
   "source": [
    "# histogram\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3e6b03e9",
   "metadata": {},
   "outputs": [],
   "source": [
    "# boxplot\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3f811202",
   "metadata": {},
   "source": [
    "> Answer here..."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c32edf01",
   "metadata": {},
   "source": [
    "### Q9: Simulate and visualize the bootstrap (sampling) distribution of the sample median of paid claims (in the `PAID` column) using 1000 bootstrap samples of the auto claims sample (given in the `auto_claims.csv` data).\n",
    "> - Hint 1: Remember that the sample size for each of your 1000 bootstrap samples should match the original sample size\n",
    ">   - and if your bootstrap sample is always the same as the original sample then you're not correctly setting `replace=Ture`.\n",
    ">\n",
    ">\n",
    "> - Hint 2: There is no `.median()` method for an `np.array`, but  there is an `np.median()` function.\n",
    "\n",
    "- You can compare your figure and response to the answer given in the ***MarkUs*** output: the TAs may or may not spot check the work in `Q9`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1e3743df",
   "metadata": {},
   "outputs": [],
   "source": [
    "# cell for coding your bootstrap sampling\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "23ce9faa",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "8c2820fa",
   "metadata": {},
   "source": [
    "### Q10: Create a 95% bootstrap confidence interval for the median of paid claims using the simulation in `Q9`.\n",
    "#### Give your answer as a continuous numeric value rounded to two decimal positions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1a955d4c",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "213e4f97",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Q10: your answer will be tested!\n",
    "your_1000_bootstrapped_medians = None # Assign your list/numpy array variable here instead of `None`\n",
    "your_95percent_confidence_interval_lower_bound = None # use np.round(..., 2)\n",
    "your_95percent_confidence_interval_upper_bound = None # use np.round(..., 2)\n",
    "Q10 = (your_1000_bootstrapped_medians, \n",
    "       (your_95percent_confidence_interval_lower_bound, your_95percent_confidence_interval_upper_bound))\n",
    "# Assign your 1000 bootstrapped medians and `tuple` \"interval\" of two-decimal rounded `float`s \n",
    "# as another `tuple` to Q10; e.g., ([bootstrapped_prop_1, ..., bootstrapped_prop_2000], (0.12, 0.34))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2fbe34ad",
   "metadata": {},
   "source": [
    "### Q11: Load the population defined by `auto_claims_population.csv`, compute the median and standard deviation parameters of the paid claims (`PAID` column) of the population, and select which of the following is the most accurate characterization of the sample in `auto_claims.csv` with respect to this population.\n",
    "\n",
    "A. The sample and population medians are very similar and within \\$10 of each other, whereas the sample and population standard deviations are not; however, the 95\\% bootstrap confidence interval for the sample median captures the true population median nonetheless.  \n",
    "\n",
    "B. The sample and population medians are not within \\$10 of each other, whereas the sample and population standard deviations are; however, the 95\\% bootstrap confidence interval for the sample median captures the true population median nonetheless.  \n",
    "\n",
    "C. The sample and population medians are very similar and within \\$10 of each other, whereas the sample and population standard deviations are not; and, the 95\\% bootstrap confidence interval for the sample median does not capture the true population median as a result.  \n",
    "\n",
    "D. The sample and population medians are not within \\$10 of each other, whereas the sample and population standard deviations are; and, the 95\\% bootstrap confidence interval for the sample median does not capture the true population median as a result.  \n",
    "\n",
    "> Hint: the population and the sample distributions could be visually compared using the following:\n",
    "> ```python\n",
    "> px.box(pd.DataFrame({'x1':pd.read_csv('auto_claims_population.csv').PAID,\n",
    ">                      'x2':pd.read_csv('auto_claims.csv').PAID}))\n",
    "> ```\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3704f98c",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Load the 'auto_claims_population.csv' data\n",
    "# MarkUs expect you to load the file as 'auto_claims_population.csv' without any directory pathing\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3d52738c",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Q11: your answer will be tested!\n",
    "Q11 = None # Assign either 'A', 'B', 'C', or 'D' to `Q11` instead of `None`\n",
    "# E.g., Q11 = 'A'"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ea681f94",
   "metadata": {},
   "source": [
    "### Q12: Which of the following gives the best extended follow-up explanation for the previous problem?\n",
    "\n",
    "A. The sample is a very poor representation of the population as judged by their relative difference in standard deviation, which is why the 95% bootstrap confidence interval was not able to capture the true population median. \n",
    "\n",
    "B. The sample is a very poor representation of the population as judged by their relative difference in standard deviation; but, since the sample median was close to the population median, the 95% bootstrap confidence interval was always going to very easily capture the true population median. \n",
    "\n",
    "C. The sample is a fairly good representation of the population; but, since the sample median was very different from the population median, the 95% bootstrap confidence interval was not able to capture the true population median. \n",
    "\n",
    "D. The sample is a fairly good representation of the population; and, since the sample median was very close to the population median, the 95% bootstrap confidence interval was always going to very easily capture the true population median.  \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c733bbb0",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Q12: your answer will be tested!\n",
    "Q12 = None # Assign either 'A', 'B', 'C', or 'D' to `Q12` instead of `None`\n",
    "# E.g., Q12 = 'A'"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e08f001b",
   "metadata": {},
   "source": [
    "### Q13: Create 1000 samples of sample size $n=500$ from the population of claims stored in the `auto_claims_population.csv` data set; use `plotly` to visualize the simiulated sampling distrubution of the median paid claims based on these 1000 samples; and choose which of the following options is correct. \n",
    "\n",
    "#### Use `fig.add_vline(x=...)` and `fig.add_annotation` to mark the mean of these medians, the `auto_claims_population.csv` population median, and the `auto_claims.csv` sample median on this visualization. \n",
    "\n",
    "A. The mean of the sampling distribution of the sample medians is unexpectedly close to the population median; and, in this case the `auto_claims.csv` sample median happens to not be a very unusual value relative to this sampling distribution either, indicating this sample is likely not very representative of the population.\n",
    "\n",
    "B. The mean of the sampling distribution of the sample medians is unexpectedly close to the population median; whereas, in this case the `auto_claims.csv` sample median is very unusual value relative to this sampling distribution, indicating this sample is likely not very representative of the population.\n",
    "\n",
    "C. The mean of the sampling distribution of the sample medians is very close to the population median, as would be expected; and, in this case the `auto_claims.csv` sample median happens to not be a very unusual value relative to this sampling distribution either, indicating this sample is likely relatively representative of the population (at least with respect to the population median).\n",
    "\n",
    "D. The mean of the sampling distribution of the sample medians is very close to the population median, as would be expected; whereas, in this case the `auto_claims.csv` sample median is very unusual value relative to this sampling distribution, indicating this sample is likely not very representative of the population. \n",
    "\n",
    "> - Hint 1: Ensure each sample is created without replacement, so there are no repeated observations within each sample: <br>**this question is not asking for bootstrapping... it's asking you to create 1000 different samples from a population...**\n",
    "> - Hint 2: The default for `df.sample()` is `replace=False`, whereas the default for `np.random.choice(x, n)` is `replace=True`.\n",
    "\n",
    "- Your `for` loop code and figure will be manually confirmed (for the correct usage of `replace=False`, etc.) during marking."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "72fc7577",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create 1000 samples from the `auto_claims_population.csv` population.\n",
    "# Visualize the medians created from each of these samples, their average, and the\n",
    "# relative `auto_claims_population.csv` population and `auto_claims.csv` sample medians.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "12c18019",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Q13: your answer will be tested!\n",
    "Q13 = None # Assign either 'A', 'B', 'C', or 'D' to `Q13` instead of `None`\n",
    "# E.g., Q13 = 'A'"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "36b55b93",
   "metadata": {},
   "source": [
    "### Q14: Which of the following statements are true?\n",
    "\n",
    "A. The median paid claim amount in the population is given by `pd.read_csv('auto_claims_population.csv').PAID.median()`\n",
    "\n",
    "B. Each median computed in Q13 estimates this, and the overall average of these means estimates it even better; but, it's a little silly to do this because since we have the whole population we can just calculate the median paid claim amount in the population exactly.\n",
    "\n",
    "C. The sample that is bootstrapped from in Q9 is like one of the samples in Q13, so the median of that sample is an estimate of the population median and the bootstrap confidence interval created based on that sample gives us a range estimate of the population median.\n",
    "\n",
    "D. The bootstrap medians in Q9 estimate the median of the original sample being bootstrapped (which we can already calculate exactly); but, if the \"auto_claims.csv\" sample is fairly representative \"auto_claims_population.csv\" population, then the bootstrap sampling distribution of the sample median created in Q9 will behave similarly to the sampling distribution created here in Q11 (but it will just be \"biased\" to be centered around the median of the original sample as opposed to the median of the popuation), so it will give us an idea about the variability of the true sampling distribution of the sample median.\n",
    "\n",
    "E. All of the above\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "637fde19",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Q14: your answer will be tested!\n",
    "Q14 = None # Assign either 'A', 'B', 'C', or 'D' to `Q14` instead of `None`\n",
    "# E.g., Q14 = 'A'"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d477d4b0",
   "metadata": {},
   "source": [
    "### Q15: There's always a chance that once a 95% bootstrap confidence interval is constructed it won't have ended up actually capturing the population parameter it's estimating... for a 95% bootstrap confidence interval this chance is the approximately 100%-95% = 5% chance that a sample randomly drawn from the population will not end up being sufficiently representative of the population it's drawn from so as to result in an confidence interval that does not end up capturing the true population parameter it estimates... \n",
    "\n",
    "#### Since the `\"auto_claims.csv\"` \"sample\" was created based on selecting claims for the $n=500$ oldest individuals in the population (which roughly amounted to selecting `AGE>60`), which of the following comments are accurate?\n",
    "\n",
    "> Hint: Consider that the original `auto_claims.csv` \"sample\" is a \"sample\" of auto insurance claimants greater than 60 years of age, while the population includes other ages; whereas, the randomly drawn samples in `Q13` are not based upon this systematic constraint of the sample relative to the population...\n",
    "\n",
    "\n",
    "A. Based on the behaviour of random sampling, we would expect there to be about a 95% chance that the 95% bootstrap confidence interval created in Q9 would end up capturing the true population median, and it did!\n",
    "\n",
    "B. Based on the behaviour of random sampling, we would expect there to be about a 5% chance that the 95% bootstrap confidence interval created in Q9 would not end up capturing the true population median, but fortunately it did!\n",
    "\n",
    "C. While the 95% bootstrap confidence interval created in Q9 did indeed capture the true population median, there were no specific probabilistic guarantees or expectations as to the chance that it would since it was not based on a random sample from the population.\n",
    "\n",
    "D. Intentionally creating a sample in a systematically non-random manner such as only choosing claims where `AGE>60` is an efficient way to get a sample to use for creating a confidence interval. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4bfa68db",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Q15: your answer will be tested!\n",
    "Q15 = None # Assign either 'A', 'B', 'C', or 'D' to `Q15` instead of `None`\n",
    "# E.g., Q15 = 'A'"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9df40363",
   "metadata": {},
   "source": [
    "### Q16: Why have we been focused on median insurance claims, as opposed to mean insurance claims?\n",
    "\n",
    "A. Using either means or medians here doesn't matter much since the distribution under consideration is symmetric.\n",
    "\n",
    "B. Using medians is better because means in this case don't represent proportions like they would for binary data.\n",
    "\n",
    "C. Means are not robust to outliers whereas medians are not affected by the actual value of outliers, so in addition to perhaps being a more meaningful measure of \"typical\" insurance claim amounts, it will also likely be easier to provide tighter confidence intervals for medians compared to means.\n",
    "\n",
    "D. Medians are easier to use than means and automatically provide an \"apples to apples\" comparision when we're working with both bootstrapping samples as well as actual samples from an actual population, whereas means would result in a more \"apples to oranges\" comparision between bootstrap sampling and population sampling."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6cbac373",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Q16: your answer will be tested!\n",
    "Q16 = None # Assign either 'A', 'B', 'C', or 'D' to `Q16` instead of `None`\n",
    "# E.g., Q16 = 'A'"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b055dcd8",
   "metadata": {},
   "source": [
    "### Q17: Return to your simulation in `Q13`, but this time draw samples of size $n=100$ and add another `for` loop into your code (inside of your first `for` loop which created the 1000 samples) which creates 80% bootstrap confidence intervals for the median of paid claims for each sample (now of size $n=100$) and checks whether or not each of these intervals capture the true population median.\n",
    "\n",
    "#### Save the 1000 samples (now each of size $n=100$) taken from the population as a list of `np.array`s and report how many of them produced an 80% confidence interval that captures the population median.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "32928dc9",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Q17: your results below will be tested:\n",
    "#      you don't need to assign anything into Q17\n",
    "\n",
    "# Complete the code here\n",
    "np.random.seed(130) # Use this line to create reproducible pseudorandomness for your bootstrapping\n",
    "# Leave the line above as is as it is required for testing purposes\n",
    "population = pd.read_csv('auto_claims_population.csv')\n",
    "\n",
    "sample_size_n = 100 # this has changed: use this value now\n",
    "number_of_samples = 1000\n",
    "number_of_bootstrap_samples_per_sample = 100 # this has changed: use this value now\n",
    "number_of_80percent_bootstrap_confidence_intervals_capture_population_median = 0\n",
    "all_samples = []\n",
    "for i in range(number_of_samples):\n",
    "    # `np.random.choice` is an alternative to utilizing `df.sample()` which uses `replace=False` by default\n",
    "    one_sample = np.random.choice(population['PAID'], sample_size_n, replace=False)\n",
    "    # samples are of size 30\n",
    "    all_samples += [one_sample]\n",
    "\n",
    "    for j in range(number_of_bootstrap_samples_per_sample):\n",
    "        pass # Simulate bootstrap sampling distribution of sample median...\n",
    "        \n",
    "    # create 80% bootstrap confidence interval for the current sample\n",
    "        \n",
    "    if True: # fix to check if created interval capture the true population median\n",
    "        number_of_80percent_bootstrap_confidence_intervals_capture_population_median += 1\n",
    "    \n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7c07065f",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}