07-STAT_220_InferenceSingleProportionPreLab-tidy.Rmd

---
title: 'Inference for a single proportion pre-lab: tidy'
author: "Professor McNamara"
output:
  html_document:
    toc: true
    toc_depth: 3
    toc_float: true
---

We use three packages in this course: `Lock5Data`, `tidyverse` and `infer`. To load a package, you use the `library()` function, wrapped around the name of a package. I've put the code to load one package into the chunk below. Add the other two you need. 

```{r setup, include=FALSE}
knitr::opts_chunk$set(echo = TRUE, error = TRUE)
library(Lock5Data)
# put in the other two packages you need here
```

## Loading in data

As usual, we'll load the example data, `GSS_clean.csv`. It should be inside the `data` folder in your RStudio Cloud. We'll use the `read_csv()` function to read in the data. 

```{r data-load}
GSS <- read_csv("data/GSS_clean.csv")
```

## Missing data

Let's make our lives easier by dropping NA values for the variable we'll be considering today,

```{r}
GSS <- GSS |>
  drop_na(self_emp_or_works_for_somebody)
```


## Inference

So far, we've done inference using simulation methods (lab 5 was about the bootstrap and lab 6 was about randomization). Now, we're going to do inference using distributional approximations. 

Today, we'll cover inference for a single proportion. 

When we're doing inference about a single proportion, we approximate using the standard normal distribution, but we need a formula to approximate the standard error. 

Let's review the formulas:

For one proportion, in a hypothesis test

$$
SE_{p_0}=\sqrt{\frac{p_0(1-p_0)}{n}}
$$

For one proportion, in a confidence interval,
$$
SE_{\hat{p}} = \sqrt{\frac{\hat{p}(1-\hat{p})}{n}}
$$

We can only use these formulas (and the Normal distribution) for inference if the conditions for inference are met. What are those conditions?



## One proportion

For this lab, we'll consider the proportion of people who are self-employed. We learned how to compute a proportion way back in lab 2. See if you can remember how to compute the sample proportion (also known as the point estimate).

```{r}

```

Okay, so that's a little more than 10% who say they are self-employed. But, is it significantly more? That's what hypothesis testing can help us answer. 

### Hypothesis testing for one proportion


Say we wanted to know if the proportion of people who are self employed is greater than 10\%. We could write out our null and alternative hypotheses,


$$
H_0: p \leq 0.1 \\
H_A:p > 0.1
$$
Then, we could use R to help us perform a hypothesis test.
```{r}
GSS |> 
  prop_test(response = self_emp_or_works_for_somebody, 
            alternative = "greater", p = 0.1)
```

Notice that I have to specify the alternative (the options are "two-sided", "greater" or "less") and the null proportion I'm testing against. 

What is our generic conclusion here?

What does that mean in context? 

### Confidence interval for one proportion

Okay, so it's not significantly great than 10%. But, what are some other reasonable values we could have observed? That's what a confidence interval helps us determine. 

Again, I could use `prop_test()` to help me,

```{r}
GSS |> 
  prop_test(response = self_emp_or_works_for_somebody, conf_int = TRUE)
```

How can we interpret this confidence interval? 



## Appendix-- doing it "by hand"

Obviously, the `prop_test()` function helps us do these type of inferential tasks quickly! If we wanted to, we could do the work "by hand," using R as a calculator. 

### Hypothesis test by hand

The first thing we would need is a z-score, which is our observed proportion minus our null proportion over the standard error, 

$$
z = \frac{\hat{p}-p_0}{SE}
$$
and we'd need to use the appropriate formula for the standard error, 

$$
SE_{p_0}=\sqrt{\frac{p_0(1-p_0)}{n}}
$$

We could use R as a calculator to find the standard error

```{r}
sqrt((0.1*0.9)/2261)
```

and then the z-score,

```{r}
(0.103-0.1)/0.006309152
```

Then we'd need to see how extreme 0.475 is in the context of the normal distribution. Based on the facts I know about the normal, I don't think it's going to be significant. But, I could find the p-value using `pnorm()`,

```{r}
1-pnorm(0.4754997)
```

Why did I need to do 1 minus the number?

We could also build this into a long data pipeline,  

```{r}
GSS |>
  group_by(self_emp_or_works_for_somebody) |>
  summarize(n = n()) |>
  mutate(prop = n / sum(n), total = sum(n)) |>
  filter(self_emp_or_works_for_somebody == "Self-employed") |>
  mutate(se = sqrt(prop * (1 - prop) / total)) |>
  mutate(z_stat = (prop - 0.1) / se) |>
  mutate(pvalue = pnorm(z_stat, lower.tail = FALSE))
```

### Confidence intervals by hand

To do a confidence interval, we need a slightly different standard error. We can refer to our formula,

$$
SE_{\hat{p}} = \sqrt{\frac{\hat{p}(1-\hat{p})}{n}}
$$

and use R as a calculator

```{r}
sqrt((0.103*(1-0.103))/2261)
```

To compute a confidence interval, we need one more thing, which is a critical value. In the case of a proportion, this will be a critical z-value, which you might have memorized (it's 1.96 for a 95\% confidence interval) or you may need to compute. Let's check to make sure I'm right about this 1.96 thing.

```{r}
qnorm(0.975)
```

Why did I use 0.975?

Now, we can compute our confidence interval by hand:

```{r}
0.103 - 1.96 * 0.00639
0.103 + 1.96 * 0.00639
```

We could also do this in a longer (but perhaps more readable?) way in a data pipeline,

```{r}
GSS |>
  group_by(self_emp_or_works_for_somebody) |>
  summarize(n = n()) |>
  mutate(prop = n / sum(n), total = sum(n)) |>
  filter(self_emp_or_works_for_somebody == "Self-employed") |>
  mutate(
    se = sqrt(prop * (1 - prop) / total),
    critical_z = 1.96,
    me = critical_z * se
  ) |>
  mutate(low = prop - me, high = prop + me)
```