Skip to content

added 1 proof #158

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
JoramSoch merged 4 commits into from
Apr 14, 2022
Merged

added 1 proof #158

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
668d265
Merge pull request #82 from StatProofBook/master
JoramSoch Apr 14, 2022
678616d
added 1 proof
JoramSoch Apr 14, 2022
75a2531
edited table of contents
JoramSoch Apr 14, 2022
8861d3d
corrected proof
JoramSoch Apr 14, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 16 additions & 15 deletions I/ToC.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -520,21 +520,22 @@ title: "Table of Contents"
&emsp;&ensp; 1.3.5. **[Expectation of estimates](/P/slr-olsmean)** <br>
&emsp;&ensp; 1.3.6. **[Variance of estimates](/P/slr-olsvar)** <br>
&emsp;&ensp; 1.3.7. **[Distribution of estimates](/P/slr-olsdist)** <br>
&emsp;&ensp; 1.3.8. **[Effects of mean-centering](/P/slr-meancent)** <br>
&emsp;&ensp; 1.3.9. *[Regression line](/D/regline)* <br>
&emsp;&ensp; 1.3.10. **[Regression line includes center of mass](/P/slr-comp)** <br>
&emsp;&ensp; 1.3.11. **[Projection of data point to regression line](/P/slr-proj)** <br>
&emsp;&ensp; 1.3.12. **[Sums of squares](/P/slr-sss)** <br>
&emsp;&ensp; 1.3.13. **[Transformation matrices](/P/slr-mat)** <br>
&emsp;&ensp; 1.3.14. **[Weighted least squares](/P/slr-wls)** (1) <br>
&emsp;&ensp; 1.3.15. **[Weighted least squares](/P/slr-wls2)** (2) <br>
&emsp;&ensp; 1.3.16. **[Maximum likelihood estimation](/P/slr-mle)** (1) <br>
&emsp;&ensp; 1.3.17. **[Maximum likelihood estimation](/P/slr-mle2)** (2) <br>
&emsp;&ensp; 1.3.18. **[Sum of residuals is zero](/P/slr-ressum)** <br>
&emsp;&ensp; 1.3.19. **[Correlation with covariate is zero](/P/slr-rescorr)** <br>
&emsp;&ensp; 1.3.20. **[Residual variance in terms of sample variance](/P/slr-resvar)** <br>
&emsp;&ensp; 1.3.21. **[Correlation coefficient in terms of slope estimate](/P/slr-corr)** <br>
&emsp;&ensp; 1.3.22. **[Coefficient of determination in terms of correlation coefficient](/P/slr-rsq)** <br>
&emsp;&ensp; 1.3.8. **[Correlation of estimates](/P/slr-olsdist)** <br>
&emsp;&ensp; 1.3.9. **[Effects of mean-centering](/P/slr-meancent)** <br>
&emsp;&ensp; 1.3.10. *[Regression line](/D/regline)* <br>
&emsp;&ensp; 1.3.11. **[Regression line includes center of mass](/P/slr-comp)** <br>
&emsp;&ensp; 1.3.12. **[Projection of data point to regression line](/P/slr-proj)** <br>
&emsp;&ensp; 1.3.13. **[Sums of squares](/P/slr-sss)** <br>
&emsp;&ensp; 1.3.14. **[Transformation matrices](/P/slr-mat)** <br>
&emsp;&ensp; 1.3.15. **[Weighted least squares](/P/slr-wls)** (1) <br>
&emsp;&ensp; 1.3.16. **[Weighted least squares](/P/slr-wls2)** (2) <br>
&emsp;&ensp; 1.3.17. **[Maximum likelihood estimation](/P/slr-mle)** (1) <br>
&emsp;&ensp; 1.3.18. **[Maximum likelihood estimation](/P/slr-mle2)** (2) <br>
&emsp;&ensp; 1.3.19. **[Sum of residuals is zero](/P/slr-ressum)** <br>
&emsp;&ensp; 1.3.20. **[Correlation with covariate is zero](/P/slr-rescorr)** <br>
&emsp;&ensp; 1.3.21. **[Residual variance in terms of sample variance](/P/slr-resvar)** <br>
&emsp;&ensp; 1.3.22. **[Correlation coefficient in terms of slope estimate](/P/slr-corr)** <br>
&emsp;&ensp; 1.3.23. **[Coefficient of determination in terms of correlation coefficient](/P/slr-rsq)** <br>

1.4. Multiple linear regression <br>
&emsp;&ensp; 1.4.1. *[Definition](/D/mlr)* <br>
Expand Down
4 changes: 2 additions & 2 deletions P/slr-meancent.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ username: "JoramSoch"
---


**Theorem:** In [simple linear regression](/D/slr), when the independent variable $y$ and/or the dependent variable $x$ are [mean-centered](/D/mean), the [ordinary least squares](/P/slr-ols) estimate for the intercept changes, but that of the slope does not.
**Theorem:** In [simple linear regression](/D/slr), when the dependent variable $y$ and/or the independent variable $x$ are [mean-centered](/D/mean), the [ordinary least squares](/P/slr-ols) estimate for the intercept changes, but that of the slope does not.

**Proof:**

Expand Down Expand Up @@ -98,4 +98,4 @@ $$ \label{eq:slr-ols-meancent-xy}
\end{split}
$$

and we can see that $\hat{\beta}_1(\tilde{x},\tilde{y}) = \hat{\beta}_1(x,y)$, but $\hat{\beta}_0(\tilde{x},\tilde{y}) \neq \hat{\beta}_0(x,y)$, specifically $\beta_0$ is now forced to become zero.
and we can see that $\hat{\beta}_1(\tilde{x},\tilde{y}) = \hat{\beta}_1(x,y)$, but $\hat{\beta}_0(\tilde{x},\tilde{y}) \neq \hat{\beta}_0(x,y)$, specifically $\beta_0$ is now forced to become zero.
49 changes: 49 additions & 0 deletions P/slr-olscorr.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
---
layout: proof
mathjax: true

author: "Joram Soch"
affiliation: "BCCN Berlin"
e_mail: "joram.soch@bccn-berlin.de"
date: 2022-04-14 17:17:00

title: "Parameter estimates for simple linear regression are uncorrelated after mean-centering"
chapter: "Statistical Models"
section: "Univariate normal data"
topic: "Simple linear regression"
theorem: "Correlation of estimates"

sources:

proof_id: "P320"
shortcut: "slr-olscorr"
username: "JoramSoch"
---


**Theorem:** In [simple linear regression](/D/slr), when the independent variable $x$ is [mean-centered](/D/mean), the [ordinary least squares](/P/slr-ols) estimates for slope and intercept are [uncorrelated](/D/corr).


**Proof:** The [parameter estimates for simple linear regression are bivariate normally distributed under ordinary least squares](/P/slr-olsdist):

$$ \label{eq:slr-olsdist}
\left[ \begin{matrix} \hat{\beta}_0 \\ \hat{\beta}_1 \end{matrix} \right] \sim \mathcal{N}\left( \left[ \begin{matrix} \beta_0 \\ \beta_1 \end{matrix} \right], \, \frac{\sigma^2}{(n-1) \, s_x^2} \cdot \left[ \begin{matrix} x^\mathrm{T}x/n & -\bar{x} \\ -\bar{x} & 1 \end{matrix} \right] \right)
$$

Because the [covariance matrix](/D/covmat) of the [multivariate normal distribution](/D/mvn) contains the pairwise covariances of the [random variables](/D/rvar), we can deduce that the [covariance](/D/cov) of $\hat{\beta}_0$ and $\hat{\beta}_1$ is:

$$ \label{eq:slr-olscov}
\mathrm{Cov}\left( \hat{\beta}_0, \hat{\beta}_1 \right) = -\frac{\sigma^2 \, \bar{x}}{(n-1) \, s_x^2}
$$

where $\sigma^2$ is the [noise variance](/D/slr), $s_x^2$ is the [sample variance](/D/var-samp) of $x$ and $n$ is the number of observations. When $x$ is mean-centered, we have $\bar{x} = 0$, such that:

$$ \label{eq:slr-olscov-meancent}
\mathrm{Cov}\left( \hat{\beta}_0, \hat{\beta}_1 \right) = 0 \; .
$$

Because [correlation is equal to covariance divided by standard deviations](/D/corr), we can conclude that the correlation of $\hat{\beta}_0$ and $\hat{\beta}_1$ is also zero:

$$ \label{eq:slr-olscorr-qed}
\mathrm{Corr}\left( \hat{\beta}_0, \hat{\beta}_1 \right) = 0 \; .
$$