STAD80: A Review of Preetum Nakkiran's "More Data Can Hurt for Linear Regression: Sample-wise Double Descent"
- Shawn Santhoshgeorge
- Anaqi As Shafiq Bin Amir Razif
In overparameterized linear regression, the phenomenon of double descent can be observed were the Test MSE initially decreases with a fixed parameter and an increasing sample
. It keeps on decreasing before increasing asymptotically towards infinity when
before it begins decreasing again. We call the regime where
to be the overparameterized regime and the underparameterized regime is when
. There are interesting concepts that can be found when researching and analyzing these regimes especially around the area of
. This repository looks into the claims made in "More Data Can Hurt for Linear Regression: Sample-wise Double Descent" by Preetum Nakkiran. It includes the a deeper analysis on Section 3.1 of paper and works to explain the conclusions of the paper.
This graph was created based on a simple simulation based on the configuration stated in the paper and was a test to see if we could replicate the claims.
