SRLR

Sketched Ridgeless Linear Regression

Description

This repository presents numerical simulations that analyze the empirical risks of the sketched ridgeless estimator, aiming to enhance generalization performance. The simulations focus on determining optimal sketching sizes that minimize out-of-sample prediction risks. The results reveal that the optimally sketched estimator exhibits stable risk curves, effectively eliminating the peaks observed in the full-sample estimator. Additionally, we introduce a practical procedure to empirically identify the optimal sketching size.

Suppose we observe data vectors (x_i,y_i) that follow a linear model y_i=x_i^Tβ+ε_i, i=1,...n, where y_i is a univariate response, x_i is a d-dimensional predictor, β denotes the vector of regression coefficients, and ε_i is a random error. We consider the ridgeless least square estimator β̂=(X^TX)⁺X^TY.

With this package, the simulation results in this paper can be reporduced.

Installation

Install SRLR_python from PyPI:

pip install SRLR

Examples

Please refer to tutorial.ipynb for a comprehensive example and step-by-step guide.

Reference

Chen, X., Zeng, Y., Yang, S. and Sun, Q. Sketched Ridgeless Linear Regression: The Role of Downsampling. Paper