Code for Implicit Sparse Regularization: The Impact of Depth and Early Stopping
The convergence for different choices of depth N is shown on both simulated data and real data (MNIST). We also studied the effects of depth N on initialization scale and early stopping window. The connection to incremental learning and kernel regime is also empirically shown. To get the numerical results, simply run
chmod +x run.sh
./run.sh
- Jiangyuan Li, Thanh V. Nguyen, Chinmay Hegde, Raymond K. W. Wong. (2021) "Implicit Sparse Regularization: The Impact of Depth and Early Stopping". Conference on Neural Information Processing Systems (NeurIPS).