Implicit Self Regularization

Repo to accompany papers

Implicit Self-Regularization in Deep Neural Networks: Evidence from Random Matrix Theory and Implications for Learning

Traditional and Heavy Tailed Self Regularization in Neural Network Models

Video Presentation

Recent talk (presented at NERSC Summer 2018)

Name		Name	Last commit message	Last commit date
Latest commit History 53 Commits
allennlp		allennlp
img		img
.gitignore		.gitignore
AlexNet-Baseline-Ensemble.ipynb		AlexNet-Baseline-Ensemble.ipynb
AlexNet-Baseline-wDropout.ipynb		AlexNet-Baseline-wDropout.ipynb
AlexNet-Baseline.ipynb		AlexNet-Baseline.ipynb
AlexNet-Generalization-Gap.ipynb		AlexNet-Generalization-Gap.ipynb
AlexNet-RankCollapse.ipynb		AlexNet-RankCollapse.ipynb
AlexNet-Regularized.ipynb		AlexNet-Regularized.ipynb
AlexNet-pytorch-MPFit.ipynb		AlexNet-pytorch-MPFit.ipynb
AlexNet-pytorch.ipynb		AlexNet-pytorch.ipynb
All-pytorch-models-wCNNs-Slices.ipynb		All-pytorch-models-wCNNs-Slices.ipynb
All-pytorch-models-wCNNs.ipynb		All-pytorch-models-wCNNs.ipynb
All-pytorch-models.ipynb		All-pytorch-models.ipynb
All-pytorch-ranks.ipynb		All-pytorch-ranks.ipynb
Automated-MPFit.ipynb		Automated-MPFit.ipynb
CalibrateLargeNPL.ipynb		CalibrateLargeNPL.ipynb
CalibratePowerLawFits.ipynb		CalibratePowerLawFits.ipynb
EvenMore-pytorch-models-Accuracies-Copy1.ipynb		EvenMore-pytorch-models-Accuracies-Copy1.ipynb
EvenMore-pytorch-models-Accuracies.ipynb		EvenMore-pytorch-models-Accuracies.ipynb
EvenMore-pytorch-models-OSMR.ipynb		EvenMore-pytorch-models-OSMR.ipynb
EvenMore-pytorch-models.ipynb		EvenMore-pytorch-models.ipynb
GaussianRMT.ipynb		GaussianRMT.ipynb
HeavyTailedUniversalityClasses.ipynb		HeavyTailedUniversalityClasses.ipynb
Inception-pytorch.ipynb		Inception-pytorch.ipynb
Keras-PowerLaws.ipynb		Keras-PowerLaws.ipynb
Keras-Save-Load-Test.ipynb		Keras-Save-Load-Test.ipynb
LeNet5.ipynb		LeNet5.ipynb
MLP3-Basline.ipynb		MLP3-Basline.ipynb
Marchenko_Pastur_Plots.ipynb		Marchenko_Pastur_Plots.ipynb
PowerLawAccuracyTest.ipynb		PowerLawAccuracyTest.ipynb
README.md		README.md
RMT_Util.ipynb		RMT_Util.ipynb
Spiked-Covariance-Model.ipynb		Spiked-Covariance-Model.ipynb
WeightWatcherTest.ipynb		WeightWatcherTest.ipynb
WeightWatchers.ipynb		WeightWatchers.ipynb
WordEmbeddingMatrices.ipynb		WordEmbeddingMatrices.ipynb
accuracies		accuracies
attention_weights		attention_weights
gpt2_weights.py		gpt2_weights.py
rmt.ipynb		rmt.ipynb

jqhoogland/implicit-self-regularization

Folders and files

Latest commit

History

Repository files navigation

Implicit Self Regularization

Video Presentation

About

Resources

Stars

Watchers

Forks

Languages