Skip to content
A Matlab implementation of the Soft Committee Machine and learning algorithms + an implementation of differential equations for the thermodynamic limit. Ideal for comparing theoretical results with simulations.
Jupyter Notebook MATLAB Mathematica Python M
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.idea
.ipynb_checkpoints
MATLAB
Python
Writings
README.md
Simulation Dropout.ipynb
TODO

README.md

Physics_of_Learning

A description of the repository and usage of the scripts is following soon.

Upcoming functionality:

  • Add the Swish activation function in the simulation and potentially in the theory. Experiment with ReLU vs. Swish learning a rule defined by a sigmoidal Erf teacher.
  • Drift processes and weight decay.
  • Exact theoretical on-line ReLU dynamics for general learning rate, if possible.
  • Hidden-to-output weights, additional layers, tree-like structures.
  • Other learning scenarios, batch learning.
  • Active learning
You can’t perform that action at this time.