Machine Learning Lectures

Themes to cover:

• Introduction to Probability: Discrete and continuous probability distributions, Likelihood function, Bayes theorem, Limit theorems, Smirnov's theorem and Inverse method for random variables generation, Acceptance-Rejection method for random variables generation, Dependence and Copulas, Entropy;
• Introduction to Machine Learning: Iris dataset for classification and clusterization tasks;
• Chapter 1 - Perceptrons (batch and on-line learning);
• Chapter 2 - Bayes classifiers;
• Chapter 3 - Regression classifier and Maximum A Posteriori estimates;
• Chapter 4 - Mean-Squares estimation, Gradient descent, Gauss-Newton algorithms;
• Chapter 5 - Multilayer Perceptron;
• Chapter 6 - K-Means clustering algorithm and variants;
• Chapter 7 - Kernel methods introduction (kernel trick, kernel classifier, feature space);
• Chapter 8 - Radial-Basis interpolation (a bit of regularization), RBF networks for classification;
• Chapter 9 - Wisdom of crowds and Ensemble learning;
• Chapter 10 - ARIMA (time series analysis) + Kalman filter;
• Chapter 11 - Singular Spectrum Analysis (seasonal time series analysis);
• Chapter 12 - Bayesian Inference;

Links for datasets:

• scikit-learn.org
• www.kaggle.com
• https://vincentarelbundock.github.io/Rdatasets/datasets.html

Blogs:

• http://colah.github.io/
• http://www.inference.vc/deep-learning-is-easy/

Books:

• Simon Haykin Neural Networks and Learning Machines: https://www.amazon.com/Neural-Networks-Learning-Machines-3rd/dp/0131471392 (2nd edition was translated to russian)
• Sebastian Raschka Python Machine Learning: https://www.packtpub.com/big-data-and-business-intelligence/python-machine-learning
• Christopher Bishop Pattern Recognition and Machine Learning https://www.amazon.com/Pattern-Recognition-Learning-Information-Statistics/dp/0387310738 (a free pdf copy is available - first link in Google)
• Trevor Hastie, Robert Tibshirani, Jerome Friedman The Elements of Statistical Learning http://statweb.stanford.edu/~tibs/ElemStatLearn/printings/ESLII_print10.pdf

Articles:

• Automatic Differentiation of Algorithms for Machine Learning https://arxiv.org/pdf/1404.7456.pdf