This is the repository of my course in Foundations of Machine Learning (EE 298M/CoE 197M). Similar to my previous course in Deep Learning, I would like to strike a balance between theory and practical implementation of concepts. The course materials are still under development. Please expect occasional errors, missing parts, bugs, etc. Apologies in advance.
With some modifications, the theory part is based on Mathematics for Machine Learning book. It is freely available at this link.
Thank you! I greatly appreciate the authors for having the book available online. This is helpful for students who have limited resources.
As much as possible, code examples are written to better understand key concepts. The code examples in this course are in Jupyter Notebook. I tried using Google Colab but encountered errors in saving my notebooks. So, I switched to Jupyter Notebook. In theory, you can upload the notebooks here to Google Colab.
- Why Machine Learning - Importance of Foundations of Machine Learning, Course Outline
- ML Toolkit - Environment, Code Editor, Python, Numpy, Matplotlib, etc
- Linear Algebra - Tensors, Operations, Basis, Rank, Spaces/Subspaces, Groups, Linear Mappings
- Slides
- Notebook (Code Examples):
- Analytic Geometry - Distance, Metric, Norm, Inner Product, Basis, Projection, Gram-Schmidt, Rotation
- Matrix Decomposition - Eigenvalues, Eigenvector, Eigendecomposition, Spectral Theorem, Singular-Value Decomposition, Matrix Approximation
- Slides
- Notebook (Code Example):
- Vector Calculus - Learning, Taylor Series, Gradients, Jacobian, Backpropagation, Hessian
- Probability Distributions - Distributions, Gaussian, Bayes, Sufficient Statistics, Exponential Family, Conjugacy, Transformation
- Optimization - Gradient Descent, Stochastic Gradient Descent, Convex Optimization, Linear & Quadratic Programming, Convex Conjugate
- Machine Learning Principles - Empirical Risk Minimization (ERM), Maximum Likelihood Estimation (MLE), Maximum A Posteriori (MAP), Intro to Information Theory, Directed Graphical Models
- Linear Regression - MLE, MAP
- Slides
- Notebook (Code Example):
- Principal Component Analysis - Low-dimensional Representation, Latent Variable Model
- Gaussian Mixture Models Responsibilities, Parameter Updates, Expectation Maximization
- Slides
- Notebook (Code Example):
- Support Vector Machines
If you find the materials in this repo useful, please give it a star or fork it.
If you find this work useful, please cite:
@misc{atienza2020ml,
title={Foundations of Machine Learning},
author={Atienza, Rowel},
year={2020},
publisher = {GitHub},
journal = {GitHub repository},
howpublished = {\url{https://github.com/roatienza/ml}},
}