This repository contains course materials from the "Numerical Analysis for Artificial Intelligence" course I presented at UCSD, CSE department during Summer session 2018.
Week 1,2 Review of Programming in Python+NumPy+IPython notebook and �Calculus and Linear Algebra topics
- Python language basics,
- Linear Algebra in NumPy,
- Working with Jupyter notebooks,
- Example problem of solving a linear regression analytically,
- Vector spaces,
- Matrix times vector/matrix operation,
- Matrix transpose/inverse,
- Solving systems of linear equations,
- Basic properties,
- Partial Derivatives,
- Critical points,
- Chain rule and gradients,
- Characterization of critical points as local/global minima/maxima.
Week 3 Gradient descent and convex optimization
- Backpropagation algorithm,
- gradient checking of a backpropagation implementation,
- Avoiding problems with convergence by decreasing the learning rate,
- Accelerated gradient descent (Nesterov momentum method),
- Minimizing a quadratic function,
- Solving linear regression using gradient descent.
Week 4,5 Nonconvex optimization : supervised learning of feed-forward Neural Networks
- Difference in Convex/Nonconvex optimization,
- Classical Blum/Rivest proof that training a 3-node NN is NP-Complete,
- Single hidden layered networks,
- Linear, ReLU , tangential networks,
- Mean squared error loss, cross entropy loss,
- Multiple hidden layered feed forward networks,
- Newton’s method.