Numerical Analysis for Artificial Intelligence, Jacek Cyranka

This repository contains course materials from the "Numerical Analysis for Artificial Intelligence" course I presented at UCSD, CSE department during Summer session 2018.

course syllabus

• Week 1,2 Review of Programming in Python+NumPy+IPython notebook and �Calculus and Linear Algebra topics

  • Python language basics,
  • Linear Algebra in NumPy,
  • Working with Jupyter notebooks,
  • Example problem of solving a linear regression analytically,
  • Functions,
  • Vector spaces,
  • Matrices,
  • Matrix times vector/matrix operation,
  • Matrix transpose/inverse,
  • Solving systems of linear equations,
  • Basic properties,
  • Partial Derivatives,
  • Critical points,
  • Chain rule and gradients,
  • Characterization of critical points as local/global minima/maxima.

• Week 3 Gradient descent and convex optimization

  • Backpropagation algorithm,
  • gradient checking of a backpropagation implementation,
  • Avoiding problems with convergence by decreasing the learning rate,
  • Accelerated gradient descent (Nesterov momentum method),
  • Minimizing a quadratic function,
  • Solving linear regression using gradient descent.

• Week 4,5 Nonconvex optimization : supervised learning of feed-forward Neural Networks

  • Difference in Convex/Nonconvex optimization,
  • Classical Blum/Rivest proof that training a 3-node NN is NP-Complete,
  • Perceptrons
  • Single hidden layered networks,
  • Linear, ReLU , tangential networks,
  • Mean squared error loss, cross entropy loss,
  • Multiple hidden layered feed forward networks,
  • Newton’s method.