Skip to content

"We have sought for firm ground and found none. The deeper we penetrate, the more restless becomes the universe; all is rushing about and vibrating in a wild dance."― Max Born

Notifications You must be signed in to change notification settings

manjunath5496/ML-Lectures

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

85 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Lectures (Tom Mitchell and Maria-Florina Balcan)

Lecture Topics Readings and useful links Handouts
Intro to ML
Decision Trees
  • Machine learning examples
  • Well defined machine learning problem
  • Decision tree learning
Mitchell: Ch 3
Bishop: Ch 14.4
The Discipline of Machine Learning
Slides
Decision Tree learning
Review of Probability
  • The big picture
  • Overfitting
  • Random variables and probabilities
Mitchell: Ch 3
Andrew Moore's Basic Probability Tutorial
Slides
Annotated Slides
Probability and Estimation
  • Bayes rule
  • MLE
  • MAP
Mitchell: Estimating Probabilities Slides
Annotated Slides
Naive Bayes
  • Conditional Independence
  • Naive Bayes: why and how
Mitchell: Naive Bayes and Logistic Regression Slides
Annotated Slides
Gaussian Naive Bayes
  • Gaussian Bayes classifiers
  • Document Classification
  • Brain image classification
  • Form of decision surfaces
Mitchell: Naive Bayes and Logistic Regression Slides
Annotated Slides
Logistic Regression
  • Naive Bayes - the big picture
  • Logistic Regression: Maximizing conditional likelihood
  • Gradient ascent as a general learning/optimization method
Mitchell: Naive Bayes and Logistic Regression Slides
Annotated Slides
Linear Regression
  • Generative/Discriminative models
  • Minimizing squared error and maximizing data likelihood
  • Regularization
  • Bias-variance decomposition
  Slides
Annotated Slides
Learning Theory I
  • Distributional Learning
  • PAC and Statistical Learning Theory
  • Sample Complexity
Mitchell: Ch 7
Notes on Generalization Guarantees
Slides
Learning Theory II
  • Sample Complexity
  • Shattering and VC Dimension
  • Sauer's Lemma
Mitchell: Ch 7
Notes on Generalization Guarantees
Slides
Learning Theory III
  • Rademacher Complexity
  • Overfitting and Regularization
  Slides
Graphical Models I
  • Bayes Nets
  • Representing joint distributions with conditional independence assumptions
Bishop chapter 8, through 8.2 Slides
Annotated Slides
Graphical Models II
  • Inference
  • Learning from fully observed data
  • Learning from partially observed data
  Annotated Slides
Graphical Models III
  • EM
  • Semi-supervised learning
Bishop Chapter 8
Mitchell Chapter 6
Slides
Annotated Slides
Exam #1
EM and Clustering
  • Mixture of Gaussian clustering
  • K-means clustering
Bishop Chapter 8
Mitchell Chapter 6
Slides
Annotated Slides
Spring Break
Boosting
  • Weak vs Strong (PAC) Learning
  • Boosting Accuracy
  • Adaboost
Slides
Adaboost, Margins, Perceptron
  • Adaboost: Generalization Guarantees(naive and margins based).
  • Geometric Margins and Perceptron
Notes on Perceptron Slides
Slides (PPT)
Kernels
  • Geometric Margins
  • Kernels: Kernelizing a Learning Algorithm
  • Kernelized Perceptron
Bishop 6.1 and 6.2 Slides
SVM
  • Geometric Margins
  • SVM: Primal and Dual Forms
  • Kernelizing SVM
  • Semi-supervised Learning
  • Semi-supervised SVM
Notes on SVM by Andrew Ng Slides
Semi-supervised Learning
  • Transductive SVM
  • Co-training and Multi-view Learning
  • Graph-based Methods
Slides
Active Learning
  • Batch Active Learning
  • Selective Sampling and Active Learning
  • Sampling Bias
Slides
  • Partitional Clustering
  • Hierarchical Clustering
  • k-means, Lloyd's method, k-means++
  • Agglomerative Clustering
Slides
  • Learning Representations
  • Dimensionality Reduction
  • Principal Component Analysis
  • Kernel Principal Component Analysis
    Bishop 12.1, 12.3
Slides
Never Ending Learning     Slides
Neural Networks
Deep Learning
  Mitchell, Chapter 4 Slides
Reinforcement Learning
  • Markov Decision Processes
  • Value Iteration
  • Q-learning
Slides
Deep Learning
Differential Privacy
Discussion on the Future of ML
    Slides (Privacy)
Slides (Deep Nets)

Andrew Ng's review notes on:


Lecture Notes


C19 ML lectures [Andrew Zisserman]


ML Lecture Notes

(Prof. Qiangfu Zhao, Prof. Yong Liu and Prof. Yuichi Yaguchi)

Contents
History of AI and ML
Fundamentals of machine learning
Introduction to concept learning
Basic statistic learning
Bayesian network
Project-I
Multilayer perceptron
Convolutional neural network
Autoencoder
Restricted Boltzmann machine
Decision trees
Project-II

Statistical ML Lecture Notes [Thomas Schön]

  • [pdf] Introduction (notes)
  • [pdf] Linear regression 
  • [pdf] Linear classification 
  • [pdf] Neural networks, kernel methods intro. 
  • [pdf] Kernel methods 
  • [pdf] EM and clustering (Notes)
  • [pdf] Approximate inference (Notes)
  • [pdf] Graphical models 
  • [pdf] Graphical models and message passing 
  • [pdf] MCMC and sampling methods 
  • [pdf] Bayesian nonparametric models 

Papers


ML Lecture Slides [Sargur Srihari]


    1. Introduction
      1. Machine Learning-Overview(28MB)
      2. Python and ML Frameworks(13.9MB)
      3. Linear Algebra(4.5MB) 
      4. Example: Curve Fitting(934KB)
      5. Probability Theory(4.9MB)
      6. Numerical Computation(1.4MB)
      7. Decision-Theory(488KB)
      8. Information Theory(715KB)
    2. Probability Distributions
      1. Discrete Distributions(1MB)
      2. Gaussian Distribution(833KB)
      3. Gaussian Bayesian Networks(738KB)
    3. Linear Models for Regression
      1. Regression with Basis Functions(7.3MB)
      2. Gradient Descent(3.2MB)
      3. Bias-Variance(950KB)
      4. Bayesian Regression(2.5MB)
      5. Bayesian Model Comparison(478KB)
      6. Evidence Approximation(746KB)
      7. Example: Computer Science Ranking(126KB)
    4. Linear Models for Classification
      1. Overview(4.6MB)
      2. Discriminant Functions(5MB)
      3. Probabilistic Generative Models(1.3MB)
      4. Probabilistic Discriminative Models
        1. Fixed Basis Functions(254KB)
        2. Logistic Regression(3.6MB)
        3. Iterative Reweighted Least Squares(5.1MB)
        4. Multiclass Logistic Regression(4.6MB)
        5. Probit Regression(356KB)
        6. Canonical Link Functions(263KB)
      5. Laplace Approximation (1.3MB)
      6. Bayesian Logistic Regression(1.1MB)
      7. Variational Bayesian Logistic Regression(3.3MB) 
    5. Neural Networks
      1. Biology(4.5MB) 
      2. Feed-forward Network Functions(5.3MB)
      3. Network Training(2.6MB)
      4. Backpropagation(8.7MB)
      5. The Hessian Matrix(562KB)
      6. Regularization in Neural Networks
        1. Norm Penalty: Bayesian Interpretation(1.2MB)
        2. Convolutional Networks(4.9MB)
        3. Soft Weight Sharing(1.2MB)
      7. Mixture Density Networks (634KB)
      8. Bayesian Neural Networks(716KB)
      9. Deep Learning Overview(5.2MB)
    6. Kernel Methods
      1. Kernel Methods(6.3MB)
      2. Radial Basis Function Networks(812KB)
      3. Gaussian Processes(6.8MB)
    7. Sparse Kernel Machines
      1. Support Vector Machines(5.4MB)
      2. SVM for Overlapping Distributions(1.3MB)
      3. Multiclass SVMs (1.4MB)
      4. Relation to Logistic Regression (446KB)
    8. Mixture Models and EM
        1. Unsupervised Learning(1.9MB)
    9. K-means Clustering(1.4MB)
    10. Gaussian Mixture Models(1.5MB)
    11. Latent Variable View of EM(1.1MB)
    12. Bernoulli Mixture Models(3.1MB)
    13. Theoretical Basis of EM(693KB)

ML Course Notes [Nando de Freitas]


  • Lecture 1: Introduction slides 
  • Lecture 2: Linear prediction slides 
  • Lecture 3: Maximum likelihood slides.pdf 
  • Lectures 4 & 5: Regularizers, basis functions and cross-validation slides.pdf 
  • Lecture 6: Optimisation slides.pdf 
  • Lecture 7: Logistic regression slides.pdf 
  • Lecture 8: Back-propagation and layer-wise design of neural nets slides.pdf 
  • Lecture 9: Neural networks and deep learning with Torch slides.pdf 
  • Lecture 10: Convolutional neural networks slides.pdf 
  • Lecture 11: Max-margin learning and siamese networks slides.pdf 
  • Lecture 12: Recurrent neural networks and LSTMs slides.pdf 
  • Lecture 15: Reinforcement learning with direct policy search slides.pdf 
  • Lecture 16: Reinforcement learning with action-value functions slides.pdf 
  • Practical on week 2: (1) Learning Lua and the tensor library. pdf
  • Practical on week 3: (2) Online and batch linear regression. pdf
  • Practical on week 4: (3) Logistic regression and optimization. pdf
  • Practical on week 6: (4) Feedforward neural networks, and implementing your own layer. pdf
  • Practical on week 7: (5) Intro to nngraph for graph-shaped modules. pdf
  • Practical on week 8: (6) Training a LSTM language model. pdf
  • Class on Week 3Problem set
  • Class on Week 5Problem set
  • Class on Week 7Problem set
  • Class on Week 8Problem set

Lectures on Machine Learning (Mark Schmidt)

1. Supervised Learning

2. Unsupervised Learning

3. Linear Models

4. Latent-Factor Models

5. Deep Learning

Part 2: Data Science 573 and 575

Part 3: Computer Science 540

A. Fundamentals

B. Density Estimation

C. Graphical Models

D. Bayesian Learning

E. More Deep Learning

Part 4: Large-Scale Machine Learning

Part 5: Machine Learning Reading Group


Lecture Notes on Data Analysis, Statistics, and Machine Learning (Leland Wilkinson)


Lecture Notes (Arun Debray)


The collected works of F. W. Lawvere

Title Year
The Category of Probabilistic Mappings – With Applications to Stochastic Processes, Statistics, and Pattern Recognition 1962
Functorial Semantics of Algebraic Theories (short notice) 1963
The group ring of a small category (abstract) 1963
An Elementary Theory of the Category of Sets (cf. 2005 long version with commentary) 1964
Algebraic Theories, Algebraic Categories, and Algebraic Functors 1965
Functorial Semantics of Elementary Theories (abstract) 1966
The Category of Categories as a Foundation for Mathematics 1966
Theories as Categories and the Completeness Theorem (abstract) 1967
Some Algebraic Problems in the Context of Functorial Semantic of Algebraic Theories 1968
Ordinal Sums and Equational Doctrines 1969
Diagonal Arguments and Cartesian Closed Categories 1969
Adjointness in Foundations 1969
Equality in Hyperdoctrines and Comprehension Schema as an Adjoint Functor 1970
Quantifiers and Sheaves 1970
Introduction to "Toposes, Algebraic Geometry and Logic" 1971
Theory of Categories over a Base Topos 1972
Metric Spaces, Generalized Logic, and Closed Categories 1973
Introduction to "Model Theory & Topoi" 1975
Variable Sets Etendu and Variable Structure in Topoi 1975
Continuously Variable Sets – Algebraic Geometry=Geometric Logic 1975
Variable Quantities and Variable Structures in Topoi 1976
Categorical Dynamics 1978
Toward the Description in a Smooth Topos of the Dynamically Possible Motions and Deformations of a Continuous Body 1980
Introduction to "Categories in Continuum Physics" 1982
Functorial Remarks on the General Concept of Chaos 1984
State Categories Closed Categories and the Existence of Semi-continuous Entropy Functions 1984
State Categories and Response Functors 1986
Categories of Spaces may not be Generalized Spaces as Exemplified by Directed Graphs 1986
Taking Categories Seriously 1986
On the Complete Lattice of Essential Localizations (with G.M. Kelly) 1988
Display of Graphics and their Applications Exemplifed by 2 Categories and the Hegelian Taco 1989
Qualitative Distinctions Between Some Toposes of Generalized Graphs 1989
Intrinsic Co-Heyting Boundaries and the Leibniz Rule in Certain Toposes 1991
More on Graphic Toposes 1991
Some Thoughts on the Future of Category Theory 1991
Categories of Space and Quantity 1992
Cohesive Toposes and Cantor's Lauter Einsen 1994
Tools for the Advancement of Objective Logic Closed Categories and Toposes 1994
Adjoints in and among Bicategories 1996
Grassmann's Dialectics and Category Theory 1996
Unity and Identity of Opposites in Calculus and Physics 1996
Toposes of Laws of Motion 1997
Volterra's Functionals and Covariant Cohesion of Space 1997
Outline of Synthetic Differential Geometry 1998
Are Homotopy Types the Same As Infinitesimal Skeleta? (abstract) 1998
Kinship and Mathematical Categories 1999
Comments on the Development of Topos Theory 2000
The Role of Cartesian Closed Categories in Foundations (Interview with Felice Cardone) 2000
Categorical Algebra for Continuum Micro Physics 2001
On the Duality Between Varietes and Algebraic Theories (with J. Adámek & J. Rosický) 2003
How Algebraic is Algebra? (with J. Adámek & J. Rosický) 2001
Linearization of Graphic Toposes via Coxeter Groups 2002
Foundations and Applications – Axiomatization and Education 2003
Continuous Categories Revisited (with J. Adámek & J. Rosický) 2003
Functorial Semantics of Algebraic Theories and Some Algebraic Problems in the Context of Functorial Semantics of Algebraic Theories 2004
Functorial Concepts of Complexity for Finite Automata 2004
Left and Right Adjoint Operations on Spaces and Data Types 2004
An Elementary Theory of the Category of Sets (long version) with commentary 2005
Grassmann Book Reviews 2005
John Isbell's Adequate Subcategories 2006
Axiomatic Cohesion 2007
Cohesive toposes: combinatorial and infinitesimal cases (lecture notes) 2008
Core Varieties Extensivity and Rig Geometry 2008
Interview with Maria Manuel Clementino and Jorge Picado 2008
Foreword To "Algebraic Theories" 2009
Open Problems in Topos Theory 2009
The Hopf Algebra of Möbius Intervals (with M. Menni) 2010
Categorical Dynamics (abstract) 2011
Euler's Continuum Functorially Vindicated 2011
What are Foundations of Geometry and Algebra? (abstract) (transcript) 2013
Internal Choice Holds in the Discrete Part of any Cohesive Topos Satisfying Stable Connected Codiscreteness (with M. Menni) 2015
Alexander Grothendieck and the Concept of Space 2015
Birkhoff's Theorem from a Geometric Perspective: A Simple Example 2016
Everyday physics of extended bodies or why functionals need analyzing 2017

About

"We have sought for firm ground and found none. The deeper we penetrate, the more restless becomes the universe; all is rushing about and vibrating in a wild dance."― Max Born

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages