| layout |
|---|
default |
| Lecture | Pre-reading | Slides |
|---|---|---|
| 1 | no pre-reading | intro, L1 |
| 2 | Introduction to Probability with Statistical Applications, Géza Schay Chapter 4 - Random Variables (p.71 - 126) Chapter 5 - Expectation, Variance, Moments (p.127 - 176) |
L2 |
| 3 | 1. Variational Inference: A Review for Statisticians (p. 1-13, exl. 3.3) 2. An Introduction to MCMC for Machine Learning (p.1-23 exl. 3.5) or alternatively: Pattern Recognition and Machine Learning - 11.2. Markov Chain Monte Carlo (p. 537 - 542) - 11.3. Gibbs Sampling (p. 542 - 546) - notation as introduced in section 11. Sampling Methods (p. 523) |
L3 |
| 4 | no pre-reading | L4 |
| 5 | A Tutorial on Bayesian Nonparametric Models, p. 1-9, sec. 1 and 2 | L5 |
| 6 | no pre-reading | L6 |
| 7 | no pre-reading | L7 |
| 8 | Data Analysis with Latent Variable Models | L8 |
| 9 | no pre-reading | L9 |
| 10 | no pre-reading | L10 |
| 11 | Hidden Markov Models Simplified | L11 |
| 12 | Rep. Lecture 5 | |
| 13 | no pre-reading | L13 |
| 14 | Exam - round 0 | |
| 15 | Individal Q&A |
| Round | Date | Form |
|---|---|---|
| 0 | 12.06.2020 9:15 | remote |
| 1 | 18.06.2020 11:15 | remote |
| 2 | 25.06.2020 11:15 | remote |
- Probability Space. Sample space. Events. Venn diagrams. Relations between events. Probability. Probabilistic vs frequentonist. Kolmogorov’s Axioms. Symmetry. Deductions from Axioms. Conditional Probability. Independence. Law of Total Probability. Bayes’ theorem. Random Variables. Representatives of interesting RV.
- Joint Distributions. Independence of Random Variables. Expected Value. Variance and Standard Deviation. Moments and Generating Functions. Covariance and Correlation. Conditional Expectation. Median and Quantiles. Bayesian probability. Likelihood function. Bayesian inference. Bayesian versus Frequentist Inference. Prior. Empirical Bayes. Large Sample Theory. Ability to solve similar to `the number game'. Occam razor. Beta-binomial model. Conjugate priors (to Bernoulli, Binomial, Categorical, Multinomial likelihood). Bayesian Linear Regression vs Frequentonist regression. Gaussian Mixture Model. Multivariate Gaussian. Mutual information.
- Statistic. Estimators. Test Statistics. Point vs. Interval estimators. Information criteria. AIC. BIC. Likelihood-ratio test. Least squares estimation. Maximum likelihood estimation. MLE procedures: explicit, iterative. Bayesian estimation. MAP. MMSE. MCMC. Metropolis-Hastings algorithm. Variational Inference. KL Divergence. ELBO. Classes of approximate Bayesian inference.
- Bias-variance. Bias error. Variance error. Curse of dimensionality. Vapnik–Chervonenkis dimension. Shattering. Evaluation of non bayesian models. Evaluation of bayesian models. f-divergence. Integral probability metrics.
- Bayesian nonparametrics. BNP vs Bayesian. Stochastic Process. Stationarity. Exchangeability and de Finetti’s Theorem. Nonparametric model representation. Chinese restaurant process. Nonlinear bayesian regression. Gaussian process. Kernels. Kernel trick. Gaussian processes for regression. Kernel density estimation. Kolmogorow extension theorem (https://en.wikipedia.org/wiki/Kolmogorov_extension_theorem)
- Mixture model. Estimation in Mixture models. Estimation in Gaussian Mixture Model. Variational Inference. K-means clustering. Expectation-Maximization (EM) algorithm. Probabilistic Graphical Models. Graph characteristics (node, cycle, loop, etc.). Chain rule. Conditional independence. D-separation. Path blocking.
- Directed graphical models. Plate notation. Markov Random Fields. Conditional independence. Markov blanket. Factorization. Undirected. Potential function. Relation of directed and undirected models. D-map. I-map. Inference in Graphical Models. Inference on a chain. Factor graphs. Moralization. The sum-product algorithm.
- Conditinal Random Fields. CRF vs MRF. Gradient method for CRF training. Latent Dirichlet Allocation. Dirichlet distribution. Estimation in LDA. Approximate Inference in PGMs. Loopy belief propagation.
- Frequentionist approach Neural Networks. Criterions. Activation functions. Optimization. Autograd. Regularization techniques (incl. dropout). Bayesian Neural Networks. Bayesian approach. Variational inference (recap). Reparametrization trick. Bayesian Approximation through dropout.
- Variational Auto-Encoders. Autoencoder vs VAE. Normalizing flows.
- HMM. Sequential data and Markov Models. Maximum Likelihood for the HMM (EM). The forward-backward, sum product and Viterbi algorithm.