The objective of this repository is to have a place where we can keep track of all our presentations.
During our weekly sessions, we will be covering the following topics:
-
Generative Models
Generative Adversarial Networks (GAN). Variational Autoencoders (VAE) Stochastic Backpropagation and Approximate Inference in Deep Generative Models (paper) Flow based methods (normalized flows)
- AVB: Adversarial Variational Bayes: Unifying Variational Autoencoders and Generative Adversarial Networks (paper)
- Gumbel-SoftmaxVAE: Categorical Reparameterization with Gumbel-Softmax (paper) The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables (paper)
- VQVAE: Neural Discrete Representation Learning (paper)
- TripleGAN: Triple Generative Adversarial Nets (paper)
- ConditionalVAE: Semi-Supervised Learning with Deep Generative Models (paper)
- AAE: Adversarial Autoencoders (paper)
- f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization (paper)
- ImprovedGAN: Improved Techniques for Training GANs (paper)
- DCGAN: Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks (paper)
- WGAN: Wasserstein Generative Adversarial Networks (paper)
Tutorials
- NIPS 2016 Tutorial: Generative Adversarial Networks (paper slides video)
- CVPR 2017 Tutorial: Theory and Application of Generative Adversarial Networks (slides video)
- Tutorial on Variational Autoencoders (paper)
- Bayesian Deep Learning and Generic Bayesian Inference (slides)
1.1. Differencial Privacy:
Area of research which seeks to provide rigorous, statistical guarantees against what an adversary can infer from learning the results of some randomized algorithm (Alan Turing Institute)
-
Information Theory, Information Geometry and Natural Gradients
Basic tools (definitions, etc) Go through a textbook or some basic review papers.
-
Reinforcement Learning
Bandits Theory/background of Markov Decision Process (MDP) Optimality Value iteration, policy iteration Value based methods Policy search (policy gradient) methods
-
Statistical Learning Theory
Framework; Rademacher Complexity; VC dimension; basic bounds SVMs
-
Semi Supervised Learning (SSL)
Overall assumptions; positive and negative theoretical results Graph based methods (Laplacian framework) Label propagation methods Modern Neural Network methods
-
Active Learning
Learning theory Fisher information methods ensemble methods
-
Metric Learning
Formal theory (Mahalanobis methods) Triplet loss function Siamese networks and other modern approaches (embeddings, etc)
-
Neural Networks
Basics (activation functions, computation graphs, backpropagation, loss functions and objective functions) Basics II (standard practices in training/testing; detecting overfitting/underfitting; regularization methods) Convolutional Neural Networks Recurrent Neural Networks/Long Short Term Memory Attention Auxiliary tasks (e.g. multi-headed NN's)
-
Kernel Methods
Reproducing kernel Hilbert spaces Kernel density estimation approximating kernel methods
- Manifold Learning
- Bayesian methods; Gaussian processes; Markov Chain Monte Carlo(MCMC)
- Natural Language Processing (NLP)
- Image Processing/Computer Vision
- Adversarial methods
- Transfer learning (zero shot, few shot, one shot learning)
- Optimization methods