Skip to content

Deep Learning [merged]

Jump to bottom
Yee Whye Teh edited this page Nov 26, 2018 · 1 revision

The meetings are held in the 1st floor meeting room (1.13). We'll meet on a bi-weekly basis on Fridays 1pm-2pm.

Organisers: Hyunjik

Hilary term 2017/18

Date Presenter Topic Materials
23/02/2018 Hyunjik Towards a Neural Statistician arxiv
23/03/2018 Xiaoyu Deep Autoencoding Gaussian Mixture Model for Unsupervised Anomaly Detection pdf
04/05/2018 Valerio A Likelihood-Free Inference Framework for Population Genetic Data using Exchangeable Neural Networks arxiv

Michaelmas term 2017/18

Date Presenter Topic Materials
17/11/2017 Leonard Opening the Black Box of Deep Neural Networks via Information arxiv
3/11/2017 Hyunjik Numerics of GANs arxiv
20/10/2017 Jovana Memory for RL: Model-Free Episodic Control, Neural Episodic Control MFEC,NEC
12/01/2017 Adam K. Metrics for Deep Generative Models arxiv

Michaelmas & Hilary term 2016/17

Date Presenter Topic Materials
15/09/2016 Organisational Meeting
22/09/2016 Hyunjik Introduction & Intuition for Feed-forward NNs notes
29/09/2016 Leonard Introduction to CNNs notes - module 2 of Stanford CS231n course
29/09/2016 Hyunjik Introduction to RNNs notes
06/10/2016 Group Discussion Show, Attend and Tell: Neural Image Caption Generation with Visual Attention pdf
13/10/2016 Kaspar Introduction to Deep Generative Models OpenAI blog, R.Salakhutdinov's talk @ DLSS,S.Mohamed's talk @ DLSS, Ch.20-DL Book
20/10/2016 Stefan Batch Normalization & Layer Normalization BN, LN
27/10/2016 Leon A Theoretically Grounded Application of Dropout in Recurrent Neural Networks https://arxiv.org/pdf/1512.05287v2.pdf
03/11/2016 Group Discussion Composing graphical models with neural networks for structured representations and fast inference pdf
10/11/2016 Tammo Rukat Hierarchical Compositional Feature Learning arxiv
17/11/2016 Qinyi Kernel Analysis of Deep Networks Paper
24/11/2016 Jovana Stories from the MILA Lab
19/01/2017 All NIPS 2016 Review
26/01/2017 Group Discussion Learning in Implicit Generative Models arxiv
2/02/2017 Jovana Causal inference and Deep Learning
9/02/2017 Postponed
16/02/2017 Hyunjik f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization arxiv
16/02/2017 Leonard Wasserstein GAN arxiv
20/02/2017 Xiaoyu Density Estimation using Real NVP arxiv
02/06/2017 Ron Using Dilated Convolutional Models for Regulatory Genomics
TBC Chris One-vs-Each Approximation to Softmax for Scalable Estimation of Probabilities pdf

Introductory material

  • Supervised Learning
  • Feed-Forward Neural Networks
    • Hugo Larochelle's lecture @ Deep Learning Summer School(DLSS) 2016 [slides] [video]
    • Chapter 6 of Deep Learning Book by Goodfellow, Bengio & Courville [pdf]
    • Module 1: Neural Networks of Stanford CS231n course [course website]
  • Convolutional Neural Networks (CNNs)
    • Module 2: CNNs of Stanford CS231n course [course website]
    • Chapter 9 of Deep Learning Book [pdf]
    • A guide to convolution arithmetic for deep learning [pdf]
  • Recurrent Neural Networks (RNNs)
    • Yoshua Bengio's lecture @ DLSS 2016 [slides] [video]
    • Chapter 10 of Deep Learning Book [pdf]
  • Unsupervised Learning
  • Chapter 20 of Deep Learning Book [pdf]
  • Ruslan Salakhutdinov's lecture @ DLSS 2016 [slides] [video]

Papers

(Papers that have been presented previously marked by strikethrough)

  • High-level Review

    • Deep Learning [pdf]

  • Deep Generative Models

    • Auto Encoding Variational Bayes (Variational Autoencoder) [arxiv]
    • Stochastic Backpropagation and Approximate Inference in Deep Generative Models [arxiv]
    • Auxiliary Deep Generative Models [arxiv]
    • NIPS 2016 Tutorial:Generative Adversarial Networks[arxiv]
    • Training Generative Neural Networks via Maximum Mean Discrepancy optimization [arxiv]
    • Adversarial Autoencoders [arxiv]
    • f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization [arxiv]
    • InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets [arxiv]
    • Bidirectional Helmholtz Machines [arxiv]
    • Composing graphical models with neural networks for structured representations and fast inference [arxiv]
    • A Note on the Evaluation of Generative Models [arxiv]
    • A Test of Relative Similarity for Model Selection in Generative Models [arxiv]
    • Towards Conceptual Compression [arxiv]
    • Pixel Recurrent Neural Networks [arxiv]
    • Conditional Image Generation with PixelCNN Decoders [arxiv]
    • Sequential Neural Models with Stochastic Layers [arxiv]
    • Variational Autoencoder for Deep Learning of Images, Labels and Captions [arxiv]
    • Learning in Implicit Generative Models [arxiv]
    • Attend, Infer, Repeat: Fast Scene Understanding with Generative Models [arxiv]
    • Early Visual Concept Learning with Unsupervised Deep Learning [arxiv]
    • Learning Deep Parsimonious Representations [pdf]
    • Disentangling factors of variation in deep representation using adversarial training [pdf]
    • Density estimation using Real NVP [arxiv]
    • Variational Lossy Autoencoder [pdf]
    • Autoencoding beyond pixels using a learned similarity metric [arxiv]
    • Wasserstein GAN [arxiv]
    • Adversarial Variational Bayes: Unifying Variational Autoencoders and Generative Adversarial Networks [arxiv]
    • Nonparametric Variational Auto-encoders for Hierarchical Representation Learning [arxiv]
    • Learning to Discover Cross-Domain Relations with Generative Adversarial Networks [arxiv]

  • Bayesian Deep Neural Networks

    • Bayesian Dark Knowledge [arxiv]
    • Weight Uncertainty in Neural Networks (Bayes by Backprop) [arxiv]
    • Probabilistic Backpropagation for Scalable Learning of Bayesian Neural Networks [arxiv]
    • Structured and Efficient Variational Deep Learning with Matrix Gaussian Posteriors [arxiv]

  • Modelling Stochastic Neural Networks with discrete units

    • Reweighted Wake-Sleep [arxiv]
    • Variational Inference for Monte Carlo Objectives (VIMCO) [arxiv]
    • MuProp: Unbiased Backpropagation for Stochastic Neural Networks [arxiv]

  • Optimization

    • Batch normalization: Accelerating deep network training by reducing internal covariate shift [arxiv]
    • Layer normalization [arxiv]
    • Adam: A Method for Stochastic Optimization [arxiv]
    • A Theoretically Grounded Application of Dropout in Recurrent Neural Networks [arxiv]
    • Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networks [arxiv]

  • Notable Applications & Models

    • Image Classification : ImageNet Classification with Deep CNNs [pdf]
    • Video Classification: Large-scale Video Classification with CNNs [pdf]
    • Speech Recognition : DNNs for Acoustic Modeling in Speech Recognition [pdf]
    • Image Captioning: Show, Attend and Tell: Neural Image Caption Generation with Visual Attention [arxiv]
    • Question Answering: Teaching Machines to Read and Comprehend [arxiv]
    • Visual Question Answering: Ask Your Neurons: A Neural-based Approach to Answering Questions about Images [arxiv]
    • Machine Translation: Neural Machine Translation by Jointly Learning to Align and Translate [arxiv]
    • Neural Turing Machines [arxiv]
    • Memory Networks [arxiv]

  • Others

    • One-vs-Each Approximation to Softmax for Scalable Estimation of Probabilities [pdf]
    • Do Deep Nets Really Need to be Deep? [arxiv]
    • The Loss Surfaces of Multilayer Networks [arxiv]
    • Avoiding Pathologies in Very Deep Networks [arxiv]
    • Conditional Computation in Neural Networks for faster models [arxiv]
    • Learning Deep Nearest Neighbor Representations Using Differentiable Boundary Trees [arxiv]
    • "Why Should I Trust You?": Explaining the Predictions of Any Classifier [arxiv]
    • Domain-Adversarial Training of Neural Networks [arxiv]
    • PathNet: Evolution Channels Gradient Descent in Super Neural Networks [arxiv]
    • Evolution Strategies as a Scalable Alternative to Reinforcement Learning [arxiv]
    • Deep Photo Style Transfer [arxiv]
Clone this wiki locally