Information Processing and the Brain 2020/2021

Here you can find the relevant content for Information Processing and the Brain 2020/2021. This unit covers several aspects of information processing in the brain, such as sensory processing, probabilistic codes, deep learning, recurrent neural networks, credit assignment, reinforcement learning and model-based inference.

It is jointly taught by Conor Houghton and Rui Ponte Costa at the Department of Computer Science [School of Computer Science, Electrical and Electronic Engineering, and Engineering Mathematics], Faculty of Engineering, University of Bristol.

You should go to the io page for more info: comsm0075.github.io

TAs: Joe Pemberton and Dabal Pedamonti.

Recommended reading:

This field is highlight interdisciplinary, as such there is no single textbook that covers all our lectures. Relevant research papers will be highlighted during the lectures. However, below we highlight in bold the most relevant ones for this unit.

Theoretical neuroscience:

1. Theoretical Neuroscience by P Dayan and L F Abbott (MIT Press 2001), see also errata.
2. Neuronal Dynamics by Wulfram Gerstner, Werner M. Kistler, Richard Naud and Liam Paninski. Full version online.
3. Introduction To The Theory Of Neural Computation, Volume I by John Hertz. (Classical and accessible book on neural computation)
4. Bayesian Brain: Probabilistic Approaches to Neural Coding
5. Elements of Information Theory by TM Cover and JA Thomas (Wiley), worth owning, but there is an online pdf from www.cs-114.org

Machine/statistical Learning:

1. General ML book: Information Theory, Inference and Learning Algorithms by David MacKay. Full version available online
2. Deep Learning (including Recurrent neural nets): Deep Learning by Ian Goodfellow, Yoshua Bengio and Aaron Courville
3. Unsupervised learning: Natural Image Statistics by Aapo Hyvarinen, Jarmo Hurri, and Patrik O. Hoyer. Full version available online.
4. Reinforcement learning: Reinforcement Learning: An Introduction by Richard S. Sutton and Andrew G. Barto. Full version available online.

Super useful math/stat cheat-sheet by Iain Murray: homepages.inf.ed.ac.uk/imurray2/pub/cribsheet.pdf

Draft schedule

Conor (weeks 1-3):

Rui (weeks 4-7): Neural circuits and learning

Coursework (week 8-10):

...TBC

Revision week (week 11):

Formative labs:

Rui: 1. Biological plausability of backprop 2. Reinforcement learning: TD