Predictive Coding in Python

A Python implementation of An Approximation of the Error Backpropagation Algorithm in a Predictive Coding Network with Local Hebbian Synaptic Plasticity

[Paper]

Based on the MATLAB implementation from [@djcrw]

Requirements

• numpy
• torch
• torchvision

Tasks

• Include model from A tutorial on the free-energy framework for modelling perception and learning
• Add additional optimisers
• Measure number of iterations
• The initial space of mu needs to be sufficently large - ensembles of amortised weights or slow learning rate?
• Test pure PC accuracy
• Errors go down, but amortised asymptotes
• Infinite iterations - amortised learning provides mechanism for setting number, remove free parameter (replaces with threshold)
• Generative overtakes and this is inconsistent with discriminative - need some way to promote discrimination in network
  • Both trying to predict each other? Generative predicting discrimantive?
• Add gradient clamping