The Forward-Forward algorithm is a learning procedure for neural networks that replaces the forward and backward passes of backpropagation by two forward passes, one with positive data and the other with negative data. Each layer has its own objective function which is simply to have high goodness for positive data and low goodness for negative data. The sum of the squared activities in a layer can be used as the goodness but there are many other possibilities.
For more information, see: The Forward-Forward Algorithm: Some Preliminary Investigations
I have implemented the Forward-Forward algorithm for PyTorch, while attempting to stay faithful to the details outlined in the research paper. If you encounter any discrepancy or issue, please do not hesitate to contact me.
- Forward Forward Algorithm - Unsupervised
- Forward Forward Algorithm - Supervised
- Forward Forward Algorithm CNN - Unsupervised
- Forward Forward Algorithm -Unsupervised (not layerwise training)
- Forward Forward MLP - Supervised (not layerwise training)
If you have any questions or advices, or you would like to discuss this work further, please do not hesitate to contact me via email or LinkedIn.