Hopfield Networks are a specific class of recurrent neural networks where the nodes and edges represent a complete graph - every node is connected to every other one. Each node has a value of either +1 or -1, and each pair of nodes has a numberical weight.

In the update procedure of a Hopfield Network, one node is randomly selected to be updated. Then values of each of the other nodes in the network are multiplied by the associated edge weights between that node and the updating node. If the sum of these values is positive, the updating node's value becomes +1, otherwise, it becomes -1. By repeatedly performing this update step, the overall network can "evolve" over time.

This relatively simple network can be trained to "remember" a specific pattern of states. This sort of "memory" is expressed by the ability for the network to evolve back towards the remembered state. The training process used is surprisingly simple: In order to store the network's current state, set each edge weight to the product of its two endpoint nodes' values. From an intuitive perspective, this sort of makes sense: If two nodes have the same value, their product will be +1, and if they have different values, their product will be -1. This effectively correlates nodes of the same value and decorrelates nodes of different value.

This capacity to remember becomes even more interesting when multiple patterns are stored in the same network. To store multiple memories, simply average the weight distributions of each of the individual memories. Instead of now always evolving towards the one memorized state, the network in this configuration will evolve towards the most similar saved memory. This system is very robust and can efficiently reconstruct full patterns from small pieces of them - this is similar to the ability to reconstruct an entire song from just hearing the first few lines.