This project is a Python 3.8+ implementation of Numenta's HTM algorithm along with visualization software. The aim of this project is to both understand the implementation of cortical theory while creating a simplified cortical learning system.
Join the community here
This project has been moved to Oktopus.io repo
No-stake script to play around with the different components. You will also find a list with all the tuning parameters.
SDR stands for Sparse Distributed Representations. In this document you will find all methods related to SDR ( visualization, creation, operations... )
Learn more about SDR here
Classes to convert semantic meaning of input data into SDR Encoders' rules are described within the file.
Stands for Spatial Pooler. Here you will find the most relevant classes of cortical theory ( the Neuron, the miniColumn, the Connections and why not? The SpatialPooler)
You can also find a well compressed summary of cortical theory here
Cortical columns are the basic processing unit of the neocortex. They all have a universal learning algorithm which its understanding and implementation is the main goal of HTM. The brain is made of around 150,000 cortical columns connected in a hierarchical way.
Mini-columns are vertical structures through the 6 layers of cortex comprising of about 80-120 neurons. A neuron's feed forward input, the proximal dendrite, attaches to a receptive field, or particular area of sensory space in which stimulus will trigger the firing of the neuron. Neurons in a minicolumn have the same receptive field. Therefore in the HTM algorithm for organizational purposes, each column stores proximal dendrites rather than each neuron in a column.
The structure of an HTM cell, or neuron, is modeled after pyramidal neurons in the neocortex. In a real neuron when it is in an active state the axon outputs a short burst of action potentials. When it is in a predictive state the axon outputs a slower, steady rate of action potentials. These action potentials are only modeled with a binary "on" or "off" for a cell's state.
- Proximal Dendrite Segment: recieve feedforward input and used to put a neuron in an active state.
- Apical Dendrite Segments: recieve feedback from higher cortical layers and used to put a neuron in a predictive state.
- Basal Dendrite Segments: recieve lateral input from nearby neurons and is used to put a neuron in a predictive state.
- Potential Synapses: Represents all the axons that pass close enough to a dendrite segment that they could potentially form a synapse.
- Permanance: A scalar value representing the range of connectedness between an axon and a dendrite. If the permanance is above a certain threshold, the synapse is connected.
The spatial pooling function operates on a layer of a cortical region. It uses each column's proximal dendrite input to return active columns that reflect the greatest connectivity with that layer's input.
- Potential Pool: Each column sees a random subset of the input bits.
- Overlap Score: How many of column's connected synapses(permanance above threshold) are binary 1.
- Inhibition: enforce sparcity by selecting active columns from the top overlap scores.
- Learning: Look at all the active columns and strengthen or weaken connections.
- Output: active columns
The temporal memory function operates at the neuronal level by by using each neuron's basal dendrite input. Apical dendrites have not been implemented in this version of the algorithm.