Simple SDR Encoder

#Sparse Distributed Representation (SDR)

Since most of my interest is putting input data streams of integer and float values in the HTM network (instead of pixels), I have created a simple class to generate input SDRs from float values. A good SDR has the following properties:

• Only a small percentage of the cells active for a given value
• Neighboring values have some overlap to aid in classification
• Maximize the resolution of the input value by taking advantage of all cells
• Do not to randomize too much without a good reason (easier to debug)

There is tons of information on SDR creation and encoding on the net. Clearly there are lots of ways of doing this for different applications, the encoding that I created is very simple.

To setup the SDR class, the class needs a few initialization parameters:

• range: The number of cells that the active cells will be contained in
• active: The number of active cells used to represent a value
• XY dimensions: This is the size of the 2D array of cells used
• Max/Min values: The max and min input value.

For this encoder, a representation is created in a 1D array, and then the 1D array space is converted to 2D so it can be applied to the 2D HTM network. In this implementation all data spaces (regions, input, classifiers) assume a 2D space.

The pattern for representing numbers is shown below in the diagram. Basically the active cells are spread across the range and they slowly move across the full set of input cells depending on how large the input value is that we are trying represent. In this system, only a certain number of representation can be made, this is called the resolution_steps, the number of unique values that can be represented with the given input values. In this encoder, a max and min value are set at initialization, then each input value is assigned to the closest representation that the resolution of the SDR supports. A certain amount of quantization error will be occurred on input, and it can be adjusted by changing the setup params of the SDR encoder.

The diagram below shows how the first 9 values are encoded to an SDR with these params:

• active: 3
• range: 6
• X/Y: 12x12

In this example you can see the pattern that is being formulated, for more detail, look into the source code (sdr.cpp). In this example, 142 unique values can be encoded in the 144 cell matrix. Granted, there isn't much activation, and not much overlap, but this shows the mechanism. Larger range and active values can create SDR with much more overlap, and a larger number of values that can be represented.

Since we would like to have some overlap with neighboring values, the table below shows the amount of overlap for the given values.

This overlap amount is an important value going forward given that HTMs will have some amount of error in the predictions. This will allow us to make a estimated guess at the next input value when we run the "classifier".