Iterative winners-take-all

This repository contains Python implementation of our paper

FORMATION OF CELL ASSEMBLIES WITH ITERATIVEWINNERS-TAKE-ALL COMPUTATION ANDEXCITATION–INHIBITION BALANCE

available on arxiv at https://arxiv.org/abs/2108.00706.

Quick start

Python 3.6+ is required.

First, install the requirements:

pip install -r requirements.txt

Then you're ready to run the scripts to reproduce the figures in the paper.

For PyTorch implementation, refer to the nn directory.

Should you have any questions, please open an issue or email us.

Reproducing the figures

To reproduce the plots, run python figX.py in a terminal, where figX.py is one of the following:

• fig2.py - how the output populations sparsity depends on the weight sparsity (weights are random fand fixed);
• fig3b.py - habituation;
• fig4.py - clustering;
• fig5.py - dependence on the input density (not shown in the paper);
• fig6.py - similarity preservation (not show in the paper);
• decorrelation.py - decorrelation (not shown in the paper).

This section compiles all figures, listed and supplementary, in one page.

Figure 2 (from the paper). Dependence of the density of encodings on the weight density

The weights are random and fixed.

The figures from the next sections involve learning. Learning the weights means letting the network evolve over time from input stimuli x rather than "training" it from labelled data. In other words, the learning is local and unsupervised.

Habituation

Decorrelation

Clustering

Comparison of iWTA (with permanence-varying learning) and kWTA on a clustering task. Run the fig4.py script to generate this plot.

Complete run of each learning method:

Figure 5. Dependence on the input density

Dependence of the density of encodings on the input density. The main feature is that the dependence is nonlinear and the same for different layer sizes.

Figure 6. Similarity preservation

The figure shows how the cosine similarity between two input vectors changes in the encoding space. The sparsity of encodings influences the dependence. Results for iWTA are comparable to kWTA. Similar inputs are encoded in similar output populations. The higher the encoding density (lower sparsity) the better preservation.