Breakout STDP having low rewards

[jethrokuan]

Hi!

I'm particularly interested in spiking neural networks in the reinforcement learning framework, so I thought I'd run the breakout example in breakout_stdp.py. Running the script, I get the results:

Episode 95 total reward:1.0
Episode 96 total reward:0.0
Episode 97 total reward:2.0
Episode 98 total reward:1.0
Episode 99 total reward:4.0

Is this to be expected? What's the mean/variance of the reward values I should be expecting? I'm looking to get something working so I can establish some baselines. Thanks!

[Hananel-Hazan]

Hello, thank you for using BindsNET. The code breakout_stdp.py demonstrates how to use a spiking network to play an ATARI game.
The results that you reported are normal and can be expected from a random choice of the untrained spiking network.

Currently, it is not an easy task for training spiking neurons to perform well in the RL environment.
However, in this paper, we show a way to train regular neuronal network and convert it to the spiking network.