Skip to content

Breakout RL experiments#146

Merged
djsaunde merged 9 commits intoBindsNET:masterfrom
dee0512:master
Oct 28, 2018
Merged

Breakout RL experiments#146
djsaunde merged 9 commits intoBindsNET:masterfrom
dee0512:master

Conversation

@dee0512
Copy link
Copy Markdown
Collaborator

@dee0512 dee0512 commented Oct 26, 2018

Major changes:

  1. Make the softmax decision on "sum of spikes" using spike monitors.
  2. Add 2 new breakout experiments.

else:
self.history = {}

# monitor spikes for selecting action based on sum of spikes
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems like this shouldn't be done in the Pipeline object, but outside in an experimental script. If the layers of the network are really large, for example, this could make things really slow.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. You are right. I will make the change.

@@ -0,0 +1,56 @@
import torch
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the purpose of this script? We want to have only a few examples in the BindsNET repo so we don't have to maintain a bunch.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Hananel-Hazan wanted an experiment for reinforcing only the right action via STDP. This was to see if it performs well enough to include the results in the paper. At this point, I am not really sure which one we should use. The breakout.py example performs the best. Which is not saying much because it has no learning.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it, thanks. It might be better to keep this script in the bindsnet_experiments repo.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok. I will remove it for now. Since I am not sure about the utility of this experiment.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It should be in directory with the rest of the benchmark scripts and example scripts that will appear in the paper for Frontier. @djsaunde will create a spacial tag for it

Copy link
Copy Markdown
Collaborator

@djsaunde djsaunde left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See my comments!

@djsaunde djsaunde merged commit da730bc into BindsNET:master Oct 28, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants