New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Learning matches Nengo learning more closely #139

Merged
merged 5 commits into from Jan 16, 2019

Conversation

Projects
None yet
3 participants
@hunse
Copy link
Contributor

hunse commented Nov 5, 2018

This improves both the match between emulator and Loihi learning, and emulator and Nengo learning. The result is that everything matches Nengo better.

Based off of #137. Replaces #98.

TODO: I haven't tested this in extreme cases where we actually see overflow in the weights. I think some of these cases were referenced in #98. I (or someone) should look at these cases.

@hunse hunse force-pushed the learn-nengo branch 2 times, most recently from 1bc9372 to 57ba8df Nov 6, 2018

@hunse

This comment has been minimized.

Copy link
Contributor Author

hunse commented Nov 6, 2018

Okay, I fixed the problem from #89 with overflowing (when initializing test_multiple_pes with lambda x: 0), and added a test for it. That doesn't actually test weight overflow; what we were encountering was other overflows (specifically in q0, the input accumulator). To get weights to overflow, I think the easiest way is to set the wgtExp for learning connections (which is always 4) down to e.g. 0. However, we currently don't have a way to set that locally for a specific test, so we couldn't add a unit test for that.

@tbekolay tbekolay force-pushed the master branch from 701f997 to 1fe237c Nov 11, 2018

@hunse hunse force-pushed the learn-nengo branch from 57ba8df to 3dd3b3f Dec 7, 2018

@hunse hunse referenced this pull request Dec 21, 2018

Merged

Refactor files, emulator, api #159

7 of 7 tasks complete

@hunse hunse assigned hunse and unassigned hunse Jan 8, 2019

@hunse hunse force-pushed the learn-nengo branch from 87896d5 to 6934a0d Jan 9, 2019

@hunse

This comment has been minimized.

Copy link
Contributor Author

hunse commented Jan 9, 2019

I tried using the config system to set the weight exponent for learning connections. It wasn't pretty, and finally I gave up when core Nengo tests were failing because they added SPA-specific parameters which I then tried to copy but didn't exist in the new config objects because they weren't initialized. You can see the progress here.

So now I've switched to just having an attribute on the Model to set this (see the new test). We can revisit the config stuff later if we wish. Fingers crossed everything passes now.

@drasmuss
Copy link
Contributor

drasmuss left a comment

Looks good overall, mainly just documentation/testing questions! It'll definitely be nice to have the learning matching more closely (and nice to have the math worked out explicitly rather than relying on some approximations we worked out heuristically).

Show resolved Hide resolved nengo_loihi/loihi_interface.py Outdated
Show resolved Hide resolved nengo_loihi/builder.py Outdated
Show resolved Hide resolved nengo_loihi/loihi_api.py
Show resolved Hide resolved nengo_loihi/loihi_api.py
Show resolved Hide resolved nengo_loihi/loihi_api.py Outdated
Show resolved Hide resolved nengo_loihi/loihi_api.py Outdated
Show resolved Hide resolved nengo_loihi/loihi_cx.py
Show resolved Hide resolved nengo_loihi/loihi_cx.py
Show resolved Hide resolved nengo_loihi/tests/test_learning.py Outdated
Show resolved Hide resolved nengo_loihi/tests/test_learning.py Outdated
@hunse

This comment has been minimized.

Copy link
Contributor Author

hunse commented Jan 10, 2019

Ok, I think I've addressed all of Dan's comments.

@drasmuss

This comment has been minimized.

Copy link
Contributor

drasmuss commented Jan 10, 2019

Everything looks good on the emulator, but test_pes_comm_channel is failing for me on hardware.

@hunse

This comment has been minimized.

Copy link
Contributor Author

hunse commented Jan 11, 2019

Ok, I've fixed test_pes_comm_channel on hardware.

As part of that, I removed the fudge factor in the learning overflow. This had originally been to make the emulator more like the chip, but I don't think it actually did. It made the chip learn significantly more slowly than the emulator. With it removed, the chip learns slightly faster. I'm still not sure why this is; it's possible it has to do with how stochastic rounding happens on the chip. Anyway, the emulator and chip are closer now, so overall it's a win I think.

The two new commits can probably get squashed into existing ones if we want. I just kept them separate for now.

Finally, as part of our testing for this, we should probably look at the adaptive control notebook and check it on the emulator and the chip.

@drasmuss
Copy link
Contributor

drasmuss left a comment

I added a commit to fix the adaptive control notebook, and added a changelog entry.

I can't connect to the hardware at the moment, but I trust you that tests are passing there now 😄 . For the same reason I couldn't test the adaptive control notebook on hardware.

I just had one more question up above about the change to wgtExp. Not blocking or anything, that's just for my own curiosity.

Assuming that all looks good to you, this looks good to me.

@drasmuss drasmuss removed their assignment Jan 11, 2019

@drasmuss

This comment has been minimized.

Copy link
Contributor

drasmuss commented Jan 11, 2019

Confirmed that the tests pass and notebook looks good on hardware 👌

@drasmuss drasmuss self-assigned this Jan 11, 2019

@drasmuss drasmuss force-pushed the learn-nengo branch from 15c1dd9 to d56f64f Jan 11, 2019

@drasmuss drasmuss removed their assignment Jan 11, 2019

@hunse hunse force-pushed the learn-nengo branch 2 times, most recently from 673bd67 to 9cb8614 Jan 14, 2019

@hunse

This comment has been minimized.

Copy link
Contributor Author

hunse commented Jan 14, 2019

Ok, I think I've tested almost all the new added lines.

There's one emulator error for spikes being dropped in the trace that is not being tested. I'm not quite sure how to test this: I think we'd need some sort of input that's spiking really fast (possibly just having a number of neurons converge on the same input axon in the post population would do it). I'd definitely want to do some sort of test that makes sure the emulator matches the chip in this respect, so I think for now I'd recommend just leaving this untested.

There's also a line in discretize_weights for lossy_shift = False that isn't tested. I mostly added this in to do quick comparisons of what things would be like if the chip didn't do this. The current chip does do this, though, so it's not tested without anywhere. Again, I'm inclined to say that's fine.

@drasmuss
Copy link
Contributor

drasmuss left a comment

This all looks good to me now! I'll clean up the history and re-run the tests on hardware, then this should be good to merge.

@drasmuss drasmuss self-assigned this Jan 15, 2019

@drasmuss drasmuss force-pushed the learn-nengo branch 2 times, most recently from bd27bcd to d66394d Jan 15, 2019

@drasmuss drasmuss removed their assignment Jan 15, 2019

@tbekolay tbekolay assigned tbekolay and unassigned tbekolay Jan 15, 2019

hunse and others added some commits Oct 17, 2018

Check learning axon indices
Before, we were checking that the number of learning axons was
greater than the first learning index, which doesn't make sense.
Avoid overflow on zero-weight connections
By reducing `vth_max` for these connections.

This is most common on learning connections with zeroed
initial weights, so we'll test in `test_multiple_pes`.
Match learning between Nengo/emulator/chip
This commit ensures that both the emulator and chip match Nengo
in terms of basic PES learning. The emulator now does learning
much more accurately, which allows us to better map PES learning
parameters (i.e., learning rate, synapse decay) to the chip.

This commit also makes several changes to learning tests:

- The PES learning test has been simplified and explicitly
  compares PES learning on Nengo Loihi with core Nengo.
- `test_multiple_pes` was fixed. It was not passing when using
  Nengo as the simulator, so it had been designed with the old
  Loihi learning rates in mind and needed fixing now that Loihi
  learning is closer to Nengo learning.
- Several test tolerances have been adjusted.
  Note that some tolerances were changed for the chip; in general,
  the chip seems to learn slightly faster than the emulator
  (I'm not quite sure why), but this difference seems to be less
  apparent for more dimensions/neurons.

And adds additional tests:

- Add test for PES error clipping.
- Add test for learning overflow.
- Add test for PES learning with 'simreal' emulator.
- Add test for trace increment clip warning.
- Add test for discretize_weights.lossy_shift.
- Add test for learning trace dropping

Co-authored-by: Daniel Rasmussen <daniel.rasmussen@appliedbrainresearch.com>

@tbekolay tbekolay force-pushed the learn-nengo branch from d66394d to 32f8bd6 Jan 16, 2019

@tbekolay tbekolay merged commit 32f8bd6 into master Jan 16, 2019

1 of 2 checks passed

codecov/patch 95.76% of diff hit (target 100%)
Details
codecov/project 80.39% (+1%) compared to 3cd5bac
Details

@tbekolay tbekolay deleted the learn-nengo branch Jan 16, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment