100% coverage by hunse · Pull Request #185 · nengo/nengo-loihi

hunse · 2019-02-06T19:10:10Z

Get 100% coverage in codecov.

Cover all BuildErrors and other exceptions
Remove unused code
Test code that should not fail in tests (e.g. chip connection/communication) using mocks
Cover any remaining uncovered lines, or mark as "no cover" (e.g. failed imports)

hunse · 2019-02-08T21:45:09Z

OK, I think I've got everything covered, except for the few cases that we've explicitly talked about.

Currently, this is still a lot of commits. I've organized them into added tests first, then code removals, then other stuff. I've kept them separate for now (I'm thinking it will be easier to review that way), but they can mostly be squashed before merging (maybe squashing all the tests into one commit, and most of the removals into another). The two removals we should keep separate IMO are the first two in the order right now: e29050a and dfa4da5. The first removes the NIF neurons and replaces them with SpikingRectifiedLinear for our node encoding. The second gets rid of probes from LoihiInput, which we don't use right now, but may want to add back in in the future to be able to probe parts of host->chip connections.

I also think we should keep the "other stuff" mostly separate (i.e. things that are not tests or removals), since they're all fairly different changes and some have potential functional consequences.

This is also based off #140 right now (to fix the tests), so we should merge that first.

hunse · 2019-02-15T15:47:12Z

One thing we haven't discussed is conftest.py. It technically has one uncovered line. I would recommend removing it from coverage entirely.

drasmuss

Pushed some fixups, then just one other question about test coverage.

nengo_loihi/emulator/interface.py

pytest.ini

nengo_loihi/hardware/builder.py

nengo_loihi/tests/test_inputs.py

nengo_loihi/tests/test_simulator.py

drasmuss

Looks good to me!

Need to add changelog entries before merging, but Eric is working on that now.

hunse · 2019-02-20T15:38:27Z

I've added changelog entries. There were only four commits that I thought deserved them.

This was accidentally lost in 9b68839 during refactoring.

- Test validation BuildErrors - Test splitter errors and other untested lines - Test ProbeDict interface - Test Simulator interface - Test Passthrough node removal transform errors - Test LoihiBlock.compartment configuration errors - Test error for no appropriate weight exponent - Test error for no appropriate bias scaling - Test loihi_rates with non-Loihi neuron - Test LoihiBlock too big to fit on one core - Test ChipReceiveNode run error - Test ensemble building errors - Test discretize noise amplitude warnings - Test Model.build_callback - Test zero activity error in connection - Test chip->host connections defined by points - Test block.py object string functions - Test nxsdk_objects.py object strings - Test PESModulatoryTarget interface - Test emulator states dtype errors - Test Loihi simulation exception handling Currently, this exception handling does not work properly, so this test hangs. We need to keep track of how many steps have executed, and perform the exact number of chip reads/writes required to finish the simulation. - Test unrecognized probe target type error - Ensure that we target specific warnings in tests

Also changed the error messages to make them distinguishable and able to report non-integer pop types (which are not supported). Replace untested pop_type error with assert

As part of this, I discovered that if `n2Board` is None, we raise a SimulationError but still try to call `n2Board.disconnect` if we're in a context manager. So now we only call that if `n2Board` is not None.

By running first for some time, then again for more time. This tests a number of previously uncovered lines.

This was only used for generating spikes to send to the chip, which can be accomplished equally well with SpikingRectifiedLinear.

LoihiInput is always on the chip, connecting in from a Node. If a user probes the node, though, the probe will happen off-chip. So it's currently impossible to probe these on-chip. This may be possible in the future if/when we set up connection probing for host->chip connections.

- Remove StdpProfile and StdpPreProfile While we configure these in the hardware builder, we always configure them the same for PES, so these configuration classes are not needed. - Remove unused axon names from LoihiBlock - Remove unused SpikeInput.clear_spikes - Remove unused Synapse.set_diagonal_weights - Remove unhittable signal probe function error - Remove unused helper functions in IterableState - Remove unused default for Interface.chip2host - Remove unused HardwareInterface.print_cores - Remove unused Block.cores attribute

Mostly, this is acknowledging cases that can never happen. - Earlier error if using direct neurons in builder. - Make sure increment size assert gets executed every time, and give it a better message. - Simplify connection builder NotImplemented block to not check specially for nodes. - Spike probe targets can only be LoihiBlock, so just assert this. - Only warn about noise amplitude if it's enabled, to avoid unnecessary warnings. - Check that all synapses on a core are either learning or not learning (but not a mix of both), because our current allocator does not support this. When we do, we'll need to properly test it, including the uncovered lines that are commented out in this commit. - Can never have zero axons in hardware builder, so just assert it.

A negative cx_base value is sometimes helpful to indicate an axon that is not required when making convolutional connections. The chip itself does not support negative cx_base values, so we just remove this axon when building the model. We used to use a value of -2048 to indicate this, but this led to the emulator misleadingly allowing smaller negative values. Now, we just use -1 to indicate an unused axon, and give an error for other values. Also added a test of negative cx_base values working as expected.

It now passes reliably on chip, likely due to new interneurons.

These lines depend on how we have the tests set up and hitting them is not important.

Reorganize the test files to more closely reflect the non-test file structure. - Moved test_communication.py -> test_connection.py - Moved test_ens_on_host.py -> test_connection.py - Moved test_model.py -> test_simulator.py - Moved test_multiple.py -> test_node.py - Moved test_precompute.py -> test_simulator.py - Moved test_radius.py -> test_ensemble.py - Moved test_snips.py -> test_simulator.py - Moved emulator/tests/test_emulator.py -> emulator/tests/test_interface.py - Moved hardware/tests/test_simulator.py -> hardware/tests/test_interface.py - Moved test_simulator.py::test_snip_input_count -> hardware/tests/test_interface.py

The chip allows noise exponents down to zero; this now works on the emulator as well. However, a known abnormality in shifting on the chip means that noise exponents less than 7 act as one greater than they should. This commit addresses that problem by subtracting one from weight exponents before putting them on the chip. We also improve the test for noise, so that it tests different levels of noise and asserts that the statistics of the resulting Wiener process are as expected.

tbekolay

History looks great, I only had to fix one test (test_split_conv2d_transform_error) post-rebase as it changed due to #142. Merging once CI finishes.

hunse force-pushed the coverage100 branch 7 times, most recently from 5ee4dcc to 6a3b99d Compare February 8, 2019 21:02

drasmuss force-pushed the coverage100 branch from c66bfc7 to 6ad3949 Compare February 19, 2019 15:25

drasmuss reviewed Feb 19, 2019

View reviewed changes

nengo_loihi/emulator/interface.py Show resolved Hide resolved

hunse commented Feb 19, 2019

View reviewed changes

pytest.ini Show resolved Hide resolved

hunse commented Feb 19, 2019

View reviewed changes

nengo_loihi/hardware/builder.py Outdated Show resolved Hide resolved

drasmuss force-pushed the coverage100 branch from d9793ba to f40bf9d Compare February 19, 2019 20:01

hunse commented Feb 19, 2019

View reviewed changes

nengo_loihi/tests/test_inputs.py Outdated Show resolved Hide resolved

drasmuss force-pushed the coverage100 branch 2 times, most recently from d738b23 to 4f40b08 Compare February 19, 2019 21:49

hunse force-pushed the coverage100 branch from 348007d to cf0b95e Compare February 20, 2019 05:18

drasmuss reviewed Feb 20, 2019

View reviewed changes

nengo_loihi/tests/test_simulator.py Show resolved Hide resolved

drasmuss approved these changes Feb 20, 2019

View reviewed changes

hunse force-pushed the coverage100 branch from 9c53d1e to acee2f3 Compare February 20, 2019 15:37

Check LoihiBlock probe target

02333c1

This was accidentally lost in 9b68839 during refactoring.

tbekolay force-pushed the coverage100 branch 2 times, most recently from 135a212 to 7716ba6 Compare February 21, 2019 23:25

tbekolay mentioned this pull request Feb 22, 2019

Only collect coverage for nengo_loihi #191

Closed

hunse and others added 4 commits February 21, 2019 20:12

Test board builder pop type errors

a9d758b

Also changed the error messages to make them distinguishable and able to report non-integer pop types (which are not supported). Replace untested pop_type error with assert

Test errors for failing to connect to the board

8242401

As part of this, I discovered that if `n2Board` is None, we raise a SimulationError but still try to call `n2Board.disconnect` if we're in a context manager. So now we only call that if `n2Board` is not None.

Test simulator picking up where it left off

cfb68f6

By running first for some time, then again for more time. This tests a number of previously uncovered lines.

hunse and others added 12 commits February 21, 2019 20:12

Test for cluster errors in passthrough.py

a7bd368

Fix and test Model string

4896447

Remove NIF neuron type

1ff15cf

This was only used for generating spikes to send to the chip, which can be accomplished equally well with SpikingRectifiedLinear.

Remove test_input_node xfail

18420dc

It now passes reliably on chip, likely due to new interneurons.

Run nengo RefSimulator tests

1769f55

Mark specific lines as not covered

ef2eb49

These lines depend on how we have the tests set up and hitting them is not important.

tbekolay force-pushed the coverage100 branch from 7716ba6 to 40981e4 Compare February 22, 2019 01:18

tbekolay approved these changes Feb 22, 2019

View reviewed changes

tbekolay merged commit 40981e4 into master Feb 22, 2019

tbekolay deleted the coverage100 branch February 22, 2019 02:46

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

100% coverage#185

100% coverage#185
tbekolay merged 17 commits intomasterfrom
coverage100

hunse commented Feb 6, 2019 •

edited by tbekolay

Loading

Uh oh!

hunse commented Feb 8, 2019

Uh oh!

hunse commented Feb 15, 2019

Uh oh!

drasmuss left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

drasmuss left a comment

Uh oh!

hunse commented Feb 20, 2019

Uh oh!

tbekolay left a comment •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Milestone

Development

Uh oh!

3 participants

Conversation

hunse commented Feb 6, 2019 • edited by tbekolay Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

hunse commented Feb 8, 2019

Uh oh!

hunse commented Feb 15, 2019

Uh oh!

drasmuss left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

drasmuss left a comment

Choose a reason for hiding this comment

Uh oh!

hunse commented Feb 20, 2019

Uh oh!

tbekolay left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Milestone

Development

Uh oh!

3 participants

hunse commented Feb 6, 2019 •

edited by tbekolay

Loading

tbekolay left a comment •

edited

Loading