Converting some manualtests to tests used by installcheck #774

hakonsbm · 2017-06-30T11:58:41Z

This PR updates and converts some of the tests in the manualtests folder to proper unittests. This includes the test_sinusoidal_poisson_generator_{5,6} tests, and therefore resolves issue #348.

I was unable to convert all the tests because of problems such as missing input files, dependence on external programs, and erroneous output.

heplesser

@hakonsbm Thanks for moving the tests! I have some minor comments, see inline. The two sinusoidal poisson tests contain a note in their documentation saying that they don't work with the testsuite. Those comments can now be deleted :).

heplesser · 2017-06-30T12:17:49Z

pynest/nest/tests/test_facetshw_stdp.py

+        stim = nest.Create('spike_generator')
+        neuronA = nest.Create('parrot_neuron')
+        neuronB = nest.Create('parrot_neuron')
+        # recorder = nest.Create('spike_detector')


Remove unused code.

heplesser · 2017-06-30T12:17:58Z

pynest/nest/tests/test_facetshw_stdp.py

+            'weight': float(startWeight) / 15.0 * Wmax,
+            'delay': delay, 'model': modelName})
+        # nest.Connect(neuronA, recorder)
+        # nest.Connect(neuronB, recorder)


Remove unused code.

heplesser · 2017-06-30T12:19:48Z

pynest/nest/tests/test_facetshw_stdp.py

+
+        # just before theoretical updates
+        weightTraceMod36pre = weightTrace[35::36]
+        weightTraceMod36 = weightTrace[::36]  # just after theoretical updates


The comment for line 123 is on line 122, the one for 124 on the same line.

heplesser · 2017-06-30T12:21:46Z

pynest/nest/tests/test_pp_psc_delta.py

+        self.assertLess(ratio, 1.5)
+
+        isi = []
+        for i in xrange(1, len(spikes)):


xrange disappears in Python 3. A much more efficient way to do this would be using numpy arrays.

heplesser · 2017-06-30T12:22:16Z

pynest/nest/tests/test_pp_psc_delta.py

+        self.assertLess(ratio, 1.5)
+
+        isi = []
+        for i in xrange(1, len(spikes)):


heplesser · 2017-06-30T12:23:11Z

pynest/nest/tests/test_pp_psc_delta.py

+        # reproducible in NEST. Adaptive threshold changes rate, thus
+        # the ratio is not asserted here.
+        isi = []
+        for i in xrange(1, len(spikes)):


heplesser · 2017-06-30T12:23:28Z

pynest/nest/tests/test_pp_psc_delta.py

+        isi_m1 = isi[-1]
+        isi_m2 = isi[-1] ** 2
+        isi_12 = 0.
+        for t, t1 in zip(isi[:-1], isi[1:]):


heplesser · 2017-06-30T12:28:20Z

testsuite/unittests/test_hh_cond_exp_traub.sli

+
+(unittest) run
+/unittest using
+


This test requires GSL.

heplesser · 2017-06-30T12:31:21Z

testsuite/mpitests/test_sinusoidal_poisson_generator_5.sli

@@ -104,15 +104,10 @@ See also: test_sinusoidal_poisson_generator_{1,2,3,4,6}, test_sinusoidal_poisson
  % get events, replace vectors with SLI arrays
  % keep only non-empty arrays; empty ones are from off-process parrots
  (ALL LINES SHOULD BE EQUAL) ==  
-  sdets { [/events /times] get cva } Map { empty not exch ; } Select
+  sdets { [/events /times] get cva } Map { empty not exch ; } Select dup /spike_times Set
  { == } forall


This line just prints all the spike times, filling the log with potentially garbled data from the two processes. I would remove it; then, you don't need the dup /spike_times Set.

heplesser · 2017-06-30T12:31:45Z

testsuite/mpitests/test_sinusoidal_poisson_generator_6.sli

@@ -117,15 +117,10 @@ See also: test_sinusoidal_poisson_generator_{1,2,3,4,6}, test_sinusoidal_poisson
  % get events, replace vectors with SLI arrays
  % keep only non-empty arrays; empty ones are from off-process parrots
  (ALL LINES SHOULD BE DIFFERENT) ==  
-  sdets { [/events /times] get cva } Map { empty not exch ; } Select
+  sdets { [/events /times] get cva } Map { empty not exch ; } Select dup /sts Set
  { == } forall


Removed unnecessary comments, printing; using numpy functions for efficiency; skipping if without GSL where needed.

heplesser

@hakonsbm Thanks for the fixes, just a few details left.

heplesser · 2017-07-02T09:14:41Z

pynest/nest/tests/test_pp_psc_delta.py

-        isi_var = (isi_m2 - isi_m1) ** 2 / len(isi)
+        isi_mean = np.mean(isi)
+        isi_var = np.var(isi)
+        isi_12 = np.sum(np.multiply(isi[:-1], isi[1:]))


isi_12 = np.sum(isi[:-1] * isi[1:])

heplesser · 2017-07-02T09:15:17Z

testsuite/mpitests/test_sinusoidal_poisson_generator_5.sli

@@ -104,10 +98,9 @@ See also: test_sinusoidal_poisson_generator_{1,2,3,4,6}, test_sinusoidal_poisson
  % get events, replace vectors with SLI arrays
  % keep only non-empty arrays; empty ones are from off-process parrots
  (ALL LINES SHOULD BE EQUAL) ==  


This output should also be removed.

heplesser · 2017-07-02T09:15:34Z

testsuite/mpitests/test_sinusoidal_poisson_generator_6.sli

@@ -117,10 +112,8 @@ See also: test_sinusoidal_poisson_generator_{1,2,3,4,6}, test_sinusoidal_poisson
  % get events, replace vectors with SLI arrays
  % keep only non-empty arrays; empty ones are from off-process parrots
  (ALL LINES SHOULD BE DIFFERENT) ==  


This output should also be removed.

jougs

Many thanks for the effort. I sent you a pull requests with minor changes to formatting of the comments. If those are merged, I have no objections to merging this one.

jougs · 2017-07-06T07:41:56Z

@heplesser, @hakonsbm: does this PR also mean that all other tests in manualtests cannot be converted to automatic tests? If yes, should we delete them or rather convert them to examples? Are you planning to work on that?

Minor changes to comments and formatting

jougs · 2017-07-06T08:32:43Z

Sorry, my commit introduced a pep8 error (see Travis log). I've sent another PR fixing this.

The current failure above, however, is not in the code touched by this PR or my changes but in the compilation of MUSIC. It happens when the make target pyconfig.pxi runs the script test.py, which tries to import mpi4py, which apparently fails with ImportError: No module named mpi4py. I don't understand how this can happen, as we install mpi4py as we always did.

Maybe a simple re-run helps...

pep8ify

hakonsbm · 2017-07-06T08:47:43Z

@jougs: @heplesser and I just went though the rest of the tests to see which can be fixed/converted to examples and which should be deleted. @stinebuu and I will work on it.

hakonsbm · 2017-07-06T10:04:38Z

My last commit message was a bit unclear. The tests deleted are tests that are now irrelevant.

lekshmideepu · 2017-07-06T13:38:31Z

travis-ci/travis-ci#8048

lekshmideepu · 2017-07-07T05:31:54Z

Probably we could switch back to the old images by adding the group: deprecated-2017Q2 instead of "edge"
https://blog.travis-ci.com/2017-06-21-trusty-updates-2017-Q2-launch

hh_phaseplane.py example.

jougs · 2017-07-17T20:47:05Z

@hakonsbm: I've just merged #785, which makes our setup use the old image. Please revert 2e8d285 and merge upstream/master into this branch. Thanks!

heplesser · 2017-08-09T13:48:47Z

@jougs @hakonsbm All tests pass now on Travis. Does this mean that this PR is ready to be merged?

hakonsbm · 2017-08-09T14:09:18Z

@heplesser Some of the tests were missing reference values, or had other problems. @stinebuu and I are still waiting for replies from the authors of these tests, requesting guidance as to how they can be converted, or if they can be deleted.

hakonsbm · 2017-08-21T11:53:40Z

The remaining problematic tests in the manualtests directory are now

cross_check_test_mip_corrdet.py
stdp_dopa_check.py
test_stdp_dopa.py

The SLI modifications are now discussed in issue nest#811.

jougs · 2017-08-21T16:26:58Z

@suku248, @abigailm: can you please have a look at the *dopa* tests mentioned in @hakonsbm's comment and check if they a) are still needed and b) should be converted to proper automatic tests?

@mhelias: can you please do the same for cross_check_test_mip_corrdet.py? In particular, does this test add anything on top of test_corr_det.sli in the unittests directory?

Thanks!

suku248 · 2017-09-05T15:45:36Z

stdp_dopa_check.py and test_stdp_dopa.py are outdated and can be removed. A proper automatic test is required.

heplesser · 2017-09-05T19:58:14Z

@suku248 Could you create an issue to remind us to create proper tests for dopa?

mhelias · 2017-09-08T07:29:34Z

As far as I see the
test_mip_corrdet.sli does not add anything to the existing test
test_corrdet.sli.
But it is a useful example how to use the mip generator and to
verify that it produces correlated spike trains. So I propose to move
it to examples.

converted cross_check_mip_corrdet to PyNEST example

…tor.git into manualtests

compatible.

Pep8ify, removed manualtest from testsuite cmakelist.

hakonsbm · 2017-09-20T12:53:14Z

@jougs The cross_check_mip_corrdet.py test has now been converted to an example, and the rest of the tests have been removed from the manualtests directory.

heplesser · 2017-10-03T21:26:16Z

@jougs I added this to 2.14, since it appears done and ready to merge.

jougs

Many thanks for this important clean-up :-)

hakonsbm added 2 commits May 9, 2017 17:01

Updated and moved some of the tests from "manualtests" folder

1bbf42c

Fixed PEP 8 errors

db52266

heplesser requested review from heplesser and jougs June 30, 2017 12:14

heplesser requested changes Jun 30, 2017

View reviewed changes

Made requested changes.

ffe5b41

Removed unnecessary comments, printing; using numpy functions for efficiency; skipping if without GSL where needed.

heplesser requested changes Jul 2, 2017

View reviewed changes

Small cleanup

96d3941

heplesser approved these changes Jul 3, 2017

View reviewed changes

Minor changes to comments and formatting

46ee0e0

jougs reviewed Jul 6, 2017

View reviewed changes

hakonsbm and others added 2 commits July 6, 2017 09:45

Merge pull request #16 from jougs/hakonsbm-manualtests

accd1d5

Minor changes to comments and formatting

pep8ify

568fd4a

Merge pull request #17 from jougs/hakonsbm-manualtests

720bad8

pep8ify

Deleted manualtests that could not be fixed

df6feae

hakonsbm and others added 4 commits July 7, 2017 11:28

Switched back to old Ubuntu image

2e8d285

Merge remote-tracking branch 'origin/master' into manualtests

18c874d

Converted hh_phaseplane.m and test_hh_phaseplane.sli to PyNEST

5e9fc81

hh_phaseplane.py example.

Proper skip without gsl declaration

6b10cd0

hakonsbm added 2 commits July 18, 2017 09:03

Reverted 2e8d285

1e57e86

Merge branch 'master' into manualtests

d02e279

Started converting one of the tests to pythontest

8993de9

hakonsbm force-pushed the manualtests branch from ea39e7d to d02e279 Compare August 15, 2017 11:58

Removed superfluous tests

a631286

heplesser mentioned this pull request Aug 21, 2017

Explore design options for the SLI language #811

Closed

Removed SLI language design study files

66d0b36

The SLI modifications are now discussed in issue nest#811.

jougs mentioned this pull request Sep 4, 2017

Refactor Connection Generator Interface #483

Merged

suku248 mentioned this pull request Sep 6, 2017

Unittest of dopamine synapse model #820

Open

hakonsbm and others added 7 commits September 8, 2017 13:01

Removed outdated tests

936b110

Converted cross_check_mip_corrdet to python example.

ff35615

Merge pull request #19 from stinebuu/manualtests

c915b3c

converted cross_check_mip_corrdet to PyNEST example

Merge branch 'manualtests' of https://github.com/hakonsbm/nest-simula…

caf9806

…tor.git into manualtests

pep8ify

36db301

Removed manualtests from testsuite cmakelist, made a test python 3

745758d

compatible.

Merge pull request #20 from stinebuu/manualtests

cc22ee1

Pep8ify, removed manualtest from testsuite cmakelist.

heplesser added this to the NEST 2.14.0 milestone Oct 3, 2017

jougs approved these changes Oct 11, 2017

View reviewed changes

jougs merged commit 4ff76ac into nest:master Oct 11, 2017

hakonsbm deleted the manualtests branch October 12, 2017 08:25

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Converting some manualtests to tests used by installcheck #774

Converting some manualtests to tests used by installcheck #774

hakonsbm commented Jun 30, 2017

heplesser left a comment

heplesser Jun 30, 2017

heplesser Jun 30, 2017

heplesser Jun 30, 2017

heplesser Jun 30, 2017

heplesser Jun 30, 2017

heplesser Jun 30, 2017

heplesser Jun 30, 2017

heplesser Jun 30, 2017

heplesser Jun 30, 2017

heplesser Jun 30, 2017

heplesser left a comment

heplesser Jul 2, 2017

heplesser Jul 2, 2017

heplesser Jul 2, 2017

jougs left a comment

jougs commented Jul 6, 2017

jougs commented Jul 6, 2017

hakonsbm commented Jul 6, 2017

hakonsbm commented Jul 6, 2017

lekshmideepu commented Jul 6, 2017

lekshmideepu commented Jul 7, 2017

jougs commented Jul 17, 2017

heplesser commented Aug 9, 2017

hakonsbm commented Aug 9, 2017

hakonsbm commented Aug 21, 2017

jougs commented Aug 21, 2017

suku248 commented Sep 5, 2017

heplesser commented Sep 5, 2017

mhelias commented Sep 8, 2017

hakonsbm commented Sep 20, 2017

heplesser commented Oct 3, 2017

jougs left a comment


		(unittest) run
		/unittest using

Converting some manualtests to tests used by installcheck #774

Converting some manualtests to tests used by installcheck #774

Conversation

hakonsbm commented Jun 30, 2017

heplesser left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

heplesser left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jougs left a comment

Choose a reason for hiding this comment

jougs commented Jul 6, 2017

jougs commented Jul 6, 2017

hakonsbm commented Jul 6, 2017

hakonsbm commented Jul 6, 2017

lekshmideepu commented Jul 6, 2017

lekshmideepu commented Jul 7, 2017

jougs commented Jul 17, 2017

heplesser commented Aug 9, 2017

hakonsbm commented Aug 9, 2017

hakonsbm commented Aug 21, 2017

jougs commented Aug 21, 2017

suku248 commented Sep 5, 2017

heplesser commented Sep 5, 2017

mhelias commented Sep 8, 2017

hakonsbm commented Sep 20, 2017

heplesser commented Oct 3, 2017

jougs left a comment

Choose a reason for hiding this comment