Add userprojects to Jenkins

[neworderofjamie]

Jenkins already tracks test duration and outputs could be archived on Jenkins so a lot of the work this script does (slightly buggily if #222 and #77 are anything to go by) and means it gets run more often.

[neworderofjamie]

@tnowotny - now the userprojects are fixed (I found the really stupid mistake in the Schmuker one) I guess we should figure out some acceptance bounds for them. I was thinking we could figure out an acceptable range of spike counts for the various populations in maybe Izh_sparse_project, MBody1_project and PoissonIzh_project and have Jenkins run them from generate_run to test the whole procedure

[tnowotny]

yes ... sounds good. How long do you think we should allow ourselves to run these? Would it be long enough to have some learning in the MBody one? For the other ones it is less of an issue.

[neworderofjamie]

I think that would be fine - the default size is pretty small so the 50s simulation I sent you the raster plot of probably doesn't take much more than a minute to run. Running all the tests is currently taking between 10-20 minutes depending on the machine so I think, in exchange for a pretty good vote of confidence that learning is actually working, another minute to run that would be worth it.

[tnowotny]

ok ... so on the MBody example, can we require between 1 and N_output/2 spikes for the last last 100 of the 50ms cycles?

For the Izhikevich models, I would probably measure the typical activity (say average number of spikes in the last 1s of simulated time and accept as correct if any future run is within +/- 10% of that?

[jamesturner246]

How about running the tests a bunch of times, getting the mean and std spike activity in the last second, and failing only the test runs that have activity outside of [(mean - x * std), (mean + x std)] for some x?

Just in case the "gold standard" run is an outlier.

[neworderofjamie]

Yeah - that would indeed be a better way of establishing bounds

[tnowotny]

good point - using std (maybe x=3?) a lot more sensible than guessing +/- 10%

[neworderofjamie]

If anyone wants to take this task on that would be awesome 😄 the generate run executables for all of the userprojects in the genn_4 branch now share a lot of code through https://github.com/genn-team/genn/blob/genn_4/userproject/include/generateRun.h. I guess adding a google test mode could either be implemented in either:

1. The generate run which would have the advantage of catching compile errors but would mean a bit of extra work reading back results from files.
2. The generate run could just add a macro to sizes.h and probably set a build flag when it makes the simulator. Between these they could link google test and invoke the simulation as a test.