Parallelize synaptic effects application in target mode for heterogeneous delays WITHOUT bundle mode and WITHOUT atomics

[moritzaugustin]

EDIT by denisalevi:
Summary: sort synapse IDs in spike queues by post neuron IDs and use thread <-> postNeuron correspondence.

---

currently as the current synapse queue contains synapse ids for which the synaptic effects have to be applied. this is done in a sequential fashion for each block.

[see denis comment for the first attempt too]

we should make it parallel for each block by

• loading synapse ids and postneuron ids of the spike queue into shared memory (outer loop might be necessary)
• sort that data w.r.t postneuron id (using bitonic merge sort)
• run the synaptic code in parallel with the relation threadid-occuringpostneuronid

[denisalevi]

first approach: not bitonic sort but instead:
direct paralellization by threadid:postneuronid assignment (and each thread runs through all synapses in the current queue) and use shared memory for the required data, i.e., postneuron index and synapse index

[moritzaugustin]

for the finally targeted more advanced approach of sorting of synapses (value array in shared memory) according to postsynaptic neurons (key array in shared memory) for coalesced application of synaptic effects the following code snippet might be very helpful (actually a radixsort instead of a bitonic sort is performed which should be fast enough). note the code relies on https://nvlabs.github.io/cub/classcub_1_1_block_radix_sort.html where also simple/more readable examples are given: http://stackoverflow.com/a/34607511

[denisalevi]

do this approach onlz if the bundle way is slower... as that one is trivially parallizable in the effect application (given not more than 1 pre post pair)

[denisalevi]

We are using the bundle mode by default now as it is faster in most use cases. We are keeping the synapse push mode as a user preference, therefore this issue stays open. But it is only relevant it we don't use bundles AND can't use atomics (#78). It has therefore low priority for now.

[denisalevi]

Check #30, which seems to be the same issue as this one.