Mutable connection table

[thorstenhater]

Add functionality, docs, and examples on editing the connection table. This is a first, small PR
on the topic, further functionality will come as requested.

Teaser example

    // This recipe -- omitted -- implements a mutable connection table 
    // and returns it via the `connections_on` callback. New entries are
    // added using `add_connection` which makes one link from a central 
    // `spike_source_cell` to the given `gid`
    auto rec = recipe();
    rec.add_connection(1);

    auto sim = arb::simulation(rec, ctx);

    // Record spikes
    std::vector<arb::spike> spikes;
    sim.set_local_spike_callback([&spikes](const auto& s) { spikes.insert(spikes.end(), s.begin(), s.end()); });

    // Run and print all spikes per gid
    sim.run(0.25, 0.025);
    show_spikes(spikes);

    // Add a new connection
    rec.add_connection(2);
    sim.update_connections(rec);

    // Run and print all spikes per gid
    sim.run(0.5, 0.025);
    show_spikes(spikes);

Update July '22

• added re-wiring of Event Generators
• implemented a split between a recipe and a recipe-update (cheekily dubbed topping until further notice)
• recipes are toppings via inheritance

[halfflat]

It's great to see this functionality in! Some queries about the API:

1. What does (or should) happen to spikes in-flight? Afaiui the current simulation loop ensures that any spikes generated in the last epoch will have been exchanged, but their corresponding events are waiting to be delivered, and I think as things stand these would still be delivered to targets when we restart the simulation even though they may no longer be connected to the corresponding spike sources. I think this may be a matter of determining what the correct behaviour ought to be, and making any required changes to support that.
2. While updating the recipe makes sense, it's also messy from an ontological point of view: other changes to the recipe won't be seen, including any updates to source or target labels. The approach in the PR is efficient if there are very large disruptions to the connectivity, but for small changes it feels very heavy handed — when thinking about how we might do this in the past, I thought we might introduce changes to connectivity (and other aspects of the model) based on providing a series of 'edit' descriptors that describe the sort of delta we want to apply; these would refer to entities introduced by the recipe, but wouldn't rely upon the recipe itself still being present. What do you think?

[thorstenhater]

Good questions, I'll answer the easy one first (2)
I decided on using the recipe since it is a familiar interface to users. Also, it allows us to easily extend the functionality
to event generators, gap junctions, etc.

Regarding the edit command approach, I wanted to reach for that first. However the code for building the actual table
made it clear that all the expensive work would have to be done again anyhow. I am referring to the sorting and partitioning of the actual backing tables. The rest is relatively cheap. Also, we'd need to keep around (or re-create) a set of items that is used in builing the table, resolving the labels: domain_decomposition, context and the label_resolvers.
In general, performance is not really a consideration, since this is expected to occur every 1s-1min of biological time
or so (in vivo it's more like 10min-1h).

There's a additional wrinkle though: What if connectivity is random? Resetting the pRNG helps, but might not be sufficient
as other parts of the recipe/sim-building might call into it. I am considering to expose (parts of) R123 via Arbor.

[thorstenhater]

Regarding spikes in flight: This is really a question of what we believe is correct, ensuring this happens, and then documenting it.

My gut instinct says they can just be delivered, since they have been injected into a working connection
and who's to say 'where' the spike is at the moment of disconnect (especially as Arbor lumps in axon, synaptic cleft,
and post-synaptic processes into (weight, delay)).

Additionally I expect that usually a connection will decay smoothly in time rather than just being turned from
(delay=0.1, weight=1) into essentially (delay=\inf, weight=0) within a time much smaller than delay.
There might be instances where someone is modelling traumatic injuries and this happens, but then again: We cannot
locate an event along the line of transmission and who konws what's 'correct' in this case.

[halfflat]

On the rebuilding connection tables front, the cheapest solution for an edit-based approach is to just allow changes to weights, not topology. This would mean all potential connections need to be declared in the original recipe, with weights set to zero as appropriate, but given that modifying the model to permit new sources and targets is a whole different and larger kettle of fish, I don't think that's necessarily a severe restriction.

Connections would be identified by an associated connection-specific key or label that's attached when they are described in the recipe; an edit would be something like: modify (e.g. assign, or scale, or, add a delta) weight on connections matching filter (at simplest: with a given label) by or from a given value.

[halfflat]

For random things: we have to be sure that every 'random' sequence is essentially functional in specification, with the possibility that some of the parameters of that function corresponding to seeds or stream identifiers might be implicit. If new random connections are required, they will have to be pulled from a different random sequence (identified by a stream id) or be a different segment of the same random sequence.

But we can do all that without exposing R123 itself through the interface.

[thorstenhater]

Declaring all potential connections has been considered, but obviously grows like O(N^2) in the number of cells,
where the factor is determined by the number of (pre-declared) synapses on a cell.
Despite zero weight, spikes over those connections are still injected into the network (we abuse this in the busyring
for benchmarking). So, even for reasonably sized networks we would incur loads of extra traffic. Also, mid term
we will (probably!) have to enabling adding/removing more items.

[halfflat]

It would be all potential connections given the biological model; it might grow as O(N^2) for small N, where every neuron is 'close enough' to every other neuron, but it should still be O(N) for large models. Extra traffic would only arise in the cases where a source has all of its potential targets with zero weights (i.e. connected to nothing at all); not creating events with zero weight at the delivery side of things would be the optimization.

[thorstenhater]

Hmm, about that for a compromise (fixing the perfectly valid ontological concerns): I factor out the rebuildable parts into
their own components and recipes aggregate those components as do the 'update-recipes'. Just different subsets thereof.

[halfflat]

This could be a good solution! Would you be modifying the recipe interface? (e.g. to inherit from a connection-set-specifying interface which captures connections_on and maybe gap_junctions_on?)

[thorstenhater]

I am not going to inflict Alexandrescu-style policy template aggregation on our users ;)
So, it's going to be virtual interfaces. There's two ways to slice this:

1. the parts we can update ./. the parts we cannot
   Example: connections, event generators ./. gap junctions, cells, ...
   These subsets might change laters on, for example when we enable distributed gap junctions.
   In this model recipe inherits both parts, the simulation_updater* just the latter part.
2. Each callback essentially gets its own interface. Recipe inherits all, updaters only parts.
   Some sensible grouping may be applied, like bunching gid -> cell kind together with
   gid -> cell description.

Note that 2. is essentially 1. in disguise, but leaves the aggregation to the actual classes.
However, this still requires to keep or recreate the label-resolving functionality around which
will likely end in updater reimplementing recipe. (Edit: I was confused, this lives in simulation,
not in recipe. Sorry.)

• We could call this a topping to keep with the theme.

[thorstenhater]

One more thing: proper definitions for partition, domain and perhaps cell_group (this is sometimes, maybe, used as alias for domain, but it is not, I think!) are not present anywhere in the docs, yet relevant in the discussion of arbor::communicator. They would go well in the currently very brief doc/concepts/domdec.rst.

Note that partition's use is overloaded. In this context it means either an ordered set of indices I such that
an array A of grouped items has groups with starting indices I[i] or it refers to the load balancing done by
partition_load_balance.

For the rest: I added some definitions.

[thorstenhater]

Anything still amiss @brenthuisman @lukasgd

[brenthuisman]

The second heading (Exchange of Spikes) shows an implementation for communicator::exchange, not a description of a communicator. The struct is shown much further down, line 205. I think it's easier to follow this text if people are aware of what it contains up front.