Codejam #6 : remove scalar object

[samuelgarcia]

Every body agree for removing "scalar" object.

Spike is remove.
Event is remove.
Epoch is remove.
AnalogSignal is remove.

AnalogSignalArray is rename AnalogSignal and can be ND.
SpikeTrain is kept like this.
EventArray is rename Event.
EpochArray is rename Epoch.

[toddrjen]

Where will AnalogSignalArray end up? Will it stay in RCG, move to RC, or be allowed in both?

What about the handling of the channel indexes? There may be one or more indexes, which may or may not correspond to a dimension in the array. Will there be some rule about which dimension indexes have to correspond to?

Say we choose dimension 1 as the "channel" dimension. We find an array with 1 channel index, and that has a length of 1 in dimension 1. How can we tell whether this means dimension 1 is channel 1, our that it is an arbitrary array that happens to have that shape?

[samuelgarcia]

This is the main point in the new model. Ideally, I would like that it would attached to both.

There are several use case:

• AnalogSignal are 1D each one is link to one RC and RC can be grouped in several RCG, this is flexible and this is what I used in OpenElectrophy for spike sorting because you can create RCG on fly whitout cancatenate. This is more or less the old model because many IOs used o load 1D signals.
• AnalogSignal ar 2D, naturally they would be link to RCG. But for creating sub group with that approach we would have to slice signals and recreate RCG.

The problem is that names are not natural. Maybe the best aproach would be:

• RecordingChannel hold a channel_indexes as vector even if it is one channel.
• RecordingChannelGroup is rename as SomethingGroup* because RCG is also a group of Units

*replace SomethingGroup by a good idea

[toddrjen]

That makes sense.

However, this still has the question of which dimension, if any, corresponds to the "channel_indexes" vector, and how to tell apart arrays that have one channel index corresponding to one singleton dimension vs. an array that has no channel/dimension relationship but happens to have a singleton dimension.

[samuelgarcia]

Channel_indexes could be
AnalogSignal.shape[1:]

So for 1D it is a scalar
for 2D it is a vector
for 3D it a 2D array.

It is logical but complex.

[toddrjen]

So are we agreed that this is the best approach?

[samuelgarcia]

An example for a 500ms signal to be sure:

case 1D:
a = AnalogSignal( np.random.randn( 50), units = 'mV', sampling_rate = 100*Hz)
a.rcordingchannel = RecordingCahnnel(indexes = 12)

case 2D:
a = AnalogSignal( np.random.randn( 50 , 4), units = 'mV', sampling_rate = 100*Hz)
a.rcordingchannel = RecordingCahnnel(indexes = [12, 25, 26, 63])

case 3D:
a = AnalogSignal( np.random.randn(50 , 4, 2), units = 'mV', sampling_rate = 100*Hz)
a.rcordingchannel = RecordingCahnnel(indexes = [[12, 25, 26, 63],[13, 29, 30, 64]] )

Is it what you propose ? For me it is quite logical but we should accept this:
case 1D:
a = AnalogSignal( np.random.randn( 50), units = 'mV', sampling_rate = 100*Hz)
a.rcordingchannel = RecordingCahnnel(indexes = [12])

[toddrjen]

I agree a consistent type would be better. I am not sure whether this should be a list or an array.

[rproepp]

I'm a bit unclear as to what exactly a channel and indexes represent with multidimensional signals. Could you axplain a bit more?

The removal of the Spike object can cause problems with representing spike sorting, where you often have "template" or mean spike waveforms. How would we accommodate that in the new model?

[toddrjen]

For spikestrains, a spike would just be a spiketrain of length 1.

Currently we don't have any cases that use ndim>2 as far as I recall. I guess what they would be used for depends on the signal in question. So dimension 0 is time, dimension 1 (for block/rcg) would be channel, and above that would depend on the specific signal being recorded.

[rproepp]

So there would need to be multiple spike trains for a unit-segment combination? And an ad hoc method or convention on how to distinguish between "real" spike trains with a single spike and these template spike trains? I'd be okay with that and it's not much worse than the current situation with Spike objects. Having at least a fixed convention on how to represent abstract spike waveforms would improve things a bit. Should we be more explicit for this probably rather common use case?

Currently we don't have any cases that use ndim>2 as far as I recall. I guess what they would be used for depends on the signal in question. So dimension 0 is time, dimension 1 (for block/rcg) would be channel, and above that would depend on the specific signal being recorded.

I'm mostly wondering what the channel indexes are supposed to represent. Are they arbitrary ids assigned by the user? What, if any, connection do they have to recording channel groups?

[toddrjen]

So there would need to be multiple spike trains for a unit-segment combination?

Sorry, I am not clear what you mean by this. Could you please elaborate?

Having at least a fixed convention on how to represent abstract spike waveforms would improve things a bit. Should we be more explicit for this probably rather common use case?

I am not clear on what you are asking here, either.

I'm mostly wondering what the channel indexes are supposed to represent. Are they arbitrary ids assigned by the user? What, if any, connection do they have to recording channel groups?

Usually the channel indexes in a recordingchannel object are arbitrary, while the channel indexes in a signal or unit are supposed to match up with the channel index in the corresponding recordingchannel.

[rproepp]

Could you please elaborate?

In the current model, it makes no sense to have more than one SpikeTrain for a particular unit/segment combination (at least to me, and it is not done by any IO as far as I know). For spike sorting, you often need to represent a "template waveform" for a particular unit. As this template can change over time, it makes sense to be able to link it to a Segment as well.

Currently, these templates are best represented by Spike objects. If we drop these objects, they can be represented by a spike train with just a single spike (where the time is irrelevant but the waveform is important). But that means that two spike trains are needed for a unit/segment combination. And it might not be immediately clear which SpikeTrain represents what, there might be only a single real spike for a unit in a particular segment as well. So we would need some kind of convention to determine when a spike train represents a template waveform.

I believe that spike sorting is an important use case for Neo, so it might be worth it to support it more explicitly than just by convention. That might be an optional attribute for SpikeTrain to represent a template, or retaining some form of the Spike object for this (and maybe similar?) purposes.

Usually the channel indexes in a recordingchannel object are arbitrary, while the channel indexes in a signal or unit are supposed to match up with the channel index in the corresponding recordingchannel.

What does it mean if they don't match? If they always have to match, why are they needed in AnalogSignal? And I guess the unit has this to specify the channels where its spikes are found, but isn't that also covered by the RecordingChannelGroup?

[toddrjen]

In the current model, it makes no sense to have more than one SpikeTrain for a particular unit/segment combination (at least to me, and it is not done by any IO as far as I know). For spike sorting, you often need to represent a "template waveform" for a particular unit. As this template can change over time, it makes sense to be able to link it to a Segment as well.

Is there a reason this couldn't be handle by an annotation? Just have an annotation that is a 2D quantity array.

What does it mean if they don't match? If they always have to match, why are they needed in AnalogSignal?

Reading the documentation again, I was remembering incorrectly. The documentation indicates it is arbitrary, although seems to imply that AnalogSignal, AnalogSignalArray, and Unit should share a common indexing scheme.

[rproepp]

Is there a reason this couldn't be handle by an annotation? Just have an annotation that is a 2D quantity array.

You're right, i forgot about quantities as annotations. That seems like a good idea, ideally with a convention in the docs. @samuelgarcia, you're also using Neo in spike sorting, does that cover your use case?

Reading the documentation again, I was remembering incorrectly. The documentation indicates it is arbitrary, although seems to imply that AnalogSignal, AnalogSignalArray, and Unit should share a common indexing scheme.

I haven't dealt with the indexes before. Are they used in any IO in a meaningful way? @samuelgarcia, I guess you implemented them initially, could you shed some light on this?