storing trial data

[ajtritt]

1. define trial data types
   • AbstractTrialData neurodata_type?
   • categorical?
     • what types of categories?
   • response and stimulus?
2. add location for trials in NWBFile

Maybe define state-machine type?

[bendichter]

The categorial description of trials will generally correspond to a controlled differentiation in stimulation, stimulus, behavior, etc. These categorical differences are often the key to the novelty of the experiment, so trying to anticipate and define a set of categorical variables that will work broadly is IMO going to be impossible. I would prefer a framework where one can easily define new features and attach sufficient meta-data to understand what they mean.

Also, we should not necessarily limit ourselves to categorical variables. Sometimes stimuli are from a continuous distribution, e.g. the Newsome dot experiment, and sometimes they are better represented by integers.

Here's an idea of how we might accomplish this:

• Each TrialTable should have the ability to hold zero or more trial_features which are additional columns in the table
• Each trial_feature should have name, description, dtype (any more?)
• There must be a data element per trial_feature per trial, with the option of having this element be a scalar or a vector per trial
• dtype should allow for any of the existing data types and an enumerated type. The enumerated type would be used for categorical data, and the allowable categories should be defined by the user. (the real feature here is categorical data. Enumerated types are not a necessity and there are other ways to do this that would work fine.)

I wouldn't be opposed to labeling response or stimulus in the trial_feature meta-data but I don't think that's an essential feature, because that should be obvious from the description.

[bendichter]

Similar to BIDS section 8.5

[ajtritt]

so trying to anticipate and define a set of categorical variables that will work broadly is IMO going to be impossible.

I don't mean the specific variables--I agree, that's impossible. I mean the dtypes of these variables. I'm thinking of having an abstract base type, and then a handful of instantiable subtypes. Something like this:

AbstractTrialFeature

• IntTrialFeature
  • CategorialFeature
    • array of ints that index into an array of categorical values. Additionally, it would have an attribute that holds a reference to the array of categorical values.
• FloatTrialFeature
• StrTrialFeature

The first dimension of these datasets would always be trial, so possible dimensions would be (None,) for scalar data or (None, None) for vector data.

Then TrialTable is a group that contains 1 or more AbstractTrialFeatures. Users don't need to extend TrialTable. They would only need to extend AbstractTrialFeature if they wanted to add another dtype.

I wouldn't be opposed to labeling response or stimulus in the trial_feature meta-data but I don't think that's an essential feature, because that should be obvious from the description.

This would be a requirement for programability, not for human interpretation

[bendichter]

ok all that looks good to me