nigeLab_libs_DiskData

DiskData class

Core +libs class for reading and writing data to and from disk files. An overview of properties is below.

• Update (2020-02-01): Class was re-worked. Core functionality is very similar, but it should be more efficient now and also adds some attributes to the disk file directly. Everything is now handled with h5.

• (2019-11-25): Presently, this class is a hot mess. It works, but the logic complexity is ridiculous and needs to be reworked so that the subsref and subsasgn overloaded methods work in a straightforward way. This can be accomplished by restructuring the code to take advantage of case logic that relies on combinations of the following Matlab 'builtin' methods:
  • substruct
  • subsref('builtin',...)
  • subsasgn('builtin',...)

type_

This is the "type" of DiskData. Can be either Hybrid, Matfile, or Event.

'Hybrid'

The Hybrid FileType is the main (default) data read/write file type that is essentially the core of what makes nigeLab convenient. Essentially, each Hybrid file contains a 1 x nSamples vector in a single variable (data). Most Channels and all Streams disk files are saved using the Hybrid FileType.

• 'Hybrid' files are compatible with the append method, and expand by adding columns; however, once they are constructed, the number of rows (1, always), is set.

'MatFile'

(deprecated)
Update (2020-02-01): Specify 'MatFile' to indicate that the file has a fixed number of rows and columns in the constructor. This type_ is not compatible with the append method, since it does not allow expansion of the file along either dimension.

'Event'

The Event FileType is larger than the Hybrid FileType, storing a nEvent x (nSamplesPerEvent+4) matrix in a single variable (data). On the saved Event disk file, the first column of data corresponds to type; the second column of data corresponds to value; the third column of data corresponds to tag; the fourth column of data corresponds to ts; and the remaining columns correspond to snippet.

• 'Event' files are compatible with the append method, and expand by adding new rows; however, once they are constructed, the number of columns is set.

type

The first column of the matrix pointed to by the variable DiskData.name_ in DiskData.diskfile_ represents type data. This is used to classify the 'type' of the Event, according to the following key:

• type == 0 : Channel-associated events
  • (e.g. Spikes, Clusters, SpikeFeatures, and Sorted).
• type == 1 : Recording-associated events
  • (e.g. any Event defined in nigeLab.defaults.Event).
• type == 2 : Recording-associated metadata header
  • (e.g. a small "header" where datatype info about other events is stored in snippets)

Note: The metadata header is a unique file that is created whenever there will be video scoring. Similarly, the Trial Field element is always created whenever there will be video scoring. The header has as many rows as there are videos to score, and the ts value for each row corresponds to the video alignment offset. Values of snippets assign the different Trial snippets metadata to various subcategories for the scoring interface to recognize what kind of metadata they are.

value

The second column of the matrix pointed to by the variable DiskData.name_ in DiskData.diskfile_ represents value data. This is some scalar associated with the Event. For different Event elements, this may have different meanings:

• Spikes: integer representing spike "cluster" for spikes of a given channel
  • Same for Clusters, Sorted
• Artifact: represents amplitude of each Artifact peak
• ScoredEvents: represents common trial index shared by all ScoredEvents "rows"

tag

The third column of the matrix pointed to by the variable DiskData.name_ in DiskData.diskfile_ represents tag data. tag is used as a key into a categorical or cell array of strings that defines a qualitative label for that Event. For example, multiple different spike "clusters" could correspond to a qualitative large unit spike profile, while maintaining characteristically different snippet values.

ts

The fourth column of the matrix pointed to by the variable DiskData.name_ in DiskData.diskfile_ represents ts (timestamp) data. Regardless of whether it is a channel-associated or recording-associated type, each Event contains values (in seconds) of the time relative to the start of the record for each row of this column.

snippet

The matrix formed by the sub-array consisting of each column from the fifth column to the last column of the matrix pointed to by the variable DiskData.name_ in DiskData.diskfile_ represents snippet data. Typically, this is the spike waveform, aligned to the peak; however, in some cases, it may be useful to store snippet data with other Event instances.

• Note that for .Events 'Event' files, snippet may hold various values in its columns that relate to behavioral metadata or other experiment-related event data.

diskfile_

Matlab matfile object that acts as a pointer to the contents of the actual data file. Update (2020-02-01): This is now a char array that is the full filename of the disk file.

overwrite_

If providing 3 input arguments to the constructor (e.g. specifying data on object creation), then this must be set to true using the 'name', value syntax in the constructor arguments. This is to emphasize the change in functionality for handling the constructor with three arguments, and the requirement may be removed at a later date (2020-02-01).

name_

Name of the variable that is pointed to by DiskData. By default (and in general) this is 'data'.

size_

Size (dimensions) of DiskData variable pointed to by DiskData.name_.

access_

Access type to DiskData. Can be either 'r' (read-only) or 'w' (read-write).

writable_

Heuristic property that returns true if DiskData.access_ == 'w'.

verbose_

Set this to false to suppress many of the formatted statements printed to the Command Window, particularly during DiskData construction for new files.

bytes_

Number of bytes in DiskData.diskfile_.

class_

Class of data pointed to by DiskData array.

chunks_

Size of "chunks" to read (for reading and writing large DiskData arrays, such as occurs during Block.doRawExtraction).

---

DiskData Attributes

Small variables written directly to the HDF5 root group Attributes field

• These might be useful, for example if files are moved away from their normal "home" in the folder tree structure, so that they could be re-associated with the correct experiment (animal, recording).
• These might also be useful for parsing file metadata in alternative ways.

Animal

Name of the folder two folders above the file specified by .diskfile_

Block

Name of the folder directly above the file container folder specified by .diskfile_.

Complete

• For 'MatFile' .type_, if .Index is equivalent to the length of the non-empty file, this is true.
• For 'Hybrid' and 'Event' .type_, this must be updated manually (potentially via a doMethod call).
  • Still WIP

Empty

If this is true, the file has no data or "empty" (initializing) data in it.

File

Public version of DiskData.diskfile_

Index

Indicator of where the "data cursor" is for appending data into the file.

• Still WIP, but used during intan2block method.

Locked

Returns true if the file is "locked" (meaning that it is Read-Only)

Tank

Name of the folder three folders above the file specified by .diskfile_