Failure of export_to_phy from existing mountainsort cluster data #261
Description
I have a use case where I take already data already sorted and curated with MountainSort, and funnel through SpikeInterface API to generate phy files...
import json, os, spikeinterface
import spikeinterface.core.waveform_extractor as wave
import spikeinterface.exporters.to_phy as phy
import spikeinterface.extractors.mdaextractors as mda
# Point to a prv recording and the params file
local_path = '/Volumes/GenuDrive/RY16_direct/MountainSort/RY16_36.mountain/RY16_36.nt1.mountain'
prv_file = os.path.join(local_path, 'raw.mda.prv')
params_file = os.path.join(local_path, 'params.json')
# Derive the properties in those json dicts
with open(prv_file, 'r') as F:
raw_dict = json.load(F)
with open(params_file, 'r') as F:
params_dict = json.load(F)
# Create the mda object
mda_reecording = mda.read_mda_recording(local_path,
raw_fname=raw_dict['original_path'])
mda_reecording.annotate(is_filtered=True) # we've already filtered it, so mark it thus
# Derive the spikes file
firings_file = os.path.join(local_path, 'firings_raw.mda')
sorted_spikes = mda.read_mda_sorting(firings_file, sampling_frequency=params_dict['samplerate'])
# Create waveforms per spike
waveform_file = os.path.join(local_path, 'waveforms')
waveforms = wave.extract_waveforms(mda_reecording, sorted_spikes, waveform_file)
# Export to phy
phyplace = os.path.join(local_path, "phy")
phy.export_to_phy(waveforms, phyplace)Which is great: I like phy a bit more than qt-mountainview curation. But for some of my data, this triggers an error in the export_phy step. Namely, recording chunk end points can overshoot the mda N1 x N2 size bounds, raising an error.
Disclaimer: I didn't fully read the code in the execution stack to ensure this is kosher. But bounding the end to the true sample here fixes it. In other words, it works if I alter line 55 in job_tools.py ...
def devide_recording_into_chunks(recording, chunk_size):
51 all_chunks = []
52 for segment_index in range(recording.get_num_segments()):
53 num_frames = recording.get_num_samples(segment_index)
54 chunks = divide_segment_into_chunks(num_frames, chunk_size)
55 #all_chunks.extend([(segment_index, frame_start, frame_stop) for frame_start, frame_stop in chunks])
57 # modification in the the next two lines
56 sample_max = recording.get_num_samples()
57 all_chunks.extend([(segment_index, frame_start, np.min((frame_stop, sample_max)) for frame_start, frame_stop in chunks])
58 return all_chunks