LFPBand population gives error in included timestamps

[sharon-chiang]

Describe the bug
While populating LFPBandV1, received an error that new_timestamps contained points not in lfp_band_valid_times after filtering, which prevented insertion.

To Reproduce

from spyglass.lfp.analysis.v1 import lfp_band
lfp_band_key = {'lfp_merge_id': '10be69d2-3944-ca09-ae84-b7973e10ee23', #UUID('10be69d2-3944-ca09-ae84-b7973e10ee23'),
 'filter_name': 'Theta 5-11 Hz',
 'filter_sampling_rate': 1000,
 'nwb_file_name': 'SC3820230606_.nwb',
 'target_interval_list_name': 'pos 0 valid times',
 'lfp_band_sampling_rate': 100}
lfp_band.LFPBandV1().populate(lfp_band.LFPBandSelection() & lfp_band_key)

Error Message:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[2], line 8
      1 from spyglass.lfp.analysis.v1 import lfp_band
      2 lfp_band_key = {'lfp_merge_id': '10be69d2-3944-ca09-ae84-b7973e10ee23', #UUID('10be69d2-3944-ca09-ae84-b7973e10ee23'),
      3  'filter_name': 'Theta 5-11 Hz',
      4  'filter_sampling_rate': 1000,
      5  'nwb_file_name': 'SC3820230606_.nwb',
      6  'target_interval_list_name': 'pos 0 valid times',
      7  'lfp_band_sampling_rate': 100}
----> 8 lfp_band.LFPBandV1().populate(lfp_band.LFPBandSelection() & lfp_band_key)

File ~/Documents/anaconda3/envs/spyglass/lib/python3.9/site-packages/datajoint/autopopulate.py:241, in AutoPopulate.populate(self, suppress_errors, return_exception_objects, reserve_jobs, order, limit, max_calls, display_progress, processes, make_kwargs, *restrictions)
    237 if processes == 1:
    238     for key in (
    239         tqdm(keys, desc=self.__class__.__name__) if display_progress else keys
    240     ):
--> 241         error = self._populate1(key, jobs, **populate_kwargs)
    242         if error is not None:
    243             error_list.append(error)

File ~/Documents/anaconda3/envs/spyglass/lib/python3.9/site-packages/datajoint/autopopulate.py:292, in AutoPopulate._populate1(self, key, jobs, suppress_errors, return_exception_objects, make_kwargs)
    290 self.__class__._allow_insert = True
    291 try:
--> 292     make(dict(key), **(make_kwargs or {}))
    293 except (KeyboardInterrupt, SystemExit, Exception) as error:
    294     try:

File ~/src/spyglass/src/spyglass/lfp/analysis/v1/lfp_band.py:346, in LFPBandV1.make(self, key)
    338 tmp_valid_times = (
    339     IntervalList
    340     & {
   (...)
    343     }
    344 ).fetch("valid_times")
    345 if len(tmp_valid_times) == 0:
--> 346     lfp_band_valid_times = interval_list_censor(
    347         lfp_band_valid_times, new_timestamps
    348     )
    349     # add an interval list for the LFP valid times
    350     IntervalList.insert1(
    351         {
    352             "nwb_file_name": key["nwb_file_name"],
   (...)
    355         }
    356     )

File ~/src/spyglass/src/spyglass/common/common_interval.py:426, in interval_list_censor(interval_list, timestamps)
    422 # check that all timestamps are in the interval list
    423 if len(interval_list_contains_ind(interval_list, timestamps)) != len(
    424     timestamps
    425 ):
--> 426     raise ValueError("Interval_list must contain all timestamps")
    428 timestamps_interval = np.asarray([[timestamps[0], timestamps[-1]]])
    429 return interval_list_intersect(interval_list, timestamps_interval)

ValueError: Interval_list must contain all timestamps

@samuelbray32 @CBroz1 @edeno

[edeno]

Hi @sharon-chiang , could you include the error message as well?

[edeno]

Also it looks like your LFP sampling rate is 100? and not 1000?

[samuelbray32]

@sharon-chiang I think you deleted and re-inserted the animal for this session SC3820230606. Could you re-populate the LFPV1 table for this session?

[samuelbray32]

Issue wasn't present when running on re-inserted data. Will leave issue open for now in case it arises again

[edeno]

Actually, I will close and it can be reopened if it occurs again.