10th February 2023
Main change:
Implemented "sparse" waveforms in core and propagated to all modules (#1079, #1136, #1152):
- Add waveform extractor
save()
function and Zarr backend (#1079)- Dense waveforms can be saved as sparse with the
save()
function- Sparse waveforms can be extracted directly with
sparse=True
Added IBL preprocessing tools for Neuropixels (#1025, #1059, #1070):
- Bad channel detection
- Channel interpolation
- High-pass spatial filter
Add subfolder for sorter output ("output_folder/sorter_output") andsorting_info
field to Sorting,so that "sorter_output" can be deleted without deleting metadata andsorting_info
(#1149, #1150)Refactor Quality Metrics module (#1155, #1177, #1197):
- Improve parameter handling
- Unified naming and conventions
- Add missing metrics: drift, amp medians, sliding_rp_violations
Extended Motion estimation and motion correction, which is now ready to use (#1129, #1108, #1183)
Removed deprecated functions/modules (#1261):
toolkit
modulesymmetrize
argument from compute_correlogramslocalize_units
from postprocessingMajor rewrite of the docs, with extensive module-specific documentation (#1264, #1265, #1267, #1268, #1269, #1273, #1275, #1276, #1277, #1281, #1284, #1285, #1287, #1288, #1291)
- core:
- Added possibility of extracting unfiltered waveforms (#1046)
- Prefer
folder`
argument and makezarr_path`
deprecated insave()
function (#1049) - Fix bug 'int16' is not a JSON serializable (#1069)
- Add waveform extractor
save()
function and Zarr backend (#1079) ChunkRecordingExecutor
: add max_thread_per_process to limit the number of thread using the module threadpool_limits (#1086)- Correct channel selection in tests when mono-channel recording (#1096)
- Add
split_job_kwargs()
function (#1110) - Improve handling of not dumpable objects (#1131)
- Implement ChannelSparsity object in core (#1136)
- Handle sparsity inside WaveformExtractor (#1152)
- Use absolute folder in NpySnippetsFolder (#1171)
- Add global_job_kwargs functions (#1176)
- Propagate probe annotations to recording (#1208)
- Serialization for recording and sorting objects (#1227)
- Extend WaveformExtractor rec_attributes and propagate recordingless mode (#1258)
- Improve serialization (#1227, #1272)
- extractors:
- NWBRecordingExtractor offset extraction (#1057)
- fsspec for streaming NWB data (#1090)
- Optimizing Phy-based extractors (#1153)
- Add CellExplorerSortingExtractor (#1158)
- Fix neuralynx sorting function and NeoBaseSorting when no streams (#1171)
- Add option to load synchronized timestamps in OpenEphys (#1198)
- Streaming IBL recordings (#1204, #1251)
- Load phy properties from cluster_info.csv (#1210)
- Fix the sample_shifts for spikeglx when not all channel are saved (#1232)
- preprocessing:
- Add IBL channel interpolation to preprocessing (#1025)
- Add IBL high pass spatial filter to preprocessing (#1059)
- Add IBL bad channel detection (#1070)
- Add get_spatial_interpolation_kernel in preprocessing (#1074)
- Add median/average artefact modes to
remove_artifacts
(#1106, #1143, #1250) - Fix phase-shift dtype (#1172)
- Add correct_lsb function (#1262)
- sorters:
- Refactor sorter_output folder to keep sorting metadata (#1149)
- Add sorting_info property to Sorting object (#1150)
- Set n_jobs to -1 for sorters to propagate global job kwargs (#1207)
- Added sample2volt for IBL version of pykilosort (#1211)
- Correctly find parent folders for concatenate recording/sorting objects (#1230)
- postprocessing:
- Fix template_metrics
select_units()
and extend postprocessing tests (#1112) - Support recordingless mode for PCA (#1132)
- Improve Phy+PCA sparsity handling (#1133)
- More flexibility to auto merge function (#1135)
- Fix template metrics for sparse waveforms (#1196)
- Noise level as a waveform extractor extension (#1221)
- Fix template_metrics
- qualitymetrics:
- Refactor amplitude_cutoff and presence_ratio metrics (#1142)
- Refactor QM parameter handling (#1155)
- Implement missing metrics (amplitude median, drift metrics) (#1177)
- Add sliding RP violations IBL metric (#1197)
- Quality metrics and postprocessing fixes for sparsity and global job_kwargs (#1199)
- Protect against QM returning None (#1224)
- comparison:
- Fix networkx deprecation and clean up comparison tests cache (#1189)
- curation:
- Fixed bug in remove_duplicated_spikes with 'random' method (#1061)
- Added metadata copy from parent in
RemoveDuplicatedSpikesSorting
andAlignSortingExtractor
(#1067) - Add option to return redundant pairs in
remove_redundant_units
(#1078) - Allow apply_sortingview_curation to load json curation files directly (#1104)
- Improve MergeUnitsSorting to have several merge at the same time (#1244)
- widgets:
- Increase sv timeresies tile-size (#1101)
- Updates for SV sorting summmary (#1165)
- Promote distinctipy color as a second choice because it is slow (#1222)
- sortingcomponents:
- Refactor detect_peak() and refactor method_kwargs in components (#1105)
- Port kilosort motion estimation (iterative_template registration) (#1108)
- Refactor motion estimation (#1183)
- Initial refactoring for node pipeline (#1209, #1271)
- Fix gaussian computation in motion (#1257)
- docs:
- Fix broken spikeinterface-gui links (#1072)
- Enhance docker docs (#1081, #1087)
- Refactor docs (#1264, #1265, #1267, #1268, #1269, #1273, #1275, #1276, #1277, #1281, #1284, #1285, #1287, #1288, #1291)
- ci:
- Fix CI python version (#1141)
- Fix code coverage and ignore skipped sorters (#1220)
- Time report for full tests (#1297, #1299)