6th February 2024
Main changes:
- Several improvements and bug fixes for Windows users
- Important refactoring of NWB extractors: * implemented direct backend implementation (to avoid using pynwb) * sped up streaming using remfile * added support for zarr backend
- Removed joblib dependency in favor of ParallelProcessExecutor
- Improved flexibility when running sorters in containers by adding several options for installing spikeinterface
- Add Templates class to core, which handles unit templates and sparsity (#1982)
- Added Zarr-backend to Sorting objects (sorting.save(folder="...", format="zarr")) (#2403)
- Added SharedmemRecording for shared memory recordings (#2365)
- Added machinery for moving/interpolating templates for generating hybrid recordings with drift (#2291)
- Added new fast method for unit/spike/peak localization: grid_convolution (#2172)
core:
- Add Templates class (#1982)
- Use python methods instead of parsing and eleminate try-except in to_dict
(#2157) * WaveformExtractor.is_extension --> has_extension (#2158) * Speed improvement to get_empty_units() (#2173) * Allow precomputing spike trains (#2175) * Add 'percentile' to template modes and plot_unit_templates (#2179) * Add rename_units method in sorting (#2207) * Add an option for count_num_spikes_per_unit (#2209) * Remove joblib in favor of ParallelProcessExecutor (#2218) * Fixed a bug when caching recording noise levels (#2220) * Various fixes for Windows (#2221) * Fix num_samples in concatenation (#2223) * Disable writing templates modes npy in read-only mode (#2251) * Assert renamed_channels/unit_ids is unique (#2252) * Implement save_to_zarr for BaseSorting (#2254) * Improve the BaseExtractor.to_dict() relative_to machanism to make it safer on Windows (#2279) * Make sure sampling frequency is always float (#2283) * NumpySorting.from_peaks: make unit_ids mandatory (#2315) * Make chunksize in get_random_data_chunks throw warning and clip if under limit (#2321) * ids can be a tuple in ids_to_indices (#2324) * get_num_frames to return a python int (#2326) * Add an auto adjustment if n_jobs too high on Windows (#2329) * Cache spike_vector from parent (#2353) * Refactor recording tools (#2363) * Add rename_channels method to recording extractors (#2364) * Create SharedmemRecording (#2365) * WaveformExtractor.select_units also functional if we.has_recording()=False (#2368) * Add zarrrecordingextractor.py for backward compatibility (#2377, #2395, #2451) * Improve ZarrSortingExtractor (#2403) * Improvement to compute sparsity without WaveformsExtractor (#2410) * Zarr backcompatibility: map root_path to folder_path (#2451) * Fix spikes generation on borders (#2453) * Zarr IO for Templates object (#2423) * Avoid double parsing in Plexon (#2459)
extractors:
- Add .stream.cbin compatibility to CompressedBinaryIblExtractor (#2297)
- Add stream_folders path to OpenEphysBinaryRecordingExtractor (#2369)
- Deprecate ignore_timestamps_errors in OpenEphysLegacyRecordingExtractor (#2450)
- Better auto-guess of open-ephys format (#2465)
- Several improvements to NWB extractors: * Add option for no caching option to the NWB extractor when streaming (#2246, #2248, #2268) * Fix NwbSortingExtractor reading of ragged arrays (#2255) * Add nwb sorting remfile support (#2275) * Avoid loading channel_name property in NwbRecordingExtractor (#2285) * Add hdf5 backend support for Nwb extractors (#2294, #2297, #2341) * Refactor NwbSortingSegment (#2313) * Add t_start argument to NwbSortingExtractor (#2333) * Add support for NWB-Zarr enhancement and zarr streaming (#2441, #2464)
preprocessing:
- Fix filtering rounding error (#2189)
- Fix: save a copy of group ids in CommonReferenceRecording (#2215)
- Add outside_channels_location option in detect_bad_channels (#2250)
- Fix overflow problems with CAR (#2362)
- Fix for Silence periods (saving noise levels) (#2375)
- Add DecimateRecording (#2385)
- Add margin_sd argument to gaussian filtering (#2389)
- Faster Gaussian filter implementation preprocessing (#2420)
- Faster unpickling of ZScoreRecording (#2431)
- Add bit depth compensation to unsigned_to_signed (#2438)
- Renaming: GaussianBandpassFilter -> GaussianFilter (and option for low/high pass filter) (#2397, #2466)
sorters:
- Several updates to SpykingCircus2 (#2205, #2236, #2244, #2276)
- Handling segments in SpykingCircus2 and Tridesclous2 (#2208)
- A couple updates to mountainsort5 sorter (#2225)
- run_sorter in containers: dump to json or pickle (#2271)
- run_sorter in containers: add several options for installing spikeinterface (#2273)
- Close ShellScript and pipes process at deletion (#2292, #2338)
- Remove deprecated direct function to run_sorter (e.g., run_kilosort2 -> run_sorter('kilosort2') (#2355)
- Expose lam and momentum params in the appropriate kilosorts (#2358)
- Tridesclous2 update (#2267)
postprocessing:
- Use sampling_frequency instead of get_sampling_frequency in _make_bins (#2284)
- Multi-channel template metrics fix (#2323)
- Fix bug in get_repolarization_slope with wrong index type (#2432)
- Estimation of depth for grid_convolution localization (#2172)
qualitymetrics:
- Implemented sd_ratio as quality metric (#2146, #2402)
- Avoid duplicated template and quality metric names (#2210)
- Fix rp_violations when specifying unit_ids (#2247)
curation:
- Fix bug in mergeunits (#2443)
- Fix sortingview curation and merge units with array properties (#2427)
- Move computation away from __init__ in duplicated spikes (#2446)
widgets:
- Sorting summary updates in sortingview (#2318)
- Add a more robust delta_x to unit_waveforms (#2287)
- Prevent users from providing a time_range after the ending of the segment in plot_traces (#2286)
- Fix sortingview checks for NaN if strings (#2243)
generation:
- Creation of a TransformSorting object to track modifications and bencharmk (#1999)
- Add a minimum distance in generate_unit_locations (#2147)
- Add Poisson statistics to generate_sorting and optimize memory profile (#2226)
- Fix add_shift_shuffle section in synthesize_random_firings (#2334)
- Machinery for moving templates and generating hybrid recordings with drift (#2291)
sortingcomponents:
- Strict inegality for sparsity with radius_um (#2277) by yger was merged on Dec 1, 2023
- Fix memory leak in lsmr solver and optimize correct_motion (#2263)
docs:
- Various improvements to docs (#2168, #2229, #2407)
- Improve ids_to_indices docstring (#2301)
- Fix for docstring of get_traces (#2320)
- Fix RTD warnings (#2348)
- Improve CMR docstring (#2354)
- Correct warning format in neo base extractors (#2357)
- Typo fix for verbose setting in Multicomparison (#2399)
ci / packaging / tests:
- Add tests for unique names in channel slice and unit selection (#2258)
- Add from __future__ import annotations to all files for Python3.8 (#2340, #2468)
- Add pickling test to streamers (#2170)