detect_bad_channels is really slow

[WeissShahaf]

the longest part of preprocessing is the detect_bad_channels part. even if i slice the recording to just 30 seconds (384 channels)
can it be optimized? perhaps using parallelization?

thanks

[h-mayorquin]

I took a brief look a the function and that makes sense.

Which method are you using? Are all of them equally slow?

https://github.com/h-mayorquin/spikeinterface/blob/bc0cbd9f74f30cfed9376f984091cd57f3634169/src/spikeinterface/preprocessing/detect_bad_channels.py#L124-L125

Also, if you could investigate what is the bottleneck with a profiler we could come up with a lan.

[WeissShahaf]

i run this script: (attached results files

import cProfile
import pstats

def function_to_profile():

bad_channel_ids, channel_labels = si.detect_bad_channels(slic_rec)
pass

List of methods to profile

methods = ["coherence+psd", "std", "mad", "neighborhood_r2"]
for method in methods:
# Create a Profile object
profiler = cProfile.Profile()

# Start profiling
profiler.enable()

# Run the function you want to profile
function_to_profile()

# Stop profiling
profiler.disable()

# Save the profiling results to a file in the correct format
profiler.dump_stats(f"profile_{method}_results.prof")

# Create a Stats object from the saved .prof file
stats = pstats.Stats(f"profile_{method}_results.prof")

# Print the stats to the console
stats.print_stats()

profiling.zip

[alejoe91]

Thanks! I'll give it a stab to parallelize it soon