Separate detect and extract subpackages. #394
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…etection's functions to just one call in extraction.
…moved to detection utils. source and sparsery now dont depend on extraction module.
This reverts commit 241a119.
…ect. Successfully separated extract and detect subpackages. Added meanImg parameters to both detection/extraction tests.
carsen-stringer
added a commit
that referenced
this pull request
Jul 19, 2020
* simplified get_frames() * moved io-bound get_frames() function to io/binary.py * simplified make_blocks() * removed unused function shift_data_subpixel() * fixed docstring and removed unused lines from rigid.shift_data() * refactored out 'offsets' data structure * removed redundant nonrigid.make_blocks() call. * changed print(warning) to warning.warn(), Exception to ValueException, * pulled 1Preg transformation out of rigid.phasecorr() * reordered re_1p n phasecorr_refernce() * split refAndMasks parameter in rigid.phasecorr() into its components and moved numba addmultiply() function to utils.py * removed unused gpu_phasecorr() * removed redundant spatial_smooth during nonrigid registration * refactored non-rigid registration into own function: nonrigid.shift() * pulled nonrigid.shift() out of compute_motion_and_shift() * removed commented-out code in compute_crop() * removed unused imports * refactored out RefAndMasks and RefAndMasksNR * refactored out phasecorr_cpu() * refactored out nonrigid.shift() * removed unused call to nonrigid.transform_data() * removed unused vars from nonrigid.phase_corr() * moved var use closer to declaration * Detection/Extraction regression tests added along with additional registration tests. (#382) * Completed 1plane1chan and 2plane2chan tests for detection/extraction. Added pre_registered data for 2plane2chan case. * Finished adding preprocessing and OASIS to regression tests. * Added regression test for classification module. Added tailored stat.npy that's used before classifier is called. * Added np allclose due to sklearn.LogisticRegression's slight differences in output * Modified tolerance value after checking on another computer. * removed unused patlib import in classification tests. * Refactored check_registration_output in test_registration_module.py * Added input data that requires bidishift. * Added 1preg regression test for registration module and its test_data * changed test_register_binary to use input_1500 with metrics * Removed two pipeline tests. (#383) * removed unused testcase * changed setup_and_teardown() to default_ops() * changed data_dir fixture call to default_ops['data_path'][0] * refactored TestUtils class/fixture to a utils.py module. * moved test methods to test functions * added pipeline Test with sourcery coverage (#385) * removed unused testcase * changed setup_and_teardown() to default_ops() * changed data_dir fixture call to default_ops['data_path'][0] * refactored TestUtils class/fixture to a utils.py module. * moved test methods to test functions * deleted unused test data * added sourcery test and test data * re-added necessary test data * re-added new 1plane2chan verion of sourcery test * Renamed io.write_tiff to changed io.save_tiff method in io tests. * split coupled "spatial_hp" ops parameter into "spatial_hp_reg" and "spatial_hp_detect". * refactored out compute_motion_and_correct() * removed redundant pre_smooth and spatial_hp value checking * register_binary() only takes a Dict ops. * structured nested if statements * used tuple unpacking on multiple assignment * removed temporary 'bidi' name and redundant nonrigid.make_blocks() call * removed unusable utils.split_multiops() line * removed redudnant np.save(ops) in register_binary() * moved depgraph scripts to own folder * added new profiler script (#388) * refactored out repeated data_smooth conditionals * moved compute_zpos() back to zalign.py * cleaned up get_pc_metrics(): collapsed nsamp conditional logic, moved nPC to optional argument * moved ops-level functions back to their original modules * fixed paths * zipped rigid.shift_data() * pulled single-frame shift_frame() out of shift_data() * pulled iteration out of rigid.shift_frames(), reducing to simpler rigid.shift_frame() * pulled memory management out of rigid.shift_frame() into the calling function. * removed temporary variables from refImg mean shift step * removed shadow assignment step * vectorized fft2 and ifft2 steps in rigid.phase_corr() * used fancy indexing on cmax to vectorize cmax filling * removed temp variables for lcorr calculation * refactored out crop() function to remove intermediate vars * cleaned rigid.phasecorr_reference * split rigid.phascorr_reference() into independent functions phasecorr_reference() and compute_masks() * removed extra reference image copying * split rigid.phasecorr() into independent functions rigid.phasecorr() and rigid.apply_masks() * created utils.convolve() function to document fft2/ifft2 step and layer math concepts * removed hard-coded module literals * removed pre-computed module-level constants, replaced with cached function call. * spacing * refactored out nonrigid.clip() * made cc2 calculation more readable * cleaned up loop * cleaned up ymax1, xmax1, cmax1 calculation * vectorized nested fft2/ifft2 calculation * removed unused nonrigid.linear_interp() function * call duplicated addmultiply() in utils instead of nonrigid * removed extra data copy and unused variables in numba function * removed inplace parameter modification to rmeove need for guard copying * removed extra copies * reordered lines in nonrigid.transform_data() * shifted array management down from mat_upsample to kernel1d * refactored nonrigid.make_nblocks(), splitting out two algorithm functions and removed one pipelined argument * step reduction in nonrigid.phasecorr_reference() * moved gaussian kernel functions to utils.py * working in 3D, output in 4D * reduced preallocation steps in nonrigid.phasecorr_reference() * refactored getSNR * replaced fft2/ifft2 call with utils.convolve() * replace zeros() with empty() * moved append() * collapsed two `if ops['1Preg']` into one * composed rigid registration steps * removed redundant ops assignment and misleading comment * merged pre_smooth conditional * lowered gaussian_filter1d() to utils and replaced it with utils.temporal_smooth() * removed some squeeze() and newaxis * added explicit padding step to utils.spatial_smooth, plus a regression test * renamed intermediate steps * moved valueerror to algorithmic function, removed unnecessary [newaxis] -> squeeze() around spatial_smooth() calls * replace concatenate() algorithm with numpy.pad * removed unused ccsm output of nonrigid.phasecorr() * changed gaussian_filter1d call to temporal_smooth() * Added smoothed data for 2 channel test. Removed 1preg test from registration module. (#396) * Separate detect and extract subpackages. (#394) * Separated extraction and detection subpackages. Consolidated all of detection's functions to just one call in extraction. * Removed preclassify code since it doesn't work. Created detection test_module. * Removed 2plane1chan test_data since we do not have any tests for it. * Added detection regression tests along with necessary test_data * Fixed up ops output for main_detect function * Separate extract and detect from run_s2p and in tests * Added regression tests for extraction module. * Checked that separate detection and extraction tests pass. Removing combined tests. * Added chan2 check for extraction tests. Removed extraction utils and moved to detection utils. source and sparsery now dont depend on extraction module. * Remove meanImg calculation from extract subpackage. * Revert "Remove meanImg calculation from extract subpackage." This reverts commit 241a119. * Removed meanImg calculations from extract again. * Moved chan2detect to detection package. Moved chan2detect call to detect. Successfully separated extract and detect subpackages. Added meanImg parameters to both detection/extraction tests. * Removed redcell from test_pipeline * changed import in chan2detect.py * BinaryFile.read() exports indices * moved batch_size argmument to read() mehtod of BinaryFile * changed BinaryFile's reg_file and raw_file parameters to read_file and write_file * removed unused options in BinaryFile * removed io.get_frames() * minor cleanup * simplified filename assignment * removed temp variable * refactored bidiphase * refactored meahgrid_mean_centered() out of gaussian_fft() and spatial_taper() * simplified convolve() * reordered import to clean up tuna stack trace * Refactor registor (#378) * split write_tiff() into generate_tiff_filename() and save_tiff() * cleaning register_to_binary() * removed ops from io.generate_tiff_filename() * rmeove unused import * pulled dim keywords out to parameters * readability * extracted prepare_ref_and_masks() from register_binary_to_ref() * pulled mean_img out of register_binary_to_ref() * minor refactor * moved file management to context manager * moved io from register_binary_to_ref to register_binary() * pulled ops out of init_offets * uplled print side effects out from register_binary_to_ref() to register_binary(). * pulled unused ops out of getSNR() * pulled most of ops out of phasecorr * pull ops out of one_photon_preprocess() * removed conditional assignment nameerror * changed if block to conditional assignment * removed newline * bugfix: added required parameter * moved if open/close blocks to context manager * moved print out to calling function * pulled ops out of upsample_block_Shifts() * pulled ops out of nonrigid.transform_data() * removed redundant calls to filter_gaussian1d() * extracted ops out of norigid.phasecorr() * simplified nonrigid check * pull io out of apply_shifts_to_binary() by changing to generator * pep8 * pulled ops out of apply_shifts * removed apply_shifts(). * pulled ops out of apply_shifts_to_binary() * moved most of ops out of compute_motion_and_shift() * rmeoved redundant data copying * made one_photon_preprocess_inplace() * pass data from inplace * simplified init_offsets() * moved register_binary() to main.py * removed unused variables * removed unused imports * move pca functions to pc.py * moved bin_paths() into register_binary() * removed unused time module * removed if branch * refactored out one_photon_preprocess_inplace() * one_photon_preprocess raises error instead of modifying its parameters. * changed ifelse branch to conditional assignment * moved ops out of get_frames() * pulled ops out of regid.phasecorr_reference() * bugfix: have compute_motion_and_shift() use smooth_sigma_time parameter instead of ops key * moved bidi parameters out of ops * moved 1p parameters out of ops in compute_motion_and_shift * seperated name of parameter from name of module. * pulled ops out of rigid.phasecorr() * pulled ops out of compute_motion_and_shift() * stopped yielding ops from register_binary_to_ref() * moved internal vars closer to used location * moved mean_img calculation out of register_binairy_to_ref() * moved progress logging out of register_binary_to_ref() * moved write functionality closer to read functionality * exposed offsets data structure by refactoring out init_offsets() * refactored out offsets assembly out of register_binary_to_ref() and into register_binary(). * removed unnecessary data rounding. * reduced data copying * pep8 * pulled ops out of nonrigid.make_blocks() * pulled ops out of compute_crop() * pulled ops out of nonrigid.phasecorr_reference() * refactored out iterative_alignment() * refactored out compute_reference_image into register_binary() * refactored out prepare_Refmasks() * removed docs mention of ops * removed unused, unworking function subsample_frames() * minor cleanup * pulled binary file management details out of register_binary_to_ref() * minor refactor * only save lower-resolution data * refactored meanImg calculation out of apply_shifts_to_binary() * removed rounding * removed data clamping * moved data yielding to mid-function * refactored out read/writing details from apply_shifts_to_binary() * regrouped computations in apply_shifts_to_binary() * moved BinaryFile to io module, refactored out register_binary_to_ref() and apply_shifts_to_binary() to main. * passed new tests * pulled ops out of pc_register() * removed unused Helper * pulled maskSlope out of phasecorr_reference() * pulled one_photon_preproces() out of nonrigid.phasecorr_reference() * pulled bidiphase.shift() out of compute_motion_and_shift() * pulled one_photon_preprocess() out of utils.py * removed unused functions sampled_mean(), sub2ind, and resample_frames() * removed unused imports * simplified get_frames() * moved io-bound get_frames() function to io/binary.py * simplified make_blocks() * removed unused function shift_data_subpixel() * fixed docstring and removed unused lines from rigid.shift_data() * refactored out 'offsets' data structure * removed redundant nonrigid.make_blocks() call. * changed print(warning) to warning.warn(), Exception to ValueException, * pulled 1Preg transformation out of rigid.phasecorr() * reordered re_1p n phasecorr_refernce() * split refAndMasks parameter in rigid.phasecorr() into its components and moved numba addmultiply() function to utils.py * removed unused gpu_phasecorr() * removed redundant spatial_smooth during nonrigid registration * refactored non-rigid registration into own function: nonrigid.shift() * pulled nonrigid.shift() out of compute_motion_and_shift() * removed commented-out code in compute_crop() * removed unused imports * refactored out RefAndMasks and RefAndMasksNR * refactored out phasecorr_cpu() * refactored out nonrigid.shift() * removed unused call to nonrigid.transform_data() * removed unused vars from nonrigid.phase_corr() * moved var use closer to declaration * Renamed io.write_tiff to changed io.save_tiff method in io tests. * split coupled "spatial_hp" ops parameter into "spatial_hp_reg" and "spatial_hp_detect". * refactored out compute_motion_and_correct() * removed redundant pre_smooth and spatial_hp value checking * register_binary() only takes a Dict ops. * structured nested if statements * used tuple unpacking on multiple assignment * removed temporary 'bidi' name and redundant nonrigid.make_blocks() call * removed unusable utils.split_multiops() line * removed redudnant np.save(ops) in register_binary() * moved depgraph scripts to own folder * refactored out repeated data_smooth conditionals * moved compute_zpos() back to zalign.py * cleaned up get_pc_metrics(): collapsed nsamp conditional logic, moved nPC to optional argument * moved ops-level functions back to their original modules * fixed paths * zipped rigid.shift_data() * pulled single-frame shift_frame() out of shift_data() * pulled iteration out of rigid.shift_frames(), reducing to simpler rigid.shift_frame() * pulled memory management out of rigid.shift_frame() into the calling function. * removed temporary variables from refImg mean shift step * removed shadow assignment step * vectorized fft2 and ifft2 steps in rigid.phase_corr() * used fancy indexing on cmax to vectorize cmax filling * removed temp variables for lcorr calculation * refactored out crop() function to remove intermediate vars * cleaned rigid.phasecorr_reference * split rigid.phascorr_reference() into independent functions phasecorr_reference() and compute_masks() * removed extra reference image copying * split rigid.phasecorr() into independent functions rigid.phasecorr() and rigid.apply_masks() * created utils.convolve() function to document fft2/ifft2 step and layer math concepts * removed hard-coded module literals * removed pre-computed module-level constants, replaced with cached function call. * spacing * refactored out nonrigid.clip() * made cc2 calculation more readable * cleaned up loop * cleaned up ymax1, xmax1, cmax1 calculation * vectorized nested fft2/ifft2 calculation * removed unused nonrigid.linear_interp() function * call duplicated addmultiply() in utils instead of nonrigid * removed extra data copy and unused variables in numba function * removed inplace parameter modification to rmeove need for guard copying * removed extra copies * reordered lines in nonrigid.transform_data() * shifted array management down from mat_upsample to kernel1d * refactored nonrigid.make_nblocks(), splitting out two algorithm functions and removed one pipelined argument * step reduction in nonrigid.phasecorr_reference() * moved gaussian kernel functions to utils.py * working in 3D, output in 4D * reduced preallocation steps in nonrigid.phasecorr_reference() * refactored getSNR * replaced fft2/ifft2 call with utils.convolve() * replace zeros() with empty() * moved append() * collapsed two `if ops['1Preg']` into one * composed rigid registration steps * removed redundant ops assignment and misleading comment * merged pre_smooth conditional * lowered gaussian_filter1d() to utils and replaced it with utils.temporal_smooth() * removed some squeeze() and newaxis * added explicit padding step to utils.spatial_smooth, plus a regression test * renamed intermediate steps * moved valueerror to algorithmic function, removed unnecessary [newaxis] -> squeeze() around spatial_smooth() calls * replace concatenate() algorithm with numpy.pad * removed unused ccsm output of nonrigid.phasecorr() * changed gaussian_filter1d call to temporal_smooth() * BinaryFile.read() exports indices * moved batch_size argmument to read() mehtod of BinaryFile * changed BinaryFile's reg_file and raw_file parameters to read_file and write_file * removed unused options in BinaryFile * removed io.get_frames() * minor cleanup * simplified filename assignment * removed temp variable * refactored bidiphase * ensured smoothing only alters high-SNR tiff by small amount and removed regression test as it fails due to OG bug * added smoothed version for non-rigid computation Co-authored-by: Chris Ki <chriski777@berkeley.edu> Co-authored-by: Carsen Stringer <carsen.stringer@gmail.com> * Separated classification from extraction (#398) * Initialized classify.py and added classifcation calls. * Removed classification from extraction subpackage. * Modified tests to include changes to both extraction and classification. Also, added a regression test to check new classify function. * Added classifcation section with appropriate classify call. * moved timing and print out of make_masks * moved roi_stats to a new stats.py module * DVC Pull doesn't require authentication anymore. (#400) * Added dvc authentification for dummy test account * ensured smoothing only alters high-SNR tiff by small amount and removed regression test as it fails due to OG bug * added smoothed version for non-rigid computation Co-authored-by: Carsen Stringer <carsen.stringer@gmail.com> * refactored get_stats() * refactored get_stats() * seperated mrs normalization * pulled ops out of roi_stats() * Fix reg offsets (#407) * Added dvc authentification for dummy test account * ensured smoothing only alters high-SNR tiff by small amount and removed regression test as it fails due to OG bug * added smoothed version for non-rigid computation * Initialized registration metric benchmark. Moved tests in test_registration to registration_module test. * add random_state parameter to pc_metrics * Added registration metrics script * Changed batch_size to not be 1500 in metrics test. * Removed unused data dvc files. Added new input tif with 1500 frames and bidi_shift. Added batch_specific reg_tifs. * Consolidated bidi_shift test with metrics test. * Revert "Consolidated bidi_shift test with metrics test." This reverts commit 29221ef. * Revert "Removed unused data dvc files. Added new input tif with 1500 frames and bidi_shift. Added batch_specific reg_tifs." This reverts commit 1c18a95. * Removed registration metrics test. * Removed registration smoothing output test data * Removed input_1500 tif used for registration tests. * Added input1500_bidi_shifted tif * Removed bidi_shifited 1500 input data * Add multi-batch test. * Removed binary files for test_data. Changed test name in pipeline_tests. * Added 1500 input dataset. * Added input_1500 dvc file * io test now only takes in input.tif. Test_pipeline now refers to correct input file. * corrected test_data name * Rigid offset shifts fixed for multiple batches. * Added 2plane2chan test with multiple batches. * Added missing F.npy test_data * Added function to combine offsets across batches. Can now combine offset values if batches have different numbers of frames. Co-authored-by: Carsen Stringer <carsen.stringer@gmail.com> * pulled ops and in-place stats assignment our of out of get_overlaps(), added count_overlaps() function. * removed stats from get_overlaps() and count_overlaps() * removed parameter tunneling from get_overlaps() * name fix * moved distance_kernel() to utils.py * simplified remove_overlappers() * removed ops form remove_overlappers() * changed: while 1: remove to: for(reversed): append * added bug message * changed output of remove_overlappers from list[dict] to list[int] * pulled stats out of remove_overlappers() * Update nonrigid.py * Update register.py * Update zalign.py * DVC Pull doesn't require authentication anymore. (#400) * Added dvc authentification for dummy test account * ensured smoothing only alters high-SNR tiff by small amount and removed regression test as it fails due to OG bug * added smoothed version for non-rigid computation Co-authored-by: Carsen Stringer <carsen.stringer@gmail.com> * Fix reg offsets (#407) * Added dvc authentification for dummy test account * ensured smoothing only alters high-SNR tiff by small amount and removed regression test as it fails due to OG bug * added smoothed version for non-rigid computation * Initialized registration metric benchmark. Moved tests in test_registration to registration_module test. * add random_state parameter to pc_metrics * Added registration metrics script * Changed batch_size to not be 1500 in metrics test. * Removed unused data dvc files. Added new input tif with 1500 frames and bidi_shift. Added batch_specific reg_tifs. * Consolidated bidi_shift test with metrics test. * Revert "Consolidated bidi_shift test with metrics test." This reverts commit 29221ef. * Revert "Removed unused data dvc files. Added new input tif with 1500 frames and bidi_shift. Added batch_specific reg_tifs." This reverts commit 1c18a95. * Removed registration metrics test. * Removed registration smoothing output test data * Removed input_1500 tif used for registration tests. * Added input1500_bidi_shifted tif * Removed bidi_shifited 1500 input data * Add multi-batch test. * Removed binary files for test_data. Changed test name in pipeline_tests. * Added 1500 input dataset. * Added input_1500 dvc file * io test now only takes in input.tif. Test_pipeline now refers to correct input file. * corrected test_data name * Rigid offset shifts fixed for multiple batches. * Added 2plane2chan test with multiple batches. * Added missing F.npy test_data * Added function to combine offsets across batches. Can now combine offset values if batches have different numbers of frames. Co-authored-by: Carsen Stringer <carsen.stringer@gmail.com> * Update nonrigid.py * Update register.py * Update zalign.py * removed intermediate var * elevated some of ops keys out of select_rois() * removed unused stats argument from select_rois() * made functions calc_radii() and aspect_ratio() * minor cleanuip of fitMVGaus * EllipseData namedtuple output of fitMVGaus(), to make output more readable. Also reduced amount of axis manipulation in function. * refactored create_neurpil_masks() for readability * changed for-break to while * pulled stats out of create_neuropil_masks() * used from...import to ease function move * removed code duplication in chan2detect by moving make_masks() to masks.py * made sparse_mode a parameter of select_rois()( * pulled high_pass out of ops and refactored out 2 new functions: high_pass_gaussian_filter() and high_pass_rolling_mean_filter(). * minor bin_movie() readability refactor * used norm() * added __getitem__() to BinaryFile, resolved merge conflict in register.py * added crop slicing to BinaryFile * minor changes * put BinaryFile in bin_movie() * used built-in index tracking * moved timing and logging out of bin_movie * moved bin_size to parameter of bin_movie * pulled bad_frames out of bin_movie, added n_frames, shape, and size properties to BinaryFile * pulled print() out of utils function * moved ops crop keys out of utils.bin_movie() * added comments * pulled ops['reg_file'] out of utils.bin_movie() * pulled ops out of bin_movie() * get_sdmov() readability * moved operations to loop * removed data copying step * made functions for reject_frames() and crop() operations * refactored threshold_reduce() * vectorized downsample() function * fixed downsample * merge conflict resolved, moved fixed downsample() to utils.py * moved downsample() and threshold_reduce() to utils.py, minor readability refactor for neuropil_subtraction * refactored square_conv2() for readability * refactored square_conv2() for readability * fixed normalization order in sd_mov() * pulled ops out of make_masks() * split make_masks() * split create_cell_masks out into create_Cell_pix() * removed get_overlaps() * Added testing page to suite2p wiki and nwb test. (#412) * Added test setup/run page to suite2p readthedocs * Fixed up nwb.py. Removed unnecessary import. * Changes to test util functions. Added functions to retrieve list of output data and test data. * Added nwb test to 1plane1chan test on pipeline tests. * added pynwb to test setup * unpacked dx and dy into roi_stats for nwb test * minor refactor, create_cell_masks() * minor refactor * DRY'd roi stat norming steps with norm_by_average() function * changed name 'radius' to 'radii' when tuple * minor type documentation + norm() replacement * roi stat aggregation dataclass: ROI * pulled roi_stats() out of detect.py and added PendingDeprecationWarning. * moved roi calc functions back to stats.py * added ROI.fit_ellipse interface, with cached-property to keep performance benefits. * minor: removed rsort from repr * readability refactor: dict.update() instead of repeated dist.__setitem__() * whitespace * moved normailizatoin steps with calculation steps * renamed remove_overlappers() to filter_overlappers() to reflect its keeping behavior, move overlap assignment to main loop. * added overlap normalization and filtering interface to ROI as classmethod. * moved overlaps() calculations to stats.py * filter in loop * changed filter_overlappers output from list of indices to list of bools, for simpler logical indexing. * ROI interface to filtering. * added type annotation support for docs * Registration Benchmark script added (#421) * Implement click package in registration_metrics script * Moved random_state assignment to ops level. * Added optional parameters to registration benchmark script * Included more registration metrics results * fixed regPC * Fix import for gui2p error * Refactored cli script in __main__.py * Removed dependence of benchmark script on click package. Instead relying on main cli script in suite2p. * removed npc as a parameter in get_pc_metrics. Rely on ops assignment * plot avg and max scores of top 10 PCs * Fixed registration metrics to display statistics on offset scores (not PC weights). * parallelizing multiplane processing (#423) * GUI now working on dev branch (tqdm removed + default_ops() function called correctly) #424 * bug with running __main__ script from gui call - reverted to old version * Renamed default_ops fixture to test_ops. (#429) * bug with int16 conversion not being forced (#428) * Fixes reg_tif issue with second channel (#431) * Removed 2plane2chan test from pipeline tests. * check_output test util function now compares tifs. Added reg_tif check for 1plane1chan test. * Added reg_tif data for 1plane1chan1500 * Removed 1plane1chan1500 reg_tifs. * Removed reg_tif option for 1plane1chan test * Add test to check reg_tif_chan2 for 2plane2chan_batch test * reset k for second channel in register.py. Fixed up frame number for batches. * Added 2plane2chan reg_tifs * Refactored __main__.py so benchmark script works. * Update __main__.py * nbytesread must be an int64 for files longer than ~4000 frames * speeding up sparsedetect to normal levels * Refactored registration_metrics into View-Model-Presenter functions and added reg_metrics entry point * Fixed issue with main script not setting input ops parameter. Co-authored-by: Chris Ki <chriski777@berkeley.edu> Co-authored-by: Nicholas A. Del Grosso <delgrosso.nick@gmail.com> * Cleaner text output Co-authored-by: Chris Ki <chriski777@berkeley.edu> Co-authored-by: Nicholas A. Del Grosso <delgrosso.nick@gmail.com> * removed cached_properties from package. (#434) * fixing some bugs with keep_movie_raw and classifier without all keys * Ref det3 (#435) * ROI interface to overlap location * reordered mask application, added ROI.ravel_indices() * minor readability * moved norm_by_average() to stats.py * moved repeated appends (xpix, ypix, lam) to single append (stats) * removed intermediate ops['Vmap'] construction * removed extra ops['Vmap'] appends and extra copy calls. * moved high_pass to parameter * moved bin_movie to io.binary, and moved io operations in detection module to main_datect() function * moved batch_size and neuropil_high_pass to parameters, abreviated "high-pass" to "hp" in utils functions * renamed 'rez' to 'mov' for consistency * moved max_iterations and threshold_scaling to parameters * moved ops assignments to end of function. * removed ops input from sparsery(), returns new dict and applies it outside of the function instead of overwriting ops in-place * added iteration for gxy, refactored out find_best_scale() function and estimate mode message type from sparsery() * minor whitespace * changed create_cell_masks() to work on a single ROI at a time. * refactored out high-level hp_filter selection step as temporal_high_pass_filter(), moved max_proj assignment to select_rois() for sparsery() * moved temporal_hp back into sparsery() refactored out spatial scale computation into own function. * minor * Moved ellipse properties from ROI to EllipseData Co-authored-by: Chris Ki <chriski777@berkeley.edu> Co-authored-by: Nicholas A. Del Grosso <delgrosso.nick@gmail.com> Co-authored-by: Chris Ki <chriski777@berkeley.edu> Co-authored-by: Carsen Stringer <carsen.stringer@gmail.com> * bug in classify import * adding ops['preclassify'] back into pipeline * Manual ROI issues fixed. (#439) * typing changes for stats.py * Traces now can be extracted from manual_rois. Fix made in drawroi.py * Overlap is already calculated in roi_stats. Hence, can remove overlap calculation when gui is closed. * Include MacOS in travis build. (#441) * Removed tqdm imports * Add osx to travis. allow for osx failure * pydrive2 instructions for developers. * Separated jobs for OSes in .travis.yml * have different conda installs for OSes in .travis.yml * changed python version for mac on travis * modifying setup script * issue with not rescaling lambda Co-authored-by: Nicholas A. Del Grosso <delgrosso.nick@gmail.com> Co-authored-by: Chris Ki <chriski777@berkeley.edu> Co-authored-by: Marius Pachitariu <marius10p@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
PR (addresses #395) contains the following:
sparsedetect,sourcery,masksandchan2detectwere moved to the detect subpackage.meanImgcalculation from extract package. Extract was calculating ameanImgthat was the same as themeanImgfound in the io module.Dependency graph below shows the separation of the two subpackages: