Speedup 10: 25 % quicker _get_array_dicts and 10 % quicker _prep_data_for_correlation #536
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Implements:
utils.correlate._get_array_dicts
(from 200 s to 154 seconds total time spent in my test)utils.pre_processing._prep_data_for_correlation
(from 702 s to 640 s total time spent)Main speedups with:
UTCDateTime.__dict__['_UTCDateTime__ns']
(5x quicker for one +/- operation)stream.traces[j]
instead ofstream[j]
(4.5x quicker)Why was it initiated? Any relevant Issues?
For big datasets run on powerful hardware, the cumulative time spent in
utils.correlate._get_array_dicts
andutils.pre_processing._prep_data_for_correlation
makes up a considerable part (>50 %) of a detection run. This PR tries to optimize the functions with some simple changes; I think further speed ups would need changes that are not so trivial...This PR is part of the list of speedups in #522
PR Checklist
develop
base branch selected?- [ ] Any new features or fixed regressions are be covered via new tests.- [ ] Any new or changed features have are fully documented.CHANGES.md
.- [ ] First time contributors have added your name toCONTRIBUTORS.md
.