Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Speedup 04: quick trace selection #526

Merged

Conversation

flixha
Copy link
Collaborator

@flixha flixha commented Dec 12, 2022

What does this PR do?

  • 4x / 100x speedup for trace selection in detection, lag_calc, pre_processing

Why was it initiated? Any relevant Issues?

  • trace selection by trace id from a stream is still something that takes more time than it needs to when working with many thousand (100K?) traces, even after ~10x speedup for selecting traces from Stream by non-wildcarded trace ID obspy/obspy#2886.
  • When a user doesn't need wildcard selection, than the simple selection function can be sped up by a factor of 4 compared to obspy.stream.select
  • For the time when the order of traces in a stream does not change and a lot of traces need to be selected from that stream, then it's possible to speed up the selection by a factor of ~100 with a dict-lookup - this is very relevant for the template preparation in _prep_data_for_correlation

This PR contributes to the summary issue in #522

PR Checklist

  • develop base branch selected?
  • This PR is not directly related to an existing issue (which has no PR yet).
  • All tests still pass.
    - [ ] Any new features or fixed regressions are be covered via new tests.
  • Any new or changed features have are fully documented.
  • Significant changes have been added to CHANGES.md.
    - [ ] First time contributors have added your name to CONTRIBUTORS.md.

Copy link
Member

@calum-chamberlain calum-chamberlain left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great to me and tests were passing. Nice speedup!

@calum-chamberlain calum-chamberlain merged commit 818e293 into eqcorrscan:develop Jan 1, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants