-
Notifications
You must be signed in to change notification settings - Fork 86
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Speedup 01 Quicker grouping of templates #524
Speedup 01 Quicker grouping of templates #524
Conversation
NCEDC - FDSN not answering properly in tests.. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks great to me! thanks for this.
@@ -586,6 +586,19 @@ def test_tribe_copy(self): | |||
self.assertEqual(t, copy_t) | |||
self.assertEqual(tribe, copied) | |||
|
|||
def test_tribe_copy_quick(self): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This looks like the wrong test for this PR.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ups.. thanks for removing!
Looks good to me @flixha - point noted on the if key in ["name", "st", "prepick", "event", "template_info"]:
continue
else:
actually compare if logic in |
Very good suggestion. Now I defined the let me know if this is not as you were suggesting.. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice, that is what I was thinking, glad you agree. Looks like that should be simpler to maintain.
What does this PR do?
Gives 50x speed up for grouping templates.
Why was it initiated? Any relevant Issues?
Grouping many templates was slow because the function was using a double loop for template-parameter comparison (O^2). So grouping took ~20 s for ~2000 templates. Sped up single loop to <0.5 s.
The new, quick function now circumvents the repeated calls to
template.same_processing
that contribute to the slowdown. So if we change whattemplate.same_processing
does (i.e., which dict-entries it compares), then we'd also need to change it here. Left in the old function for testing for now.This PR contributes to the summary issue in #522
PR Checklist
develop
base branch selected?- [ ] Any new features or fixed regressions are be covered via new tests.- [ ] Any new or changed features have are fully documented.CHANGES.md
.- [ ] First time contributors have added your name toCONTRIBUTORS.md
.