-
Notifications
You must be signed in to change notification settings - Fork 46
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Change default separate_panels=False #1424
Conversation
When separate_panels=True, outlier rejection is performed for each panel separately prior to refinement. This is appropriate when we expect the data on each panel to be drawn from different distributions (for example prior to metrology corrections). However, it causes a large increase in the execution time of refinement - dominating the overall job time for a test case using data from I23. This parameter should be set to true when doing metrology refinement, but for normal data processing use, false is likely to be acceptable.
Codecov Report
@@ Coverage Diff @@
## master #1424 +/- ##
==========================================
- Coverage 65.29% 65.27% -0.03%
==========================================
Files 617 617
Lines 69621 69617 -4
Branches 9560 9558 -2
==========================================
- Hits 45460 45443 -17
- Misses 22324 22334 +10
- Partials 1837 1840 +3 |
This comment made me want to verify how we are using it in xfel land. And yah, it's set to True for metrology refinement: Whereas for ensemble refinement (which refines the detector as a whole for batches of data), it's set to False: |
That seems right. It makes sense to do the same for scans as stills too. |
@rjgildea are you happy with this for I23 processing? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This seems like a sensible change. Unless you are explicitly doing metrology refinement, it seems sensible to assume that the metrology is sufficiently close to reality that all centroids come from the same distribution. It certainly makes a notable difference to the runtime in some cases.
When separate_panels=True, outlier rejection is performed for each panel
separately prior to refinement. This is appropriate when we expect the
data on each panel to be drawn from different distributions (for example
prior to metrology corrections). However, it causes a large increase in
the execution time of refinement - dominating the overall job time for
a test case using data from I23.
This parameter should be set to true when doing metrology refinement,
but for normal data processing use, false is likely to be acceptable.