DM-17956: Add option to output chi2 files at each outer fit iteration #127
Conversation
There was a problem hiding this comment.
Sorry for the delay on the review. Only minor comments here, but after looking at this I do think the code would benefit overall from writing these out via afw.table (but not on this ticket). You'd have to rewrite the core C++ code that dumps the results, but then you could write them with the butler quite easily (I imagine you'd have a data ID with tract, filter, and iteration data ID keys, with the latter whatever it needs to be to disambiguate). You'd then be able to drop all of the custom code here to generate filenames and probably save a lot on storage.
| FloatingPointError | ||
| Raised if chi2 is infinite or NaN. | ||
| ValueError | ||
| Raised if the model is not valid. |
There was a problem hiding this comment.
Seems like it'd be desirable to write the diagnostics even (especially) when one of these errors occurs.
| @@ -182,6 +188,49 @@ def test_load_reference_catalog(self): | |||
| self.assertEqual(refCat, fakeRefCat) | |||
|
|
|||
|
|
|||
| class TestJointcalFitModel(JointcalTestBase, lsst.utils.tests.TestCase): | |||
| def test_fit_photometry_writeChi2(self): | |||
There was a problem hiding this comment.
Hate to pick on this, but this should be camelCase, right?
python/lsst/jointcal/jointcal.py
Outdated
| @@ -723,6 +723,7 @@ def _logChi2AndValidate(self, associations, fit, model, chi2Label="Model", | |||
| raise ValueError("Model is not valid: check log messages for warnings.") | |||
| if writeChi2Name is not None: | |||
| fit.saveChi2Contributions(writeChi2Name+"{type}") | |||
| self.log.info("Wrote chi2 contributions files: %s"%writeChi2Name) | |||
There was a problem hiding this comment.
If you use a comma instead of %, I think the logger will only do the string interpolation if the appropriate log level is enabled. Doesn't matter for this case, but it's a good habit to get into.
There was a problem hiding this comment.
Thanks for the reminder. I'd forgotten about that.
Add logging of written chi2 filename, and log before exceptions are raised to aid debugging.7
These are only called by the public `saveChi2Contributions`, which manages the filenames appropriately.
No outliers are rejected in these steps, but they move very far in chi2-space because they are purely linear (or very nearly so) in the fitted parameters. Added some tests (using mocks) to check that the filename prefixes are reasonable.
parejkoj
force-pushed
the
tickets/DM-17956
branch
from
8da17c4
to
29627c8
Compare
No description provided.