-
Notifications
You must be signed in to change notification settings - Fork 7
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Speedup jointcal and separate astrometry/photometry tickets/DM-8552 #27
Commits on Jan 24, 2017
-
Big speed improvement and cleanup dead ccdImage code
Only read the calexps when building the ccdImages (and not when building the output to write) and save the Calib to its respective ccdImage to use it when preparing output. Use that calib in ccdImage to convert flux to magnitude, instead of custom code. No need for the calexp_md now: just extract the values from the calexp (eventually this should come from some metadata datasetType, e.g. `get('info')`). Add SOURCE_IO_NO_FOOTPRINTS flag to get('src') for a big speedup, since I don't need the heavy footprints (thanks @TallJimbo for the suggestion). Add individual profiling for each slow step, activated via profile_jointcal option. Remove a bunch of variables and getters from ccdImage that are totally unused.
Configuration menu - View commit details
-
Copy full SHA for dab02ba - Browse repository at this point
Copy the full SHA dab02baView commit details -
Remove skipped tests for fewer visits.
With plot_jointcal_results, I can now extract statistics from any jointcal run, so I don't need those tests sitting around here.
Configuration menu - View commit details
-
Copy full SHA for cf60c48 - Browse repository at this point
Copy the full SHA cf60c48View commit details
Commits on Feb 16, 2017
-
photometry/astrometry can be run independently, with passing tests.
Split photo/astro code with Config parameters, and a test for each option in the lsstSim test class. The tests check for empty Wcs/Calib, if they shouldn't have been written. Pulled apart _testJointcalTask so I could use the run/plot bits in the "only one" tests, since those have quite different assert conditions. Also support the split in JointcalStatistics: the nested ifs are a bit excessive, but they get the job done. Fixed inconsistent capitalization of jointcal in tests.
Configuration menu - View commit details
-
Copy full SHA for 40b323e - Browse repository at this point
Copy the full SHA 40b323eView commit details -
Make refObjLoaders configurable
Add astrometry/photometryRefObjLoaders as configurables and make jointcal.__init__() take a butler argument to configure them. JointcalRunner has to be derived from ButlerInitializedTaskRunner to get said butler. Clean up "default" filter handling somewhat (see DM-9093 for what else needs to be done), to use the most common filter in the dataRefs. CollectLSSTRefStars now takes the actual fluxField, instead of building it on the fly.
Configuration menu - View commit details
-
Copy full SHA for 82456de - Browse repository at this point
Copy the full SHA 82456deView commit details -
astro/photo split now compiles and runs; new test metrics.
Add jointcal._do_load_refcat_and_fit() to do both photometry and astrometry via kwargs. Add metrics dictionary in jointcal.py, for basic values to test, that may turn into validation/verification SQUASH tests later. associateCatalogs now properly cleans up when run a second time. This would be best handled via better memory management, but that's DM-4043. Fix some range-based for loops and cleaned up pointers/references. Remove bit-rotted Jointcal class and other unused code. More renaming of badly-named methods and variables. Filed DM-9135 to deal with this in bulk. Print some interesting values in when running JointcalStatistics verbosely. Fixed a bug I had introduced in RefStar re:setFittedStar behavior with nullptr.
Configuration menu - View commit details
-
Copy full SHA for ac9d1b1 - Browse repository at this point
Copy the full SHA ac9d1b1View commit details -
Configuration menu - View commit details
-
Copy full SHA for f5ea857 - Browse repository at this point
Copy the full SHA f5ea857View commit details -
Add GAIA refcat and no-photo/no-astro HSC tests
Add hsc config for filterMap and tweak test to work with it. Add every star number and chi2 metric for every test dataset. Metrics tests always occur in JointcalTestBase._runJointcalTask(). Default to checking every metric that gets returned, so they'll fail if that metric isn't defined by the test method. A metric can be skipped by setting it to None.
Configuration menu - View commit details
-
Copy full SHA for ff8115e - Browse repository at this point
Copy the full SHA ff8115eView commit details -
Configuration menu - View commit details
-
Copy full SHA for cf00394 - Browse repository at this point
Copy the full SHA cf00394View commit details -
Configuration menu - View commit details
-
Copy full SHA for 55c7fb3 - Browse repository at this point
Copy the full SHA 55c7fb3View commit details -
Configuration menu - View commit details
-
Copy full SHA for 5f70e89 - Browse repository at this point
Copy the full SHA 5f70e89View commit details -
Configuration menu - View commit details
-
Copy full SHA for 2473da4 - Browse repository at this point
Copy the full SHA 2473da4View commit details -
Compute common tangent plane after all ccdImages are loaded
Tweak metrics after solving common tangent plane problem.
Configuration menu - View commit details
-
Copy full SHA for 16c7c0d - Browse repository at this point
Copy the full SHA 16c7c0dView commit details -
sort dataRefs for consistent py2/3 ordering
tweak metrics for sorted dataRefs
Configuration menu - View commit details
-
Copy full SHA for 6ec66b3 - Browse repository at this point
Copy the full SHA 6ec66b3View commit details