DM-49364: Switch LATISS/bestEffortIsr to IsrTaskLSST

[mfisherlevine]

No description provided.

[czwa]

I'm not going to make the change request mandatory, but unless there's a compelling reason not to doApplyGains=True, I think that will improve things.

[czwa]

Unless you expect to never have a PTC, I would strongly suggest this be set to doApplyGains=True. The new ISR defaults largely assume that everything has had gains applied so units are in electrons. We also should have CTI calibrations in the next set, but that correction is far less likely to have visual impact on the images than doApplyGains (which should help remove amp offsets).

[mfisherlevine]

Oh, for sure - totally agreed! This is all ancient stuff that used to be necessary years ago, I'm up for any/all changes like that 👍.

[github-actions]

mypy-coverage report

❌ 166 unannotated, 25 partial definition(s).

• Root: /home/runner/work/summit_utils/summit_utils
• Config: mypy.ini
• Files scanned: 60
• Files excluded: 0

Summary

metric	value
❌ body-checked by mypy	78.6%
❌ fully annotated	75.4%
annotated	584
partial	25
unannotated	166

Files with gaps

file	fully typed %	body checked %	unannotated	partial
python/lsst/summit/utils/astrometry/anet.py	93.8%	100.0%	0	1
python/lsst/summit/utils/bestEffort.py	83.3%	100.0%	0	1
python/lsst/summit/utils/consdbClient.py	77.3%	100.0%	0	5
python/lsst/summit/utils/guiders/plotting.py	96.3%	96.3%	1	0
python/lsst/summit/utils/guiders/reading.py	84.6%	88.5%	6	2
python/lsst/summit/utils/guiders/transformation.py	68.0%	80.0%	5	3
python/lsst/summit/utils/imageExaminer.py	95.8%	100.0%	0	1
python/lsst/summit/utils/nightReport.py	96.0%	100.0%	0	1
python/lsst/summit/utils/plotRadialAnalysis.py	72.7%	100.0%	0	3
python/lsst/summit/utils/quickLook.py	66.7%	100.0%	0	2
python/lsst/summit/utils/simonyi/mountAnalysis.py	75.0%	87.5%	1	1
python/lsst/summit/utils/simonyi/mountData.py	80.0%	80.0%	1	0
python/lsst/summit/utils/spectrumExaminer.py	93.3%	100.0%	0	1
python/lsst/summit/utils/tmaUtils.py	93.8%	98.5%	1	3
python/lsst/summit/utils/utils.py	97.9%	100.0%	0	1
tests/test_bestEffortIsr.py	25.0%	25.0%	6	0
tests/test_blockUtils.py	40.0%	40.0%	6	0
tests/test_butlerUtils.py	8.1%	8.1%	34	0
tests/test_consdbClient.py	0.0%	0.0%	9	0
tests/test_dateTime.py	33.3%	33.3%	4	0
tests/test_efdUtils.py	15.4%	15.4%	11	0
tests/test_guider.py	94.4%	94.4%	1	0
tests/test_ics.py	33.3%	33.3%	4	0
tests/test_imageExaminer.py	40.0%	40.0%	3	0
tests/test_nightReport.py	10.5%	10.5%	17	0
tests/test_plotting.py	25.0%	25.0%	3	0
tests/test_quickLook.py	30.8%	30.8%	9	0
tests/test_tmaUtils.py	10.3%	10.3%	26	0
tests/test_utils.py	32.0%	32.0%	17	0
tests/utils.py	0.0%	0.0%	1	0

Unannotated definitions (166)

python/lsst/summit/utils/guiders/plotting.py

• L355 GuiderPlotter.stripPlot._zero (function)

python/lsst/summit/utils/guiders/reading.py

• L62 make_subplot (function)
• L536 GuiderData.__iter__ (method)
• L540 GuiderData.items (method)
• L545 GuiderData.keys (method)
• L549 GuiderData.values (method)
• L554 GuiderData.__getitem__ (method)

python/lsst/summit/utils/guiders/transformation.py

• L473 stampToCcd (function)
• L872 DriftResult.__str__ (method)
• L902 DriftResult.summary (method)
• L914 getObsAltAz (function)
• L953 DeltaAltAz (function)

python/lsst/summit/utils/simonyi/mountAnalysis.py

• L283 plotMountErrors.offset_time_aware (function)

python/lsst/summit/utils/simonyi/mountData.py

• L122 getAzElRotHexDataForPeriod.calcDeltaT (function)

python/lsst/summit/utils/tmaUtils.py

• L190 getAzimuthElevationDataForEvent.calcDeltaT (function)

tests/test_bestEffortIsr.py

• L33 BestEffortIsrTestCase.setUpClass (method)
• L43 BestEffortIsrTestCase.test_getExposure (method)
• L52 BestEffortIsrTestCase.test_getExposureFromExpRecord (method)
• L70 BestEffortIsrTestCase.test_raises (method)
• L77 BestEffortIsrTestCase.test_quicklook_connections (method)
• L91 setup_module (function)

tests/test_blockUtils.py

• L112 BlockParserTestCase.setUpClass (method)
• L128 BlockParserTestCase.tearDown (method)
• L134 BlockParserTestCase.test_parsing (method)
• L157 BlockParserTestCase.test_notFoundBehavior (method)
• L185 BlockParserTestCase.test_actualValues (method)
• L208 setup_module (function)

tests/test_butlerUtils.py

• L70 ButlerUtilsTestCase.setUp (method)
• L117 ButlerUtilsTestCase.test_getLatissDefaultCollections (method)
• L123 ButlerUtilsTestCase.test_RECENT_DAY (method)
• L142 ButlerUtilsTestCase.test_sanitizeDayObs (method)
• L152 ButlerUtilsTestCase.test_getMostRecentDayObs (method)
• L162 ButlerUtilsTestCase.test_getSeqNumsForDayObs (method)
• L173 ButlerUtilsTestCase.test_getMostRecentDataId (method)
• L182 ButlerUtilsTestCase.test_getDatasetRefForDataId (method)
• L191 ButlerUtilsTestCase.test__dayobs_present (method)
• L199 ButlerUtilsTestCase.test__seqnum_present (method)
• L207 ButlerUtilsTestCase.test__expid_present (method)
• L215 ButlerUtilsTestCase.test_getDayObs (method)
• L224 ButlerUtilsTestCase.test_getSeqNum (method)
• L233 ButlerUtilsTestCase.test_getExpId (method)
• L242 ButlerUtilsTestCase.test_datasetExists (method)
• L248 ButlerUtilsTestCase.test_sortRecordsByDayObsThenSeqNum (method)
• L271 ButlerUtilsTestCase.test_getDaysWithData (method)
• L277 ButlerUtilsTestCase.test_getExpIdFromDayObsSeqNum (method)
• L282 ButlerUtilsTestCase.test_updateDataIdOrDataCord (method)
• L292 ButlerUtilsTestCase.test_fillDataId (method)
• L308 ButlerUtilsTestCase.test_getExpRecordFromDataId (method)
• L313 ButlerUtilsTestCase.test_getDayObsSeqNumFromExposureId (method)
• L319 ButlerUtilsTestCase.test_removeDataProduct (method)
• L323 ButlerUtilsTestCase.test_getLatissOnSkyDataIds (method)
• L348 ButlerUtilsTestCase.test__assureDict (method)
• L361 ButlerUtilsTestCase.test__get_dayobs_key (method)
• L370 ButlerUtilsTestCase.test__get_seqnum_key (method)
• L379 ButlerUtilsTestCase.test__get_expid_key (method)
• L388 ButlerUtilsTestCase.test_updateDataId (method)
• L405 ButlerUtilsTestCase.test_getExpRecord (method)
• L434 ButlerInitTestCase.test_dafButlerRaiseTypes (method)
• L456 ButlerInitTestCase.test_makeDefaultLatissButlerRaiseTypes (method)
• L477 ButlerInitTestCase.test_DAF_BUTLER_REPOSITORY_INDEX_value (method)
• L489 setup_module (function)

tests/test_consdbClient.py

• L30 client (function)
• L37 test_table_name (function)
• L47 test_add_flexible_metadata_key (function)
• L105 test_get_flexible_metadata_keys (function)
• L120 test_get_flexible_metadata (function)
• L147 test_insert_flexible_metadata (function)
• L156 test_schema (function)
• L177 test_clean_token_url_response (function)
• L200 test_client (function)

tests/test_dateTime.py

• L41 DateTimeTestCase.test_getDayObsAsTimes (method)
• L64 DateTimeTestCase.test_calcDayOffset (method)
• L75 DateTimeTestCase.test_getDayObsForTime (method)
• L92 setup_module (function)

tests/test_efdUtils.py

• L63 EfdUtilsTestCase.setUpClass (method)
• L87 EfdUtilsTestCase.tearDown (method)
• L93 EfdUtilsTestCase.test_makeEfdClient (method)
• L97 EfdUtilsTestCase.test_getTopics (method)
• L117 EfdUtilsTestCase.test_getEfdData (method)
• L176 EfdUtilsTestCase.test_raiseIfTopicNotInSchema (method)
• L189 EfdUtilsTestCase.test_getMostRecentRowWithDataBefore (method)
• L199 EfdUtilsTestCase.test_efdTimestampToAstropy (method)
• L204 EfdUtilsTestCase.test_astropyToEfdTimestamp (method)
• L211 EfdUtilsTestCase.test_clipDataToEvent (method)
• L264 setup_module (function)

tests/test_guider.py

• L293 setup_module (function)

tests/test_ics.py

• L44 M1M3ICSTestCase.setUp (method)
• L60 M1M3ICSTestCase.tearDown (method)
• L66 M1M3ICSTestCase.test_analysis (method)
• L97 setup_module (function)

tests/test_imageExaminer.py

• L35 ImageExaminerTestCase.setUpClass (method)
• L49 ImageExaminerTestCase.test_imageExaminer (method)
• L67 setup_module (function)

tests/test_nightReport.py

• L44 NightReportTestCase.setUpClass (method)
• L64 NightReportTestCase.test_saveAndLoad (method)
• L78 NightReportTestCase.test_getSortedData (method)
• L89 NightReportTestCase.test_getExpRecordDictForDayObs (method)
• L106 NightReportTestCase.test_getObsInfoAndMetadataForSeqNum (method)
• L116 NightReportTestCase.test_rebuild (method)
• L127 NightReportTestCase.test_getExposureMidpoint (method)
• L148 NightReportTestCase.test_getTimeDeltas (method)
• L154 NightReportTestCase.test_makeStarColorAndMarkerMap (method)
• L165 NightReportTestCase.test_printObsTable (method)
• L175 NightReportTestCase.test_plotPerObjectAirMass (method)
• L191 NightReportTestCase.test_makeAltAzCoveragePlot (method)
• L205 NightReportTestCase.test_calcShutterTimes (method)
• L213 NightReportTestCase.test_getDatesForSeqNums (method)
• L219 NightReportTestCase.test_doesNotRaise (method)
• L230 NightReportTestCase.test_internals (method)
• L250 setup_module (function)

tests/test_plotting.py

• L45 PlottingTestCase.setUpClass (method)
• L49 PlottingTestCase.test_plot (method)
• L96 setup_module (function)

tests/test_quickLook.py

• L40 QuickLookIsrTaskTestCase.setUp (method)
• L60 QuickLookIsrTaskTestCase.test_runQuickLook (method)
• L82 QuickLookIsrTaskTestCase.test_runQuickLookMissingData (method)
• L87 QuickLookIsrTaskTestCase.test_runQuickLookBadDark (method)
• L109 QuickLookIsrTaskRunQuantumTests.setUp (method)
• L235 QuickLookIsrTaskRunQuantumTests.tearDown (method)
• L238 QuickLookIsrTaskRunQuantumTests.test_runQuantum (method)
• L292 raiseExitMockError (function)
• L307 setup_module (function)

tests/test_tmaUtils.py

• L62 getTmaEventTestTruthValues (function)
• L88 writeNewTmaEventTestTruthValues (function)
• L116 makeValid (function)
• L123 _turnOn (function)
• L139 TmaUtilsTestCase.test_tmaInit (method)
• L155 TmaUtilsTestCase.test_tmaReferences (method)
• L169 TmaUtilsTestCase.test_getAxisAndType (method)
• L181 TmaUtilsTestCase.test_initStateLogic (method)
• L222 TMAEventMakerTestCase.setUpClass (method)
• L236 TMAEventMakerTestCase.tearDown (method)
• L242 TMAEventMakerTestCase.test_events (method)
• L248 TMAEventMakerTestCase.test_rowDataForValues (method)
• L265 TMAEventMakerTestCase.test_monotonicTimeInDataframe (method)
• L271 TMAEventMakerTestCase.test_monotonicTimeApplicationOfRows (method)
• L288 TMAEventMakerTestCase.test_fullDaySequence (method)
• L300 TMAEventMakerTestCase.test_endToEnd (method)
• L325 TMAEventMakerTestCase.test_noDataBehaviour (method)
• L337 TMAEventMakerTestCase.test_helperFunctions (method)
• L348 TMAEventMakerTestCase.test_filterBadValues (method)
• L445 TMAEventMakerTestCase.test_getEvent (method)
• L466 TMAEventMakerTestCase.test_printing (method)
• L490 TMAEventMakerTestCase.test_getAxisData (method)
• L509 TMAEventMakerTestCase.test_plottingAndCommands (method)
• L531 TMAEventMakerTestCase.test_findEvent (method)
• L582 TMAEventMakerTestCase.test_eventAssociatedWith (method)
• L633 setup_module (function)

tests/test_utils.py

• L66 ExpSkyPositionOffsetTestCase.setUp (method)
• L82 ExpSkyPositionOffsetTestCase.test_getExpPositionOffset (method)
• L146 MiscUtilsTestCase.test_getFieldNameAndTileNumber (method)
• L179 MiscUtilsTestCase.test_getAirmassSeeingCorrection (method)
• L191 MiscUtilsTestCase.test_getFilterSeeingCorrection (method)
• L198 MiscUtilsTestCase.test_getBandpassSeeingCorrection (method)
• L214 MiscUtilsTestCase.test_quickSmooth (method)
• L222 MiscUtilsTestCase.test_getCurrentDayObsDatetime (method)
• L233 MiscUtilsTestCase.test_getCurrentDayObsInt (method)
• L240 MiscUtilsTestCase.test_getCurrentDayObsHumanStr (method)
• L247 MiscUtilsTestCase.test_getSunAngle (method)
• L265 QuantileTestCase.test_quantiles (method)
• L286 ImageBasedTestCase.test_fluxFromFootprint (method)
• L340 IdTestCase.test_exposure_id (method)
• L347 IdTestCase.test_ccdexposure_id (method)
• L360 IdTestCase.test_calcEclipticCoords (method)
• L376 setup_module (function)

tests/utils.py

• L29 getVcr (function)

[codecov]

Codecov Report

❌ Patch coverage is 86.36364% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 22.74%. Comparing base (1485dcb) to head (db73762).

Files with missing lines	Patch %	Lines
python/lsst/summit/utils/quickLook.py	90.00%	2 Missing ⚠️
python/lsst/summit/utils/bestEffort.py	50.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #167      +/-   ##
==========================================
- Coverage   22.79%   22.74%   -0.06%     
==========================================
  Files          38       38              
  Lines        7054     7049       -5     
==========================================
- Hits         1608     1603       -5     
  Misses       5446     5446              

Files with missing lines	Coverage Δ
python/lsst/summit/utils/bestEffort.py	28.15% <50.00%> (ø)
python/lsst/summit/utils/quickLook.py	94.84% <90.00%> (-0.26%)	⬇️

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.