Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DM-20763: Add initial support for Gen3 Butler to obs_decam #164

Merged
merged 3 commits into from Aug 27, 2019

Conversation

parejkoj
Copy link
Collaborator

No description provided.


First write a test case, derived from `~lsst.daf.butler.instrument_tests.InstrumentTests` and `lsst.utils.tests.TestCase`, defining ``self.data`` and ``self.instrument`` in ``setUp``. Run this test via ``pytest -sv tests/test_newObs.py``: all of the tests should fail.

Next, at minimum, override these abstract methods: `~lsst.daf.butler.Instrument.getName`, `~lsst.daf.butler.Instrument.getCamera`.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure we want to make getCamera part of the Instrument base class API, as it implies that there is a nominal camera that can be retrieved without providing a calibration repository. That's true right now for all cameras, of course, because it's all Gen2 supports. But I'm not sure it will be true forever.

That said, I'm not sure it won't be true forever, and maybe refactoring all of the cameras if/when we make that change isn't a big deal. But that's the reason why getCamera was only defined on HyperSuprimeCam, not Instrument, before.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup, I knew that context going in. The problem is that we have to actually choose whether to make getCamera part of the Instrument API or not at all. Right now, we need it in order to have FitsRawFormatter.getDetector() work (which we need for the new SkyWcs code), so I've made it a proper part of the API, so that getDetector works in the other Cameras.

Do you have a different approach you would recommend to deal with this until versioned calibration repos become available?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do wonder if we have some guidance as to what getName should return. ObservationInfo defines an instrument name and a telescope name. How should this name relate to the translator name?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good question: in the tests I've written on this ticket, I'm assuming that getName()==camera.getName(), which should probably also equal observationInfo.instrument_name.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I'd like to be pragmatic rather than prescriptive here: I think it's highly desirable for unambiguous, short instrument-only names to be acceptable for Instrument.getName() (e.g. "HSC") even if in other contexts we need longer qualified names (e.g. "LSST-LATISS"). We may well need to add something like ObservationInfo.full_name to match this, but I don't recall if we currently have anything in e.g. RawIngestTask that requires equivalence between ObservationInfo's names and Instrument's name (but I bet @parejkoj knows now!)

On the getCamera question, let's just live with having it in getInstrument now.

python/lsst/obs/base/filters.py Outdated Show resolved Hide resolved
python/lsst/obs/base/fitsRawFormatterBase.py Outdated Show resolved Hide resolved
python/lsst/obs/base/fitsRawFormatterBase.py Outdated Show resolved Hide resolved
Copy link
Member

@TallJimbo TallJimbo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Again, only minor comments.

doc/lsst.obs.base/creating-an-obs-package.rst Outdated Show resolved Hide resolved
doc/lsst.obs.base/creating-an-obs-package.rst Outdated Show resolved Hide resolved

First write a test case, derived from `~lsst.daf.butler.instrument_tests.InstrumentTests` and `lsst.utils.tests.TestCase`, defining ``self.data`` and ``self.instrument`` in ``setUp``. Run this test via ``pytest -sv tests/test_newObs.py``: all of the tests should fail.

Next, at minimum, override these abstract methods: `~lsst.daf.butler.Instrument.getName`, `~lsst.daf.butler.Instrument.getCamera`.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I'd like to be pragmatic rather than prescriptive here: I think it's highly desirable for unambiguous, short instrument-only names to be acceptable for Instrument.getName() (e.g. "HSC") even if in other contexts we need longer qualified names (e.g. "LSST-LATISS"). We may well need to add something like ObservationInfo.full_name to match this, but I don't recall if we currently have anything in e.g. RawIngestTask that requires equivalence between ObservationInfo's names and Instrument's name (but I bet @parejkoj knows now!)

On the getCamera question, let's just live with having it in getInstrument now.

python/lsst/obs/base/filters.py Outdated Show resolved Hide resolved
python/lsst/obs/base/filters.py Outdated Show resolved Hide resolved
python/lsst/obs/base/filters.py Outdated Show resolved Hide resolved
python/lsst/obs/base/filters.py Outdated Show resolved Hide resolved
python/lsst/obs/base/filters.py Outdated Show resolved Hide resolved
python/lsst/obs/base/gen3/ingest_tests.py Outdated Show resolved Hide resolved
Copy link
Member

@TallJimbo TallJimbo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Again, only minor comments. For some reason I think the PR I'm looking at isn't showing some of @timj's comments, so I may have duplicated some of his and/or failed to reply. I'll go back and look again after I hit the submit button.

@parejkoj parejkoj force-pushed the tickets/DM-20763 branch 5 times, most recently from b08a603 to b5bf6a4 Compare August 20, 2019 22:50
@parejkoj
Copy link
Collaborator Author

@timj @TallJimbo : can you please take another look at filters.py and the new test_filters.py please? The first commit on this PR should have all the interesting bits.

Copy link
Member

@TallJimbo TallJimbo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand why the define-exactly-once guard is as complex as it is, but it's quite possible the alternative I posted has a flaw I'm just not seeing. Otherwise looks fine.

python/lsst/obs/base/filters.py Outdated Show resolved Hide resolved
python/lsst/obs/base/filters.py Outdated Show resolved Hide resolved
Filters are immutable, and thus the dataclass is frozen.
These definitions can be used by both gen2 and gen3, and should help manage
that transition.
Since we define the filters in init, we can then use always use them in
`makeFilter`.

Add filterDefinitions property to formatter test
These tests were moved from obs_subaru/tests/test_ingest.py so that other obs
packages can use them.
@parejkoj parejkoj merged commit 2e73c10 into master Aug 27, 2019
@timj timj deleted the tickets/DM-20763 branch August 17, 2020 23:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants