Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

DM-20994: Write how to make gen3 obs package guide #181

Merged
merged 7 commits into from Oct 30, 2020
Merged

DM-20994: Write how to make gen3 obs package guide #181

merged 7 commits into from Oct 30, 2020

Conversation

parejkoj
Copy link
Collaborator

No description provided.

@parejkoj parejkoj requested a review from timj November 13, 2019 00:22
timj
timj approved these changes Nov 13, 2019
View reviewed changes
Copy link
Member

@timj timj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks like a reasonable start. Is the intent to also include examples of how to write the instrument and rawformatter classes themselves?

doc/lsst.obs.base/creating-an-obs-package.rst
####################################

These instructions describe how to create a package that allows the LSST Science Pipelines to read raw data from a telescope and instrument using the gen3 middleware and ingest it into an LSST repository.
"Ingestion" is the process of reading raw files, interpreting their metadata, and formatting the data in a way the :py:mod:`Data Butler <lsst.daf.butler>` can read.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure what "formatting the data" means in this context. Ingestion always means that the file is added to the repository without being changed in any way. We associate it with a Formatter class to tell butler how to read the file but this sentence implies we might change the data on ingest.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point: I was never quite clear on what exactly happens with the Formatter during Ingest. Could you please offer an alternate wording?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"...interpreting their metadata, recording how to read the data, and copying it into the butler datastore" ?

doc/lsst.obs.base/creating-an-obs-package.rst Outdated Show resolved Hide resolved
doc/lsst.obs.base/creating-an-obs-package.rst Outdated Show resolved Hide resolved
doc/lsst.obs.base/creating-an-obs-package.rst Outdated

the tests should fail, as there is no Example `Instrument` yet.

Next, add a file in ``python/lsst/obs/example/instrument.py`` containing a subclass of `Instrument`, `ExampleCam`, which at minimum overrides these abstract methods: `Instrument.getName`, `Instrument.getCamera`, `Instrument.register`, `Instrument.filterDefinitions`, and define ``self.configPaths`` in ``__init__``.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the ExampleCam here the name of the Instrument subclass? I'd expect that to be ExampleCamInstrument (and double backticks).

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For HSC and DECam, we named them "HyperSuprimeCam" and "DarkEnergyCamera". I see you did LsstCamInstrument for LSST. Should we change the former two?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I would understand it better if the parenthetical said ", named ExampleCam,"

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want to go with ExampleCamInstrument here and update HSC/DECam? Now's our chance to enshrine the "correct" way for the future. I don't have a strong opinion one way or the other.

@parejkoj
Copy link
Collaborator Author

Is the intent to also include examples of how to write the instrument and rawformatter classes themselves?

I do have the ExampleFormatter class in the Formatter section: it is very short, as it doesn't need to have much in it. I did not yet put in the ExampleInstrument class because that one is still in flux as I finish up DECam.

@timj timj merged commit 8ad2bf2 into master Oct 30, 2020
@timj timj deleted the tickets/DM-20994 branch October 30, 2020 16:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@timj timj timj approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
2 participants
@parejkoj @timj