Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DM-24923: eliminate redundant class name & name requirement in butler command test cases #282

Merged
merged 2 commits into from May 17, 2020

Conversation

n8pease
Copy link
Contributor

@n8pease n8pease commented May 15, 2020

No description provided.

Copy link
Member

@timj timj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor comments. We are instantiating more Butler instances than before but I hope that no longer results in terrible overhead. I know that @kfindeisen had terrible performance on his computer with Butler taking seconds to be created so be aware this might not be a panacea.


instrumentClassName = "lsst.obs.subaru.HyperSuprimeCam"

instrumentName = "HSC"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this can go now can't it?

self.butler = Butler(self.root, run=self.outputRun)
self.visits = {
ingestDir = os.path.dirname(__file__)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure I'm in favor of all the blank lines between properties.


instrumentName = "HSC"

file = os.path.join(testDataDirectory, "hsc", "raw", "HSCA90402512.fits.gz")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This .gz reminds me that I think we want to change the default for ingest to also include .fits.gz and .fits.fz.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@timj did you want me to do something about this in this ticket? (FWIW if it involves more than a line or 2 of code I'd rather ticket it)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It was one line in python/lsst/obs/base/script/ingestRaws.py where we check file extension and could include the .gz/.fz there directly -- the ingestion clearly supports .fits.gz because the above file is a .fits.gz. What I'm now wondering is why we are specifying a file here at all when it seems like we could specify the directory -- maybe this is deliberate to ensure that we know exactly which dataIds we will end up with in the butler.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should take files as any number of extra positional arguments and let the shell do the globbing for us.

@@ -69,7 +73,8 @@ def checkRepo(self, files=None):
# We ignore `files` because there's only one raw file in
# testdata_subaru, and we know it's a science frame.
# If we ever add more, this test will need to change.
expanded = self.butler.registry.expandDataId(self.dataIds[0])
butler = Butler(self.root, run=self.outputRun)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this should be a readonly butler as above.


@property
def visits(self):
butler = Butler(self.root, run=self.outputRun)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this should be:

butler = Butler(self.root, collections=[self.outputRun])

because you want it to be read only and you aren't writing any data into it.

@n8pease n8pease merged commit e9bf0c5 into master May 17, 2020
@timj timj deleted the tickets/DM-24923 branch February 25, 2021 17:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants