Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DM-24584: create an ingestRaws butler command #241

Merged
merged 13 commits into from May 13, 2020
Merged

Conversation

n8pease
Copy link
Contributor

@n8pease n8pease commented May 12, 2020

No description provided.

@n8pease n8pease merged commit 4386bd2 into master May 13, 2020
Copy link
Member

@timj timj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some more philosophical comments.

"""
@staticmethod
@abc.abstractmethod
def instrumentClass():
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These do indeed force a test implementation to specify the value but in most cases the tests will fail if they do it wrong. It's a shame that a subclass can't implement by doing a simple class property but the use of static method here prevents that.

Also instrumentClass as a name makes you think this is the class but it's not, it's the class name.


@staticmethod
@abc.abstractmethod
def instrumentName():
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given that the Instrument class itself defines the name, wouldn't it be easier for everyone if a subclass could specify an Instrument class directly (not a string, although a string could be doImported) and the name was retrieve from that directly? It would prevent an obvious case of duplication of information in the tests where the test subclass now needs to specify the name and the class name and ensure that they all match with the name that they already know is defined in the Instrument class.

"""The fully qualified name of the instrument.
"""

instrumentName = ""
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above, don't ask people to tell you things you can work out for yourself from the fully qualified name of the instrument.

def setUp(self):
# Use a temporary working directory
self.root = tempfile.mkdtemp(dir=self.ingestDir)
Butler.makeRepo(self.root)
self.butler = Butler(self.root, run="raw")
createRepo(self.root)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You probably should add a comment here to say that you are deliberately using the createRepo function here for testing purposes. Maybe we can reassess but I don't think we want to encourage general users to be using the script wrappers in their code.

files = [file] if file is not None else []
if directory is not None:
files.extend(
[os.path.join(directory, f) for f in os.listdir(directory) if f.lower().endswith("fits")])
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In future we might have to make this extension configurable (or allow fits.fz and fits.gz and the like by default). We can punt on HDF5 for now.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this came directly from the script we are replacing. I'm happy to work on adding configurability when we're ready, I think punting for now is the right choice, let's make a ticket for it when we're ready to define what we want.

@timj timj deleted the tickets/DM-24584 branch June 25, 2020 22:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants