Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DM-25010: Make Alert serialization optional in diaPipe. #82

Merged
merged 2 commits into from Jun 2, 2020

Conversation

morriscb
Copy link
Contributor

No description provided.

@@ -122,6 +122,11 @@ class DiaPipelineConfig(pipeBase.PipelineTaskConfig,
target=PackageAlertsTask,
doc="Subtask for packaging Ap data into alerts.",
)
doSerializeAlerts = pexConfig.Field(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is doSerializeAlerts the write term here?

I mean, it's correct, but it seems to add an unnecessary third term beyond “package” (on the basis that it's PackageAlertsTask, rather than SerializeAlertsTask that we're referring to) and “write” (on the basis that it's really writing out the alerts that the user cares about)

It's not a big deal, it just feels like we're using three terms which all kind-of-but-not-quite mean the same thing, where one (or maybe two) would do fine.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed everything to PackageAlerts.

doSerializeAlerts = pexConfig.Field(
dtype=bool,
default=False,
doc="Serialize alerts and write them.",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe be explicit that they are being written to the filesystem (for now, at least)?

@@ -36,7 +36,7 @@
class TestDiaPipelineTask(unittest.TestCase):

@classmethod
def _makeDefaultConfig(cls):
def _makeDefaultConfig(cls, doAlerts=False):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would you consider using doSerializeAlerts here? (Or whatever you change that to, if you think my comment on naming is helpful). Two reasons: one for consistency, and the other because doAlerts is not very descriptive (“do alerts what?”).

if doAlerts:
subtasks.alertPackager.run.assert_called_once()
else:
subtasks.alertPackager.run.assert_not_called()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I am reading this correctly, that assert_not_called() is meaningless — what's happening here is an AttributeError is raised as soon as an attempt is made to access subtasks.alertPackager, because that substask was never created. Is that right?

If so, could you please change this logic to make this logic more straightforward? I'd suggest that avoiding the exception altogether — something like

if doAlerts:
    ...
else:
    self.assertFalse(hasattr(subtasks, 'alertPackager'))

— but if you prefer the exception you please take out the attribute accesses that will never fire and make clear in a comment what's happening:

def testRunWithoutAlerts(self):
    # When alert generation is disabled, `_testRun` is expected to raise
    # an AttributeError due to the lack of an `alertPackager` subtask.
    with self.assertRaises(AttributeError):
        self._testRun(False)

...

if doAlerts:
    ...
else:
    subtasks.alertPackager  # should raise AttributeError

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Picked the first option

@@ -101,11 +101,23 @@ def tearDown(self):
def testRunQuantum(self):
pass

def testRun(self):
def testRunWithAlerts(self):
"""Test running with touching alerts.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe “with serializing/packaging/writing/whatever alerts” rather than “touching”? Just to make it a bit clear what's actually happening.

self._testRun(True)

def testRunWithoutAlerts(self):
"""Test running without touching alerts.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As above, re “touching”.

@morriscb morriscb force-pushed the tickets/DM-25010 branch 3 times, most recently from 0458be0 to 0f0b29b Compare May 29, 2020 18:43
@jdswinbank
Copy link
Contributor

Hey Chris — thanks for these changes.

Looking at the changes you made to TestDiaPipelineTask, I realised that I hadn't previously understood what it was doing. Unfortunately, that means that the change I suggested (specifically, the self.assertFalse(hasattr(subtasks, 'alertPackager'))) is not very useful — it's only demonstrating that the test code works, not that the task itself is doing the right thing.

I spent a few minutes playing about with the code and came up with these changes: db2c0c2. I think they make the test a bit simpler and more useful than it is now. What do you think?

@morriscb
Copy link
Contributor Author

morriscb commented Jun 1, 2020

Sure, that looks good to me. Since it's on your branch, do you mind appending it to the end of this ticket branch? You're much better with git after all :D

@jdswinbank
Copy link
Contributor

Appended!

morriscb and others added 2 commits June 2, 2020 11:54
Add doSerializeAlerts=True to test config.

Create no alert call path in tests.

Debug tests.

Change configs to doPackageAlerts.

Change to doPackageAlerts in unittests.

Clarify test doc strings.

Create better test for not creating a PackageAlerts task.

Debug unittests.

Fix linting
@morriscb morriscb merged commit 76ece0e into master Jun 2, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants