Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Issue a warning when generated QuantumGraph is empty (DM-19372) #18

Merged
merged 2 commits into from
Apr 19, 2019
Merged

Issue a warning when generated QuantumGraph is empty (DM-19372) #18

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
80698fc
Issue a warning when generated QuantumGraph is empty (DM-19372)
andy-slac Apr 18, 2019
fbd7d4d
Make pipetask to capture warnings and send them to logging
andy-slac Apr 19, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
14 changes: 13 additions & 1 deletion python/lsst/ctrl/mpexec/cmdLineFwk.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@
import pickle
import re
import sys
import warnings

# -----------------------------
# Imports for other modules --
Expand Down Expand Up @@ -166,6 +167,9 @@ def configLog(longlog, logLevels):
lgr.setLevel(logging.INFO) # same as in log4cxx config above
lgr.addHandler(lsst.log.LogHandler())

# also capture warnings and send them to logging
logging.captureWarnings(True)

# configure individual loggers
for component, level in logLevels:
level = getattr(lsst.log.Log, level.upper(), None)
Expand Down Expand Up @@ -364,6 +368,14 @@ def makeGraph(self, pipeline, taskFactory, args):
graphBuilder = GraphBuilder(taskFactory, butler.registry, args.skip_existing)
qgraph = graphBuilder.makeGraph(pipeline, coll, args.data_query)

# count quanta in graph and give a warning if it's empty
nQuanta = sum(1 for q in qgraph.quanta())
if nQuanta == 0:
warnings.warn("QuantumGraph is empty", stacklevel=2)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Works great. Does the output of warnings.warn also go through the logging module?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It can be (as you have found already) redirected to logging with logging.captureWarnings. We probably want to add this to pipetask as default settings, I think it's trivial change and can be done on this ticket too.

else:
_LOG.info("QuantumGraph contains %d quanta for %d tasks",
nQuanta, len(qgraph))

if args.save_qgraph:
with open(args.save_qgraph, "wb") as pickleFile:
pickle.dump(qgraph, pickleFile)
Expand Down Expand Up @@ -424,7 +436,7 @@ def showInfo(self, showOpts, pipeline, graph):

if showCommand in ["pipeline", "config", "history", "tasks"]:
if not pipeline:
_LOG.warn("Pipeline is required for --show=%s", showCommand)
_LOG.warning("Pipeline is required for --show=%s", showCommand)
continue

if showCommand == "pipeline":
Expand Down
4 changes: 3 additions & 1 deletion tests/test_cmdLineFwk.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -191,7 +191,9 @@ def testMakeGraphFromPickle(self):
with open(tmpname, "wb") as pickleFile:
pickle.dump(qgraph, pickleFile)
args = _makeArgs(qgraph=tmpname)
qgraph = fwk.makeGraph(None, taskFactory, args)
with self.assertWarnsRegex(UserWarning, "QuantumGraph is empty"):
# this also tests that warning is generated for empty graph
qgraph = fwk.makeGraph(None, taskFactory, args)
self.assertIsInstance(qgraph, QuantumGraph)
self.assertEqual(len(qgraph), 0)

Expand Down