Dm-31054 Fix issue running Quantum when inputs are missing #131
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The recent NoWorkFound changes were intended to let a Quantum
continue to run if some of its inputs were missing as long as
enough inputs were present. However, there was a bug related
to wrong dataset types and use of defaultdict that caused the
system to always think NO inputs were present if any were
missing. This fixes that.
For those interested, adjustQuantum on a connections class indexes
things by dataset type name, a string. Most other paces use
DatasetType object in mappings. This is not a problem with
NameDict containers because they know how to translate. However in
this case a defaultdict was used to construct the mapping, and the
keys were all DatasetTypes.
Then this was passed through to update the Quantum and adjust it.
Because the container type was still defaultdict, when the updater
indexed with dataset type names (str) it created a new entry in the
defaultdict that was empty, instead of finding the entry associated
with the DatasetType that had the corresponding name.
This only triggered if there was at least one missing input, so in
normal operations everything worked as expected. A subsequent
container was then passed into adjustQuantum that had no datasetRefs
(because the defaultdict created a new empty entry) and the method
rightly said 0 is less than 1 required ref and raised a NoWorkFound.
Solution was to change to the proper NamedDict object before calling
into the update code in pipe_base.
Checklist
doc/changes