DM-34811 Fix component linkages in pipeline dot render #185
Conversation
natelust
force-pushed
the
tickets/DM-34811
branch
from
1b0ae79
to
462b27f
Compare
Creating a dot graph was previously not correctly linking component datasets to the composite that provided them when drawing a graph, this ensures the linkages are added appropriately.
natelust
force-pushed
the
tickets/DM-34811
branch
from
462b27f
to
8f327a2
Compare
Codecov Report
@@ Coverage Diff @@
## main #185 +/- ##
==========================================
- Coverage 82.67% 82.57% -0.10%
==========================================
Files 47 47
Lines 3653 3673 +20
Branches 591 596 +5
==========================================
+ Hits 3020 3033 +13
- Misses 465 470 +5
- Partials 168 170 +2
Continue to review full report at Codecov.
|
| # connect component dataset types to the composite type that | ||
| # produced it | ||
| if component is not None and (nodeName, attr.name) not in allDatasets: | ||
| _renderEdge(nodeName, attr.name, file) |
There was a problem hiding this comment.
Should this also need:
if nodeName not in allDatasets:
_renderDSTypeNode(nodeName, ...)
(in case a component is a pre-existing input and not an intermediate)?
It may be a nicer representation if a component could be rendered as a box inside a dataset type node (e.g. using record-based graphviz nodes), but that is probably out of scope for this ticket.
There was a problem hiding this comment.
I am not sure what you mean with your first point, the rendering of the DSType node is handled above, and prerequisites are handled in a separate loop.
I thought about your latter point, but decided it was not worth re-working so much for a quicker fix that seems to do its job well enough.
There was a problem hiding this comment.
I think the code above renders a node for attr.name, but here you add an edge from nodeName to attr.name. If there is no node rendered for nodeName it will be shown using default node representation without any dataset-related info (simple box with a name inside).
There was a problem hiding this comment.
ah yeah, I see what you are saying, good point, ill fix that up
Create connections to corresponding task which produces a metadata dataset if it is specified as an input connection
Sort the contents that are used to generate the dot file such that the output comes out deterministic.
python/lsst/ctrl/mpexec/dotTools.py
Outdated
| _renderDSTypeNode(nodeName, dimensions, file) | ||
| # The next if block is a workaround until DM-29658 at which time | ||
| # metadata connections should star working with the above code | ||
| if (match := re.match("^(.*)_metadata$", attr.name)) is not None: |
There was a problem hiding this comment.
Maybe compile the regex to save few CPU cycles?
Fix typos in doc-strings and optimize regular expression use.
Creating a dot graph was previously not correctly linking component
datasets to the composite that provided them when drawing a graph,
this ensures the linkages are added appropriately.
Checklist
doc/changes