Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

inconsistency in Pipeline.get_linked_items() and dispatch_plugs() #286

Open
denisri opened this issue Aug 2, 2023 · 1 comment
Open

Comments

@denisri
Copy link
Collaborator

denisri commented Aug 2, 2023

Pipeline.get_linked_items() does not behave the same way when used on a regular (leaf) process or on a sub-pipeline inside the main pipeline. direction is not handled the same way:

                    if isinstance(node, Pipeline):
                        if in_outer_pipelines:
                            directions = ('links_from', 'links_to')
                        elif plug.output:
                            directions = ('links_from',)
                        else:
                            directions = ('links_to',)
                    elif plug.output:
                        directions = ('links_to',)
                    else:
                        directions = ('links_from',)

a terminal Process will thus yield its external links, while a sub-pipeline will provide only its internal links.
So other nodes plugs connected to a sub-pipeline will not be reached from it. As a consequence, dispatch_plugs() and dispatch_value() miss some connections and actually do not dispatch to all nodes as it should.
We could use in_outer_pipelines=True, but this will also propagate to other connected sub-pipeline, which is not always what we want.
There is at least a "meaning" ambiguity on this method, it behaves differently on terminal processed and pipelines, differently depending on direction, in_sub_pipelines and in_outer_pipelines which makes its behaviour quite difficult to apprehend. And the doc currently says nothing about all this, so we don't really know what is the expected behaviour in each case.

@denisri
Copy link
Collaborator Author

denisri commented Aug 2, 2023

We should maybe define clear use cases for this function:
a. get processes/plugs connected to an outside plug of the main pipeline
b. get processes/plugs inside a sub-pipeline connected to a sub-pipeline plug (maybe this is the same as case a. by using sub_pipeline.get_linked_items()) ?)
c. get processes/plugs connected to a node plug in a pipeline (whatever its type), not going inside other sub-pipelines (sub-pipelines are treated as regular black-box processes here)
d. get processes/plugs connected to a node plug in a pipeline (whatever its type), going inside other sub-pipelines

(do I forget other useful use cases ?)
and provide the correct set of parameters go get the expected behaviour.

denisri added a commit that referenced this issue Aug 3, 2023
liked parameters with different values [fixed]
jobs dependencies [partly fixed, still not OK]
(#286)
@denisri denisri mentioned this issue Aug 4, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant