refactor: Isolate logic that finds next runnable component waiting for input #7880
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
silvanocerza
added
the
ignore-for-release-notes
github-actions
bot
added
topic:tests
topic:core
type:documentation
Pull Request Test Coverage Report for Build 9549247852Details
💛 - Coveralls |
Pull Request Test Coverage Report for Build 9549480203Details
💛 - Coveralls |
silvanocerza
force-pushed
the
refactor-pipeline-run
branch
from
71f7cb6
to
d957f6a
Compare
Pull Request Test Coverage Report for Build 9549815154Details
💛 - Coveralls |
shadeMe
requested changes
Jun 17, 2024
haystack/core/pipeline/base.py
Outdated
| for other_name, other_comp in waiting_for_input: | ||
| if name == other_name: | ||
| continue | ||
| there_are_only_lazy_variadics &= ( |
There was a problem hiding this comment.
We can rewrite this check to break early if a non(-lazy)-variadic component is encountered.
haystack/core/pipeline/base.py
Outdated
Comment on lines
910
to
911
| is_variadic = any(socket.is_variadic for socket in comp.__haystack_input__._sockets_dict.values()) # type: ignore | ||
| if is_variadic and not comp.__haystack_is_greedy__: # type: ignore[attr-defined] |
There was a problem hiding this comment.
Let's extract this into a nested helper function.
haystack/core/pipeline/base.py
Outdated
| is_variadic = any(socket.is_variadic for socket in comp.__haystack_input__._sockets_dict.values()) # type: ignore | ||
| if is_variadic and not comp.__haystack_is_greedy__: # type: ignore[attr-defined] | ||
| there_are_only_lazy_variadics = True | ||
| for other_name, other_comp in waiting_for_input: |
There was a problem hiding this comment.
From the looks of this, we only need to perform this check once. Let's move it outside of the outer loop.
haystack/core/pipeline/base.py
Outdated
| ) | ||
| if has_only_defaults: | ||
| there_are_only_components_with_defaults = True | ||
| for other_name, other_comp in waiting_for_input: |
There was a problem hiding this comment.
Same as above - Doesn't look like it needs to be recalculated for every component.
haystack/core/pipeline/base.py
Outdated
Comment on lines
942
to
945
| there_are_only_components_with_defaults &= all( | ||
| not s.is_mandatory | ||
| for s in other_comp.__haystack_input__._sockets_dict.values() # type: ignore | ||
| ) |
There was a problem hiding this comment.
Same as above - We can break early.
haystack/core/pipeline/base.py
Outdated
Comment on lines
952
to
959
| if input_socket.is_mandatory and input_socket.name not in inputs_by_component[name]: | ||
| has_enough_inputs = False | ||
| break | ||
| if input_socket.is_mandatory: | ||
| continue | ||
|
|
||
| if input_socket.name not in inputs_by_component[name]: | ||
| inputs_by_component[name][input_socket.name] = input_socket.default_value |
There was a problem hiding this comment.
I'd rewrite this to just use a single if input_socket.name not in inputs_by_component[name]: - break if it's mandatory, copy default value otherwise.
|
I updated the comment. |
Pull Request Test Coverage Report for Build 9562857219Warning: This coverage report may be inaccurate.This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.
Details
💛 - Coveralls |
|
After some discussion with @shadeMe we decided to change the refactoring approach a bit. |
Pull Request Test Coverage Report for Build 9565856215Warning: This coverage report may be inaccurate.This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.
Details
💛 - Coveralls |
shadeMe
approved these changes
Jun 18, 2024
Pull Request Test Coverage Report for Build 9566408308Warning: This coverage report may be inaccurate.This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.
Details
💛 - Coveralls |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Related Issues
Part of #7614.
Proposed Changes:
This PR isolates the logic that moves Components from
waiting_for_inputtoto_runqueue whento_runis empty andwaiting_for_inputis not.How did you test it?
I added unit tests and ran tests locally.
Notes for the reviewer
There's some formatting changes too as we changed the default formatted from
blacktoruff.Checklist
fix:,feat:,build:,chore:,ci:,docs:,style:,refactor:,perf:,test:.