Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ParallelRunner doesn't work yet #4

Open
datajoely opened this issue Mar 7, 2022 · 1 comment
Open

ParallelRunner doesn't work yet #4

datajoely opened this issue Mar 7, 2022 · 1 comment
Labels
help wanted Extra attention is needed

Comments

@datajoely
Copy link
Owner

Description

Due to the way the ParallelRunner works in Kedro via multi-processing the existing hook implementation breaks when interacting with whatever UI thread rich needs to work.

Context

Currently if the user specifies parallel runner the progress bar logic is disabled.

Potential solutions to evaluate:

  1. Work out a way of making the existing implementation using Locks (preferred)
  2. Subclass the ParallelRunner implementation and do it there (not ideal)
@datajoely datajoely added the help wanted Extra attention is needed label Mar 7, 2022
@datajoely
Copy link
Owner Author

I can't get the progress bars to work with ParallelRunner, but the nicer logging experience still works. Ultimately without developing my own runner implementation, I think the the interference of the threading rich is using behind the scenes and the multiprocessing Kedro is doing just won't work nicely together.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

1 participant