Jupyter parallelization #1803
Merged
Jupyter parallelization #1803
+634
−78
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Works here if I run a notebook then manually submit the config.json that makes, fails to write temp files from jupyter side.
joshqsumner
added
new feature
|
Here's the code health analysis summary for commits Analysis Summary
Code Coverage Report
|
joshqsumner
added
ready to review
and removed
work in progress
also correct WorkflowInputs vs workflow_inputs
HaleySchuhl
approved these changes
Nov 21, 2025
Contributor
HaleySchuhl
left a comment
HaleySchuhl
left a comment
There was a problem hiding this comment.
Works great!
This can go in a future PR but I had minor feedback on the docs. I like the parallel_preparation.md addition. I think Why Parallelize can be generalized from what content is there now. Parallel workflows are more reproducible than what can be done with a GUI, and can we maybe add more detail about how consistent a dataset needs to be to analyze in parallel (vs when human-in-the-loop workflows are required).
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Describe your changes
Adding a
jupyterconfigclass to run parallel workflows from jupyter directly.So far it just uses the existing parallel tools and the new stuff just lets you automatically convert the notebook to a script and run that script over a dataset.
Type of update
This is a new feature, relies on #1798
Associated issues
Closes #1802
Additional context
There could be a lot of ways to implement this and I'm not sure which is best so I started with the simplest. We can always scrap it.
Depending on how this looks exactly it might be hard to test some things, the current methods mostly check if you're in a notebook before doing most of the function.
For the reviewer
See this page for instructions on how to review the pull request.
plantcv/mkdocs.ymlupdating.md