-
Notifications
You must be signed in to change notification settings - Fork 3
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Container orchestration refactor #257
Labels
in progress
Someone is working on this issue (please also assign yourself)
priority
Should be resolved first, if possible
python
Pull requests that update Python code
Comments
wpbonelli
added a commit
that referenced
this issue
Jan 16, 2022
wpbonelli
added a commit
that referenced
this issue
Jan 18, 2022
wpbonelli
added a commit
that referenced
this issue
Jan 19, 2022
This is a good opportunity to introduce task composition: check input/output types of each workflow and, if compatible, allow chaining. We should also allow task groups to be scheduled in parallel. |
wpbonelli
added a commit
that referenced
this issue
Feb 16, 2022
wpbonelli
added a commit
that referenced
this issue
Feb 16, 2022
wpbonelli
added a commit
that referenced
this issue
Feb 17, 2022
wpbonelli
added a commit
that referenced
this issue
Feb 17, 2022
wpbonelli
added a commit
that referenced
this issue
Feb 18, 2022
wpbonelli
added a commit
that referenced
this issue
Feb 19, 2022
wpbonelli
added a commit
that referenced
this issue
Feb 20, 2022
wpbonelli
added a commit
that referenced
this issue
Feb 20, 2022
Tentatively finished with 36ffc8d |
wpbonelli
added a commit
that referenced
this issue
Feb 21, 2022
wpbonelli
added a commit
that referenced
this issue
Feb 21, 2022
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
in progress
Someone is working on this issue (please also assign yourself)
priority
Should be resolved first, if possible
python
Pull requests that update Python code
There's little justification to roll our own system on top of Dask to interface with deployment targets. There's no lack of options in this space, and Dask is better suited to interactive use cases anyway, e.g., in Jupyter notebooks. (It would be nice to provide preconfigured notebooks as tutorials alongside workflows, with examples for how to use the code and parallelize it with Dask, but that's a separate concern.)
Snakemake or NextFlow do everything we need and more:
The
plantit-cli
should just expose a handy interface to theplantit
resource APIs. (Maybe it's worth spinning off the Terrain interface as its own project -- it might be nice to have a CLI to Terrain, although it should probably disallow data transfers beyond a certain size limit to avoid overloading CyVerse web APIs)The text was updated successfully, but these errors were encountered: