Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multisite support #48

Closed
4 tasks done
yadudoc opened this issue Dec 8, 2017 · 1 comment
Closed
4 tasks done

Multisite support #48

yadudoc opened this issue Dec 8, 2017 · 1 comment
Assignees
Milestone

Comments

@yadudoc
Copy link
Member

yadudoc commented Dec 8, 2017

This is to track ongoing work on the multisite branch.

Why is this important:

  1. This will allow us to manage apps with different resource requirements in a single workflow, such as apps that only need a core and should be executed on the login node vs, apps that run on multiple
    nodes with MPI.
  2. Light weight apps executing on threads could perform flowcontrol and launch heavy tasks.
  • Accept multiple executors in the dfk
  • Launch multiple executors in the DFK from the config definition
  • Support for multiple IPP executors
  • Attach specific tasks to sites via enhancements to the decorator to take sites kwarg :
@App ('bash', dfk, sites=['local'])
def foo (x):
    return 'echo $(({0} * 2))

@App ('bash', dfk, sites='all' ) # Sites = 'all is the default
def sleep_compute (x) :
    return 'sleep x'
@yadudoc yadudoc added this to the Parsl-0.4.0 milestone Dec 8, 2017
@yadudoc yadudoc self-assigned this Dec 8, 2017
@yadudoc
Copy link
Member Author

yadudoc commented Dec 12, 2017

Here's a test from the test_suite that shows how all of this would work :
https://github.com/Parsl/parsl/blob/multisite/parsl/tests/test_scaling/local_multisite/test_ipp_None_local.py

@yadudoc yadudoc mentioned this issue Dec 19, 2017
yadudoc added a commit that referenced this issue Dec 22, 2017
demonstrate key functionality for #48, #58 and potentially fix #39
@yadudoc yadudoc closed this as completed Dec 23, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant