New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for containers #45

Closed
kylechard opened this Issue Dec 7, 2017 · 3 comments

Comments

Projects
None yet
3 participants
@kylechard
Collaborator

kylechard commented Dec 7, 2017

Support for running applications in containers. Make this a high level app type. Investigate support for reusing cached containers. Investigate support for containers as a sandbox.

Potentially we would want to support different container models for different systems. We'd want to define a single app that could then use different containers on each system.

Perhaps something like the following:

app('container', dfk):
def foo(param1, param2):
return {singularity : {container-uri: uri, config: foo, run: "param1 param2"},
docker: {container-uri: uri, config: foo, run: "param1 param2"}}

Maybe better to wrap each container type as its own class?

@yadudoc yadudoc added this to the Parsl-0.5.0 milestone Feb 2, 2018

@yadudoc

This comment has been minimized.

Contributor

yadudoc commented Mar 5, 2018

Pushing to 0.6.0

@yadudoc yadudoc modified the milestones: Parsl-0.5.0, Parsl-0.6.0 Mar 5, 2018

@yadudoc

This comment has been minimized.

Contributor

yadudoc commented Mar 27, 2018

As an intermediate step, we will add support for launching IPP engines within containers. By specifying specific sites that use a particular container, and tying apps to those site, you can execute apps within containers that support them.

@yadudoc yadudoc self-assigned this Mar 27, 2018

@yadudoc yadudoc added the enhancement label Mar 27, 2018

yadudoc added a commit that referenced this issue Mar 30, 2018

@yadudoc

This comment has been minimized.

Contributor

yadudoc commented Apr 27, 2018

0.5.0 added minimal container support. This was done in the IPP executor, by having the ipengine calls wrapped by container calls. This approach however works only when ddocker containers can be launched directly for eg. on a node with docker available. We need this this logic to be pushed down to libsubmit so that containers are launched in a way suitable to the compute resource, (say Kubernetes).

@annawoodard This is additional work that needs to happen in order to support containers properly.

@yadudoc yadudoc assigned annawoodard and unassigned yadudoc Apr 27, 2018

@yadudoc yadudoc closed this Jun 18, 2018

benclifford pushed a commit that referenced this issue Aug 9, 2018

annawoodard pushed a commit that referenced this issue Sep 24, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment