Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

split local scheduler into different ones per image type #185

Closed
d4l3k opened this issue Sep 16, 2021 · 0 comments
Closed

split local scheduler into different ones per image type #185

d4l3k opened this issue Sep 16, 2021 · 0 comments
Labels
enhancement New feature or request module: runner issues related to the torchx.runner and torchx.scheduler modules

Comments

@d4l3k
Copy link
Contributor

d4l3k commented Sep 16, 2021

Currently the local scheduler has two supported paths depending on image type, docker and process. We should split these into two separate schedulers so each scheduler is tied to a single image type to reduce complexity and make it easier to work out of the box.

Proposed schedulers:

  • local_docker
  • local_cwd - overrides image with the current working directory for local testing purposes

In combination with #184 this should make all builtin components work as expected out of the box with remote docker and local docker/path based schedulers. For slurm you'll still have to manually override the image for now.
@d4l3k d4l3k added enhancement New feature or request module: runner issues related to the torchx.runner and torchx.scheduler modules labels Sep 16, 2021
@d4l3k d4l3k mentioned this issue Sep 17, 2021
Closed
facebook-github-bot pushed a commit that referenced this issue Sep 21, 2021
@d4l3k @facebook-github-bot
schedulers/local_scheduler: expose docker as local_docker instead of …
2b26952 
…using image type (#186)

Summary:
This exposes running docker images via `local_docker` scheduler type ``--scheduler local_docker` instead of via `--scheduler local --scheduler_args image_type=docker`. This makes it easier to discover and use.

This is the first step in #185.

This is a breaking API change but not many users are using the local docker scheduler right now and it's easy to change.

Pull Request resolved: #186

Test Plan:
```
$ pytest -n 8
$ pyre
$ torchx runopts
```

Reviewed By: aivanou

Differential Revision: D31008657

Pulled By: d4l3k

fbshipit-source-id: 289ff5542c73956516b80b8325813d383dd44fda
@d4l3k d4l3k closed this as completed Oct 1, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request module: runner issues related to the torchx.runner and torchx.scheduler modules
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@d4l3k