Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a @local decorator #350

Open
romain-intel opened this issue Oct 14, 2020 · 1 comment
Open

Add a @local decorator #350

romain-intel opened this issue Oct 14, 2020 · 1 comment
Labels
enhancement New feature or request good first issue Good for newcomers

Comments

@romain-intel
Copy link
Contributor

Currently you can use @batch to decorate single steps that you want to force to run on batch even without using --with batch. However, you cannot do the opposite and force steps to run locally even when doing --with batch.

A possible use-case for this would be to annotate a GPU-heavy step that you want to run locally to save on AWS costs but take advantage of fast S3 access for other steps (from the EC2 machines batch uses).

It is currently possible to have this use case by annotating every other step with @batch but this is not very developer friendly (you can't easily run your code locally to test, it's tedious, etc).

Relevant discussion on gitter:
https://gitter.im/metaflow_org/community?at=5f85f41d24a20801a8e5373d

Other questions to address when touching decorators:
https://gitter.im/metaflow_org/community?at=5f7f81e024a20801a8d509a6
https://gitter.im/metaflow_org/community?at=5f05d3baa5ab931e4f7280e3

@romain-intel romain-intel added the enhancement New feature or request label Oct 14, 2020
@savingoyal savingoyal linked a pull request Feb 9, 2021 that will close this issue
6 tasks
@savingoyal savingoyal removed a link to a pull request Feb 9, 2021
6 tasks
@romain-intel romain-intel added the good first issue Good for newcomers label Jan 11, 2024
@romain-intel
Copy link
Contributor Author

This one should be simple and useful.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request good first issue Good for newcomers
Projects
None yet
Development

No branches or pull requests

1 participant