You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently you can use @batch to decorate single steps that you want to force to run on batch even without using --with batch. However, you cannot do the opposite and force steps to run locally even when doing --with batch.
A possible use-case for this would be to annotate a GPU-heavy step that you want to run locally to save on AWS costs but take advantage of fast S3 access for other steps (from the EC2 machines batch uses).
It is currently possible to have this use case by annotating every other step with @batch but this is not very developer friendly (you can't easily run your code locally to test, it's tedious, etc).
Currently you can use
@batch
to decorate single steps that you want to force to run on batch even without using--with batch
. However, you cannot do the opposite and force steps to run locally even when doing--with batch
.A possible use-case for this would be to annotate a GPU-heavy step that you want to run locally to save on AWS costs but take advantage of fast S3 access for other steps (from the EC2 machines batch uses).
It is currently possible to have this use case by annotating every other step with
@batch
but this is not very developer friendly (you can't easily run your code locally to test, it's tedious, etc).Relevant discussion on gitter:
https://gitter.im/metaflow_org/community?at=5f85f41d24a20801a8e5373d
Other questions to address when touching decorators:
https://gitter.im/metaflow_org/community?at=5f7f81e024a20801a8d509a6
https://gitter.im/metaflow_org/community?at=5f05d3baa5ab931e4f7280e3
The text was updated successfully, but these errors were encountered: