New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support job attr defaults in the cluster config #97

Open
ericfranz opened this Issue Aug 28, 2018 · 4 comments

Comments

Projects
None yet
1 participant
@ericfranz
Copy link
Contributor

ericfranz commented Aug 28, 2018

Usage:

  job:
    adapter: "torque"
    host: "owens-batch.ten.osc.edu"
    lib: "/opt/torque/lib64"
    bin: "/opt/torque/bin"
    version: "6.0.1"
    defaults:
      reservation_id: "next.1234567"

Of course the question comes, if you set a default queue, and does that queue need to be used when checking the status, and with other commands? Maybe the individual adapters should be aware of which arguments are appropriate, so if defaults are provided they are used.

@ericfranz

This comment has been minimized.

Copy link
Contributor Author

ericfranz commented Aug 28, 2018

This is a proposed implementation of the goal: to enable providing default values for all jobs submitted to a cluster. A better solution might be a "submit filter" that gives you the script object and lets you return a script object. Then instead of just defaults, we could do something more. For example, imagine that "slots" #96 support is added, and the simple request in torque for num slots for Owens might be "nodes=1:ppn=28" and for Owens VDI might be "nodes=1:ppn=1:owens". A submit filter could take "1" and produce "nodes=1:ppn=28" or "nodes=1:ppn=1:owens"; it could take "nodes=1:ppn=1" and append ":owens" if necessary.

Or, maybe the simple case is a template string that specifies default values. Liquid templates support this: https://shopify.github.io/liquid/filters/default/

@ericfranz

This comment has been minimized.

Copy link
Contributor Author

ericfranz commented Aug 28, 2018

A broader goal of this issue is to support the ability to move cluster specific config to the cluster configs, instead of spreading that knowledge across the templates.

@ericfranz

This comment has been minimized.

Copy link
Contributor Author

ericfranz commented Jan 25, 2019

See https://discourse.osc.edu/t/global-email-setting/123

We have at least one other site interested in this ability, and adding dynamic code to it. For example, if the cluster config files were rendered using erb prior to loading:

script:
  email: <%= ENV["USER"] %>@my.domain
@ericfranz

This comment has been minimized.

Copy link
Contributor Author

ericfranz commented Jan 30, 2019

If we make the cluster configs render-able via erb that would at least allow the goal of the admin in the Discourse topic: https://discourse.osc.edu/t/global-email-setting/123/4?u=efranz

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment