Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add the ability to control job conf plugin loading from environment variables. #4154

Merged
merged 1 commit into from Jun 6, 2017

Conversation

Projects
None yet
2 participants
@jmchilton
Copy link
Member

commented Jun 6, 2017

All job configuration parameters can be set using environment variables but currently the actual loading of the plugins and destinations cannot. This is problematic for setting up single job confs for multiple environments (like done with docker-galaxy-stable) when using plugins that may hang waiting for resources (such as Kubernetes for the K8 runner or a MQ for Pulsar for instance) or may require library dependencies not be setup in every environment the job configuration is deployed in.

xref bgruening/docker-galaxy-stable#328

@jmchilton jmchilton added this to the 17.09 milestone Jun 6, 2017

@jmchilton jmchilton force-pushed the jmchilton:dynamic_disable_runners branch 2 times, most recently from 7ad5f90 to 48982f1 Jun 6, 2017


<!-- Additionally any plugin or destination (below) may define an "enabled" parameter that should
evaluate to True or False. When setup using
<param id="enabled" from_environ="<VAR>">DEFAULT</param>

This comment has been minimized.

Copy link
@bgruening

bgruening Jun 6, 2017

Member

DEFAULT should be true or false?

This comment has been minimized.

Copy link
@jmchilton

jmchilton Jun 6, 2017

Author Member

Yeah - should I write that:

<param id="enabled" from_environ="<VAR>">true|false</param> <!-- true|false is default -->
?

This comment has been minimized.

Copy link
@bgruening

bgruening Jun 6, 2017

Member

I think that makes it more clear, otherwise I could assume DEFAULT is a special keyword.

<destination id="local" runner="local"/>
<destination id="local2_dest" runner="local2">
<param id="enabled" from_environ="LOCAL2_ENABLED">True</param>
</destination>

This comment has been minimized.

Copy link
@bgruening

bgruening Jun 6, 2017

Member

indentation

Add the ability to control job conf plugin loading from environment v…
…ariables.

All job configuration parameters can be set using environment variables but currently the actual loading of the plugin cannot. This is problematic for setting up single job confs for multiple environemnts (like done with docker-galaxy-stable) when using plugins that may hang waiting for resources (such as Kubernetes for the K8 runner or a MQ for Pulsar for instance) or may require library dependencies not be setup in every environment the job configuration is deployed in.

xref bgruening/docker-galaxy-stable#328

Rebased with advice from @bgruening.

@jmchilton jmchilton force-pushed the jmchilton:dynamic_disable_runners branch from 48982f1 to d14eb9e Jun 6, 2017

@jmchilton

This comment has been minimized.

Copy link
Member Author

commented Jun 6, 2017

@bgruening Thanks for the review - I've made some changes - let me know if they look okay.

@bgruening

This comment has been minimized.

Copy link
Member

commented Jun 6, 2017

@jmchilton they are! Think it is ready to be merged as soon as it turns green.

@jmchilton

This comment has been minimized.

Copy link
Member Author

commented Jun 6, 2017

The remaining failing test is unrelated - it has been transiently failing on a few different PRs.

@bgruening bgruening merged commit ee4826f into galaxyproject:dev Jun 6, 2017

4 of 5 checks passed

api test Build finished. 279 tests run, 0 skipped, 1 failed.
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
framework test Build finished. 150 tests run, 0 skipped, 0 failed.
Details
integration test Build finished. 34 tests run, 0 skipped, 0 failed.
Details
toolshed test Build finished. 579 tests run, 0 skipped, 0 failed.
Details
@jmchilton

This comment has been minimized.

Copy link
Member Author

commented Jun 6, 2017

Thanks @bgruening - I'll take another pass at docker-galaxy-stable's compose setup now.

@jmchilton jmchilton deleted the jmchilton:dynamic_disable_runners branch Jun 7, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.