Use environment variables in actions

[dzimine]

Initially posted by @jtopjian in google groups:

Is it possible for actions to inherit the environment of the user that issued the action?

I think this would be useful in many scenarios, but the one scenario that made me think of it was playing with the OpenStack pack. That pack comes with a config.yml file that asks for the OpenStack cloud information. The problem here is that it requires me to have a credentials file in a different format than the openrc file we normally deploy (which is a bit of a hassle when we have that config file under config management). Secondly it is limited to only one cloud/region. On some of our "control" servers, we have several rc files for different clouds and regions of those clouds. If we want to run a command against a certain cloud, we just source the specific rc file.

By being able to inherit the environment, we could then gain the flexibility of being able to run actions against several clouds without creating a complex config.yml as well as prefix the environment in cron and such (though StackStorm can do cron on its own ;)

Of course this would also require modifying the OpenStack pack (for this specific scenario) but that's rather easy.

[jfryman]

/cc #1033

[dzimine]

@Kami has a pull request up: #1033

But how do we want this exactly to be used? What are the use cases?

• if I already have a recourerc file with my env variables?
• Or, if these variable are already sourced on server X for user Y (bummer we can’t run as an arbitrary user but ok, if they’re sourced for user Stanley?)
• Or, if there are no env variables sourced yet but the script wants them (the case Tomaz is covering)?
• Or I want to provide them in a file (e.g. I already have the file here locally, don’t make me re-write / re-type
• Or, I want to use grab something from KV store and throw them to env variables?
• Or, what else?

[manasdk]

Few comments on what is already available or can be done easily.

• if I already have a recourerc file with my env variables?

This use case seems reasonable. We have experienced something very similar if not exactly this and solved it by sourcing the resourcerc at top of the file or in the bash command so something like -
source ~/resourcerc; {cmd}.

This can be done by a user even now but better platform support would be nicer.

• Or, if these variable are already sourced on server X for user Y (bummer we can’t run as an arbitrary user but ok, if they’re sourced for user Stanley?)

The tricky part here would be having the ability to ssh as that user onto the box from st2. Even now st2 stanley user can be replaced with a user of choice. I believe we run all commands with shell=True so all those user setting ought to apply for run-local and even run-remote is over ssh so that would be per typical ssh behavior. Running as stanley but accessing files from another users home dir would be dubious and most likely a non-starter. On this part though @DoriftoShoes and @jfryman can provide better inputs.

• Or I want to provide them in a file (e.g. I already have the file here locally, don’t make me re-write / re-type

This should be useful. Not much different from resourcerc - right?

• Or, I want to use grab something from KV store and throw them to env variables?

Should work with @Kami's change. Since all runners take env parameter it is possible to use any value from the k-v store.

[jtopjian]

Thanks for copying me on this.

One question for @manasdk: when you say:

This use case seems reasonable. We have experienced something very similar if not exactly this and solved it by sourcing the resourcerc at top of the file or in the bash command so something like -
source ~/resourcerc; {cmd}.

Does cmd equal st2 in this case? If so, then that's exactly what I'm looking for:

source ~/resourcerc; st2 run core...

Some other thoughts:

• I think Allow user to provide environment variables by passing "env" parameter to the runners #1033 is definitely helpful and provides a bridge. I'd probably end up using it like:

$ st2 run core.remote hosts=localhost env="{"key1": '${KEY1}', "key2": '${KEY2}'"

Or something similar so it pulls existing variable values. Thought it'd definitely be preferable to not have to specify each variable manually like this.

• If the action itself has to source a file of environment variables rather than have the variables inherited from the user running st2, I can see that as a decent solution. I could then just have a parameter that specifies the location of the file.

The only problem with this, though, is it could be a pain to read in a bash-style file with environment variables and get them to be parsed by, say, Python or Ruby.

• And I definitely understand the complication of having the environment available on the remote servers. That'd be awesome to have, but really, I'd be very happy if just local actions and runners had access to the environment.

[Kami]

@jtopjian

Does cmd equal st2 in this case? If so, then that's exactly what I'm looking for:

I assume @manasdk meant the command you want executed and not st2. So something like st2 run core.remote cmd="source ./blah.sh ; my command" (@manasdk please correct me if I'm wrong).

It looks like what you want is to automatically inherit all the environment variables which are accessible to the st2 cli process, right?

If so, that's definitely possible to do, but I'm not sure if putting this functionality into the cli tool itself is the best thing to do. If we put this functionality into cli, I assume we would add some kind of flag to the run command - e.g. st2 run --inherit-env-vars cmd="...".

[jtopjian]

That's a good point. I agree with having a flag that will inherit the environment.

[Kami]

I think I would be fine with that and we don't need to do any additional checks since all the existing runners now support env parameter.

What do other ppl think?/cc @manasdk @DoriftoShoes @dzimine

[Kami]

@jtopjian #1044 - wdyt?

[jtopjian]

That looks awesome.

[Kami]

@jtopjian how would you want to handle your existing files with env variables? Should we give an ability to read a file?

I believe something like this should work:

(source foo.sh ; st2 run --inherit-env ...)

foo.sh

export FOO="bar"
...

I'm creating a sub-shell with parenthesis so it doesn't pollute current / parent shell with the variables from the file.

[jtopjian]

I had some time today to test this out and it works just as it should! This is really great and will allow us to further explore moving our collection of scripts into st2.

Thank you all for being so receptive to this feature :)

[dzimine]

ok, everyone is happy, closing as "DONE".