PATH being overwritten

[MrTravisB]

Running CentOS. Salt 0.16.3. Minion and master running as root user.

On minion:

[minion ~]# echo $PATH
/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/bin:/opt/aws/bin:/ops/bin
[minion ~]# salt-call cmd.run 'echo $PATH'
local:
    /sbin:/bin:/usr/sbin:/usr/bin:/usr/local/bin:/opt/aws/bin:/ops/bin

On master:

[master ~]# salt -G 'host:minion' cmd.run 'echo $PATH'
minion:
    /sbin:/usr/sbin:/bin:/usr/bin

The only place in saltstack codebase I can find /sbin:/usr/sbin:/bin:/usr/bin is in the Debian init file. I don't see it in any RHEL related files.

[dlanderson]

MrTravisB: where are you setting your PATH info for the salt-minion service?

Salt devs: Because the init functions for CentOS are sourced after the /etc/default/salt-xxx configs, if the user is trying to set PATH in e.g. /etc/default/salt-minion, it will be overwritten when /etc/rc.d/init.d/functions is sourced in https://github.com/saltstack/salt/blob/develop/pkg/rpm/salt-minion#L46. The system init functions (and lsb-func and lsb-vars for deb based) in all of the init scripts should probably be sourced first, then any hard-coded defaults in the init script, then the /etc/default/salt-xxx files.

[MrTravisB]

I'm not explicitly setting PATH at all for the minion. I would have hoped it would use the default PATH.

[MrTravisB]

It doesn't seem this is actually a bug with Salt. To fix it, I simply added my PATH setting to /etc/rc.d/init.d/functions and it is working fine now.

I'm going to close this issue. If anyone can think of a better way around this then feel free to reopen.

[dlanderson]

To clarify: are you wanting salt-minion's process itself to have a different PATH, or are you wanting the path for the cmd.run module/function to behave as if it were an interactive shell like you have when SSHed into the minion?

The salt-minion process does actually have the default PATH for an init script on RedHat/CentOS, "/sbin:/usr/sbin:/bin:/usr/bin". You can see this set in the /etc/rc.d/init.d/functions file, which the salt-minion script sources on startup. In order to change the PATH for the salt-minion process itself, you should be able to set it in /etc/default/salt-minion, but there is currently a bug as I mentioned in my first comment about the order of sourcing being wrong. Until that is fixed, you have to set the PATH in /etc/init.d/salt-minion on RedHat/CentOS.

I would advise against editing the PATH in the init.d/functions file, as your changes will be lost any time the package that provides those function is updated by RedHat/CentOS.

If you want the cmd.run environment to be more like your interactive (assuming bash) shell, you need to do some things first:

• cmd.run detects the default shell to use by checking the 'shell' grains item on the minion. The docs on the salt website say the default is /bin/bash, but this is not true in the code. Check your default shell like this:

  root@salt-master:# salt 'vbox*' grains.get shell
  vbox.dubkat.com:
      /bin/sh

• Since /bin/sh is not the same as /bin/bash on Ubuntu, I need to specify my shell in the cmd.run args. RedHat still links /bin/sh to /bin/bash, but you may want to confirm on your minion.

• When you say "default PATH" regarding your login shell, you are referring to variables that are set in the /etc/profile or /etc/bashrc (RedHat) or /etc/bash.bashrc (Debian). These get loaded only depending on how bash is invoked. Because salt is invoking bash non-interactively and through the subprocess.Popen call in python, bash doesn't source these files as it does in your login shells since it is not considered an interactive login. See the 'INVOCATION' section in your bash man page. In order to get bash to load /etc/profile for a non-interactive shell, we have to nudge it with the BASH_ENV variable:

  root@salt-master:/# salt 'vbox*' cmd.run 'echo $PATH' shell='/bin/bash'
  vbox.dubkat.com:
      /sbin:/usr/sbin:/bin:/usr/bin
  root@salt-master:/# salt 'vbox*' cmd.run 'echo $PATH' shell='/bin/bash' env='{BASH_ENV: "/etc/profile"}'
  vbox.dubkat.com:
      /usr/local/sbin:/sbin:/usr/sbin:/bin:/usr/bin

Looking at the salt code, when you add the 'runas' option, the shell is invoked through either 'sudo' or 'su' depending on the OS of the minion, and all of them have the 'interactive' option passed to su/sudo. So you could just use the 'runas' option to get bash to launch an interactive shell which will then read your profile/bashrc scripts:

    root@salt-master:~# salt 'vbox*' cmd.run 'echo $PATH' shell='/bin/sh' runas='root'
    vbox.dubkat.com:
        /usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
    root@salt-master:~# salt 'vbox*' cmd.run 'echo $PATH' shell='/bin/bash' runas='root'
    vbox.dubkat.com:
        /usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin

This is simply due to the quirks of bash, so it's not really a bug in salt's cmd.run module.

In any case, the init script sourcing order bug is still valid, so please feel free to reopen this issue. Otherwise I will file a separate bug.

Hope this helps.

[beherca]

This is how we solve the problem:
use
pkill salt-minion
salt-minion -d

instead of using service salt-minion restart

[qingchn]

good, I want to know is fix ?