Execute command on Salt master after salt-call cp.push from minion to Salt master.

[ghost]

Description of Issue/Question

We are running Salt state from Salt master to minion for certain task , where we are copying log file from minion to master using salt-call cp.push /tmp/test.log. It is directly copying to minion cache path on Salt master. Once copy of file is completed , we want to move the file from minion cache path to /opt of salt master. How can we automated it without executing any manual command on Salt master ?
We are executing : salt 'minion-name' state.apply sometaskon-minion -v and we want to automate the file move from minion cache directory of salt master to /opt of salt master as part of one state file which is "sometaskon-minion" as per my salt command.

Setup

(Please provide relevant configs and/or SLS files (Be sure to remove sensitive info).)
we have salt master on one VM and minion on separate VM. Salt master also has it's own minion running on salt master VM

Steps to Reproduce Issue

(Include debug logs if possible and relevant.)

Versions Report

(Provided by running salt --versions-report. Please also mention any differences in master/minion versions.)
Salt Version:
Salt: 2016.11.6

Dependency Versions:
cffi: Not Installed
cherrypy: Not Installed
dateutil: Not Installed
docker-py: Not Installed
gitdb: Not Installed
gitpython: Not Installed
ioflo: Not Installed
Jinja2: 2.8.1
libgit2: Not Installed
libnacl: Not Installed
M2Crypto: Not Installed
Mako: Not Installed
msgpack-pure: Not Installed
msgpack-python: 0.4.6
mysql-python: Not Installed
pycparser: Not Installed
pycrypto: 2.6.1
pycryptodome: 3.4.3
pygit2: Not Installed
Python: 2.6.6 (r266:84292, Aug 18 2016, 15:13:37)
python-gnupg: 0.3.8
PyYAML: 3.11
PyZMQ: 14.5.0
RAET: Not Installed
smmap: Not Installed
timelib: Not Installed
Tornado: 4.2.1
ZMQ: 4.0.5

System Versions:
dist: centos 6.9 Final
machine: x86_64
release: 2.6.32-696.3.2.el6.x86_64
system: Linux
version: CentOS 6.9 Final

[gtmanfred]

I would use the inotify beacon and run it with the minion on the salt master.

https://docs.saltstack.com/en/latest/ref/beacons/all/salt.beacons.inotify.html

You could potentially watch for the cp.push command to send an event on the master event bus, and trigger the move using a reactor. you will need to check salt-run state.event pretty=True to see what that event looks like.

https://docs.saltstack.com/en/getstarted/event/reactor.html
https://docs.saltstack.com/en/latest/topics/reactor/

Thanks,
Daniel

[ghost]

Hi Daniel,
Thanks for the update For reactor , It is possible to put it inside state file and execute as part of single command.

Example : salt 'minionname' state.apply dotask -v

Here we are executing state file against minionname and minion will trigger cp.push to master . How can I again call salt-run or salt command on salt master ? Here I am targeting to salt-minion on my state.apply and from minion , I can't run salt command as salt binary is part of Salt master .

The only option could be , I call the salt-minion on Salt master. For this complete process can I automate everything as part of same state file which will run : salt 'minionname' state.apply dotask -v
or I have to execute separate states ? One for minion and other for the salt-minion on Salt master ?
Thanks & regards
Sas

[gtmanfred]

Here is what I would do, add fire_event to your state that runs the cp.push

push file:
  module.run:
    - name: cp.push
    - path: /path/to/file
    - fire_event: salt/{{minionid}}/cp/push/finished

Add a reactor event for that

reactor:
  - salt/*/cp/push/finished
    - salt://reactor/finish_push.sls

and then write a runner module that can be called from that reactor file

salt://_runner/custom.py

import os
def dotask(minionid):
    os.rename('/var/cache/salt/master/minionfs/{0}/path/to/file'.format(minionid), '/opt/logs/{0}-filename'.format(minionid))
    return True

And then call the custom runner from the reactor file

salt://reactor/finish_push.sls

do task:
  runner.custom.dotask:
    - minionid: {{data['data']['id']}}

You will want to check the output of salt-run state.event pretty=True to make sure all the jinja will work for pulling information from the minion, and you could potentially grab the path from that as well.

[ghost]

Thanks . I will test it .

[ghost]

I have some issues. It seems reactor is not running . Below is my configuration

reactor-test.sls
push_file:
2 module.run:
3 - name: cp.push
4 - path: /root/test.txt
5 - fire_event: salt/test-vm1/cp/push/finished # Here test-vm1 is the minion id

[root@salt-master# tree
.
├── reactor
│   └── finish_push.sls
├── _runner
│   └── custom.py

My finish_push.sls reactor file contents :
do task:
runner.custom.dotask:
- minionid: {{data['data']['id']}}

My _runner file content :
#!/usr/bin/python
import os
def dotask(minionid):
os.rename('/var/cache/salt/master/minions/test-vm1/{0}/files/root/test.txt'.format(minionid), '/opt/{0}-filename'.format(minionid))
return True

I added /etc/salt/master for reactor configuration and it is : salt-master service is also restarted

reactor:
1150 - /srv/salt/*/cp/push/finished
1151 - /srv/salt/reactor/finish_push.sls

I can see file is being copied from minion to salt master minion cache directory , which is :
/var/cache/salt/master/minions/test-vm1/files/root/test.txt but file copy to /opt on salt master is not working.

reactor state file should trigger test.txt file copy from minion cache directory to /opt/ of salt master.

Questions

1- cp.push state file you have used {{ minionid }} which suppose to replace from pillar but in my case I just used it hard coded as test-vm1 . Do I need to use only pillar variable ?

2- reactor state file - minionid: {{data['data']['id']}} , What is minionid here in inside finish_push.sls

3- Same also inside _runner custom.py script . we have minionid .function.

4- Is there a way to debug if reactor state file is being really called after event is triggered. I can see event is being triggered but after that reactor is not being triggered.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

If this issue is closed prematurely, please leave a comment and we will gladly reopen the issue.