salt reactor - un-handled exception from the multiprocessing process

[Vox1984]

Description of Issue

I have following very simple reactor.conf and when I trigger it with event from commandline it crashes with

An un-handled exception from the multiprocessing process 'Reactor-4:1'

As far I understand, correct sls is being used (as defined in reactor), then it is being read to OrderedDict, then it tries populate_client_cache and fails.

Setup

reactor.conf on master (/etc/salt/master.d/reactor.conf)
it is the only reactor

reactor:
  - 'salt/beacon/ssc-001/diskusage/':
    - salt://role/ssc/cleanup.sls

cleanup.sls

create dummy file:
  file.managed:
    - tgt: 'target'
    - name: /tmp/wojtek
    - source: salt://role/ssc/files/cleanup.py

Steps to Reproduce Issue

1. run salt-master -l debug on master
2. trigger the event on minion: salt-call event.send 'salt/beacon/ssc-001/diskusage/'

[DEBUG   ] Rendered data from file: /var/cache/salt/master/files/base/role/ssc/cleanup.sls:
create dummy file:
  file.managed:
    - tgt: 'ssc-001'
    - name: /tmp/wojtek
    - source: salt://role/ssc/files/cleanup.py

[DEBUG   ] Results of YAML rendering: 
OrderedDict([(u'create dummy file', OrderedDict([(u'file.managed', [OrderedDict([(u'tgt', u'ssc-001')]), OrderedDict([(u'name', u'/tmp/wojtek')]), OrderedDict([(u'source', u'salt://role/ssc/files/cleanup.py')])])]))])
[PROFILE ] Time (in seconds) to render '/var/cache/salt/master/files/base/role/ssc/cleanup.sls' using 'yaml' renderer: 0.000705003738403
[DEBUG   ] Reactor is populating file client cache
[ERROR   ] An un-handled exception from the multiprocessing process 'Reactor-4:1' was caught:
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/salt/utils/process.py", line 765, in _run
    return self._original_run()
  File "/usr/lib/python2.7/site-packages/salt/utils/reactor.py", line 271, in run
    self.call_reactions(chunks)
  File "/usr/lib/python2.7/site-packages/salt/utils/reactor.py", line 228, in call_reactions
    self.wrap.run(chunk)
  File "/usr/lib/python2.7/site-packages/salt/utils/reactor.py", line 330, in run
    self.populate_client_cache(low)
  File "/usr/lib/python2.7/site-packages/salt/utils/reactor.py", line 324, in populate_client_cache
    self.reaction_class[reaction_type](self.opts['conf_file'])
KeyError: u'file'
[CRITICAL] Engine 'reactor' could not be started!
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/salt/engines/__init__.py", line 132, in run
    self.engine[self.fun](**kwargs)
  File "/usr/lib/python2.7/site-packages/salt/engines/reactor.py", line 36, in start
    salt.utils.reactor.Reactor(__opts__).run()
  File "/usr/lib/python2.7/site-packages/salt/utils/process.py", line 765, in _run
    return self._original_run()
  File "/usr/lib/python2.7/site-packages/salt/utils/reactor.py", line 271, in run
    self.call_reactions(chunks)
  File "/usr/lib/python2.7/site-packages/salt/utils/reactor.py", line 228, in call_reactions
    self.wrap.run(chunk)
  File "/usr/lib/python2.7/site-packages/salt/utils/reactor.py", line 330, in run
    self.populate_client_cache(low)
  File "/usr/lib/python2.7/site-packages/salt/utils/reactor.py", line 324, in populate_client_cache
    self.reaction_class[reaction_type](self.opts['conf_file'])
KeyError: u'file'

Versions Report

Salt Version:
           Salt: 2019.2.0
 
Dependency Versions:
           cffi: 1.6.0
       cherrypy: Not Installed
       dateutil: Not Installed
      docker-py: Not Installed
          gitdb: Not Installed
      gitpython: Not Installed
          ioflo: Not Installed
         Jinja2: 2.7.2
        libgit2: 0.26.3
        libnacl: Not Installed
       M2Crypto: Not Installed
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.5.6
   mysql-python: Not Installed
      pycparser: 2.14
       pycrypto: 2.6.1
   pycryptodome: Not Installed
         pygit2: 0.26.4
         Python: 2.7.5 (default, Jun 20 2019, 20:27:34)
   python-gnupg: Not Installed
         PyYAML: 3.11
          PyZMQ: 15.3.0
           RAET: Not Installed
          smmap: Not Installed
        timelib: Not Installed
        Tornado: 4.2.1
            ZMQ: 4.1.4
 
System Versions:
           dist: centos 7.6.1810 Core
         locale: UTF-8
        machine: x86_64
        release: 3.10.0-957.27.2.el7.x86_64
         system: Linux
        version: CentOS Linux 7.6.1810 Core

[waynew]

So, this is sort of a bug, but also expected behavior. It's a bug, because it's kind of a useless error message.

You have:

create dummy file:
  file.managed:
    - tgt: 'target'
    - name: /tmp/wojtek
    - source: salt://role/ssc/files/cleanup.py

But that's not how reactors work. You wanted:

create dummy file:
  local.state.single:
    - tgt: target
    - expr_form: list
    - kwarg:
        fun: file.managed
        name: /tmp/wojtek
        source: salt://role/ssc/files/cleanup.py

See #30028 for more info