salt-call returns 0 on failures #50565
Description
Description of Issue/Question
salt-call returns success (0) regardless of failures. I am using masterless Salt for GitLab/VSTS pipelines and this is a real problem as without failure codes all steps always run and cause chaos.
Setup
Here is simple formula with a bug in it ({{bug}}):
[root@sspvminmaster mule]# cat test.sls
/tmp/test.txt:
file.managed:
- mode: 0644{{bug}}
- replace: False
Steps to Reproduce Issue
Run with salt works fine:
# salt 'salt-master' state.apply test; echo $?
salt-master:
Data failed to compile:
----------
Rendering SLS 'base:mule.test' failed: Jinja variable 'bug' is undefined
ERROR: Minions returned with non-zero exit code
11 <-------------- CORRECT
Run with salt-call :
# salt-call state.apply test; echo $?
local:
Data failed to compile:
----------
Rendering SLS 'base:mule.test' failed: Jinja variable 'bug' is undefined
0 <--------------- WRONG
Versions Report
Salt Version:
Salt: 2018.3.2
Dependency Versions:
cffi: 1.11.5
cherrypy: Not Installed
dateutil: 2.6.1
docker-py: Not Installed
gitdb: 2.0.3
gitpython: 2.1.11
ioflo: Not Installed
Jinja2: 2.10
libgit2: Not Installed
libnacl: Not Installed
M2Crypto: 0.28.2
Mako: Not Installed
msgpack-pure: Not Installed
msgpack-python: 0.5.6
mysql-python: Not Installed
pycparser: 2.14
pycrypto: 2.6.1
pycryptodome: Not Installed
pygit2: Not Installed
Python: 2.7.15 (default, Oct 15 2018, 15:24:06)
python-gnupg: Not Installed
PyYAML: 3.12
PyZMQ: 17.0.0
RAET: Not Installed
smmap: 2.0.3
timelib: Not Installed
Tornado: 5.0.2
ZMQ: 4.1.6
System Versions:
dist: fedora 28 Twenty Eight
locale: UTF-8
machine: x86_64
release: 4.18.17-200.fc28.x86_64
system: Linux
version: Fedora 28 Twenty Eight