systemd.py available() breaks on latest LSB-compliant versions of systemd

[wrigtim]

Description of Issue/Question

Later versions of systemctl now follow LSB standards for return codes (https://refspecs.linuxbase.org/LSB_3.0.0/LSB-PDA/LSB-PDA/iniscrptact.html). As part of this change the STDOUT was removed and replaced with a simple STDERR response.

Setup

Any host running systemd 219-30 onward (RHEL 7.3, latest Fedora, etc)

Steps to Reproduce Issue

# salt-call service.available bogus.service
[INFO    ] Executing command ['systemctl', 'status', 'bogus.service', '-n', '0'] in directory '/root'
Error running 'service.available': Failed to get information on unit 'bogus.service'

Versions Report

Salt Version:
           Salt: 2015.8.7

Dependency Versions:
         Jinja2: 2.7.2
       M2Crypto: 0.21.1
           Mako: Not Installed
         PyYAML: 3.11
          PyZMQ: 14.7.0
         Python: 2.7.5 (default, Aug  2 2016, 04:20:16)
           RAET: Not Installed
        Tornado: 4.2.1
            ZMQ: 4.0.5
           cffi: Not Installed
       cherrypy: Not Installed
       dateutil: 1.5
          gitdb: Not Installed
      gitpython: Not Installed
          ioflo: Not Installed
        libgit2: Not Installed
        libnacl: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.4.7
   mysql-python: 1.2.5
      pycparser: Not Installed
       pycrypto: 2.6.1
         pygit2: Not Installed
   python-gnupg: Not Installed
          smmap: Not Installed
        timelib: Not Installed

System Versions:
           dist: redhat 7.3 Maipo
        machine: x86_64
        release: 3.10.0-506.el7.x86_64
         system: Red Hat Enterprise Linux Server 7.3 Maipo

[wrigtim]

New systemd behaviour:

$ systemctl status bogus.service; echo $?
Unit bogus.service could not be found.
4

Old behaviour:

$ systemctl status bogus.service; echo $?
● bogus.service
   Loaded: not-found (Reason: No such file or directory)
   Active: inactive (dead)
3

[Ch3LL]

@wrigtim thanks for the PR! we will get this closed once your PR has been reviewed and merged.

[terminalmage]

@wrigtim What version of systemd were you running when you generated that output? According to the git history for systemd, the different output when the unit cannot be found is new in systemd 231, as is the retcode of 4.

For instance, this is the result on CentOS 7 running systemd 219:

[root@cent7 /]# systemctl --version; systemctl status foo.service -n 0; echo $?
systemd 219
+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ -LZ4 -SECCOMP +BLKID +ELFUTILS +KMOD +IDN
● foo.service
   Loaded: not-found (Reason: No such file or directory)
   Active: inactive (dead)
3

As far as I can see, the change to the return code to make it properly LSB-compliant when the service is unknown (which is what we're looking for in the available() function) was merged just a few weeks before systemd 231 was released (see here). Also, the change to make systemctl status suppress the normal output in favor of the error message appears to originate here, a couple weeks after the PR which made the retcode change.

[terminalmage]

I've opened #36755 as a potential alternative to the PR opened by @wrigtim, based on my findings that both the retcode and changes to systemctl status output are only in systemd 231.

[wrigtim]

Replied on PR, but this is 7.3 beta (looks like RedHat have back-ported the upstream changes)...

[terminalmage]

@wrigtim can you please test the modified code from #36755 that takes into account your use case with the features being backported into earlier systemd? It has been merged, but as I do not have a RHEL 7.3 box available I wanted to make sure we got it tested.

[wrigtim]

Sure - will test Monday in office (don't have 7.3 here). Apologies for the delay but I've been travelling...

[terminalmage]

No problem! I'm almost 100% certain that this will work fine, but I don't have 7.3 available to test either, so getting confirmation from you will be a good sanity check.

Thanks in advance, and thanks for helping bring this to our attention.

[wrigtim]

@terminalmage Looks good to me - thanks very much!

[root@testing ~]# cat /etc/redhat-release 
Red Hat Enterprise Linux Server release 7.3 Beta (Maipo)

[root@testing ~]# salt-call service.available sshd.service                                                                                                   
[INFO    ] Determining pillar cache
[INFO    ] Executing command ['systemctl', 'status', 'sshd.service', '-n', '0'] in directory '/root'
local:
    True

[root@testing ~]# salt-call service.available BOGUS.service                                                                                                  
[INFO    ] Determining pillar cache
[INFO    ] Executing command ['systemctl', 'status', 'BOGUS.service', '-n', '0'] in directory '/root'
local:
    False

[terminalmage]

@wrigtim Thanks for taking the time to confirm, I really appreciate it! I'll mark this as closed.