service.running and service.enabled fail to enable services (Debian and systemd) #34267

morganwillcock · 2016-06-24T09:46:29Z

I'm having problems trying to enable services using states. The following is an example of trying to enable the rsync service on a Debian 8 machine (systemd). The service is started if it's stopped, but it is never enabled (even though two states claim that it is). The service execution module does correctly report it as disabled, although in this example I'm querying directly with systemctl.

To re-create:

rsync_service:
  service.running:
    - name: rsync
    - enable: True

try_harder:
  service.enabled:
    - name: rsync

check_if_enabled:
  cmd.run:
    - name: systemctl is-enabled rsync.service

salt salt-test state.apply

          ID: rsync_service
    Function: service.running
        Name: rsync
      Result: True
     Comment: Service rsync is already enabled, and is running
     Started: 10:40:41.667092
    Duration: 1099.215 ms
     Changes:   
              ----------
              rsync:
                  True
----------

          ID: try_harder
    Function: service.enabled
        Name: rsync
      Result: True
     Comment: Service rsync is already enabled, and is in the desired state
     Started: 10:28:59.907359
    Duration: 121.893 ms
     Changes:   
----------
          ID: check_if_enabled
    Function: cmd.run
        Name: systemctl is-enabled rsync.service
      Result: False
     Comment: Command "systemctl is-enabled rsync.service" run
     Started: 10:29:00.032297
    Duration: 121.514 ms
     Changes:   
              ----------
              pid:
                  19057
              retcode:
                  1
              stderr:
              stdout:
                  disabled

Versions Report

Salt Version:
           Salt: 2016.3.1
 
Dependency Versions:
           cffi: 0.8.6
       cherrypy: Not Installed
       dateutil: 2.2
          gitdb: 0.5.4
      gitpython: 0.3.2 RC1
          ioflo: Not Installed
         Jinja2: 2.7.3
        libgit2: Not Installed
        libnacl: Not Installed
       M2Crypto: Not Installed
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.4.2
   mysql-python: 1.2.3
      pycparser: 2.10
       pycrypto: 2.6.1
         pygit2: Not Installed
         Python: 2.7.9 (default, Mar  1 2015, 12:57:24)
   python-gnupg: Not Installed
         PyYAML: 3.11
          PyZMQ: 14.4.0
           RAET: Not Installed
          smmap: 0.8.2
        timelib: Not Installed
        Tornado: 4.2.1
            ZMQ: 4.0.5
 
System Versions:
           dist: debian 8.5 
        machine: x86_64
        release: 3.16.0-4-amd64
         system: Linux
        version: debian 8.5

The text was updated successfully, but these errors were encountered:

Ch3LL · 2016-06-24T19:38:33Z

@morganwillcock I was able to replicate this. weirdly enough once I enable the service maually the state started working even if I disabled it again. Are you seeing the same behavior?

morganwillcock · 2016-06-24T19:53:55Z

@Ch3LL Thank you for looking at this. Yes, I see the same thing. I manually disabled the rsync service and re-applying the states has successfully re-enabled it.

edgan · 2016-06-30T11:16:07Z

I am seeing the same issue on Ubuntu 16.04(Xenial).

edgan · 2016-06-30T11:30:38Z

It looks like salt is running the check, and is getting back disabled as an answer. Then it isn't doing anything about it.

saltstack/map.jinja:
minion:
service:
name: salt-minion

services.sls:
{% from 'saltstack/map.jinja' import saltstack with context %}

saltstack_minion_service:
service.running:
- name: {{ saltstack.minion.service.name }}
- enable: True
- watch:
- file: saltstack_minion_config_file

/var/log/salt/minion:
2016-06-30 04:20:47,605 [salt.loaded.int.module.cmdmod][INFO ][27446] Executing command ['systemctl', 'is-enabled', 'salt-minion.service'] in directory '/root'
2016-06-30 04:20:47,613 [salt.loaded.int.module.cmdmod][DEBUG ][27446] output: disabled

systemctl status salt-minion.service
● salt-minion.service - The Salt Minion
Loaded: loaded (/lib/systemd/system/salt-minion.service; disabled; vendor preset: enabled)
Active: active (running) since Thu 2016-06-30 04:20:47 PDT; 5min ago
Main PID: 30981 (salt-minion)
Tasks: 6
Memory: 90.0M
CPU: 22.535s
CGroup: /system.slice/salt-minion.service
├─30981 /usr/bin/python /usr/bin/salt-minion
├─30996 /usr/bin/python /usr/bin/salt-minion
└─31000 /usr/bin/python /usr/bin/salt-minion

edgan · 2016-06-30T11:51:07Z

What I think may be happening for me is that salt-miinion also has /etc/init/salt-minion.conf and /etc/init.d/salt-minion. The disabling of the service works the upstart way with /etc/init/salt-minion.override if /etc/init/salt-minion.conf exists. If it doesn't, if /etc/init.d/salt-minion exists, then it does it the init.d way by creating symlinks in /etc/rc3.d. If /etc/init.d/salt-minion doesn't exist, then it finally does it the systemd way by putting a symlink, /etc/systemd/system/multi-user.target.wants/salt-minion.service.

edgan · 2016-06-30T11:53:18Z

I think they only expect you to have one, but if you have more than one you can get this confused behavior. In the cause of salt-minion, it is just poor packaging.

edgan · 2016-06-30T19:17:10Z

Looks like this is fixed in master, but hasn't made it into a released version.

828cf0c

wblair · 2016-07-14T19:51:38Z

FWIW: I have similar behavior on Centos 7 not enabling an elasticsearch service when run through highstate. When we provision Amazon stack, the master starts first. Each minion independently runs salt-call to apply its highstate.

The state below will start the service but systemd reports it as disable (and it is really is because it does not start on reboot).

start-service:
service.running:
- name: elasticsearch
- enable: true
- watch:
- file: {{ ES_CONFIG_FILE }}
- file: {{ ES_PLUGIN_DIR }}/blahblahblah/{{ TK_ES_PLUGIN }}
- file: {{ ES_PLUGIN_DIR }}/blahblahblah/plugin-descriptor.properties

[centos@es01.wb-S5-1 ~]$ systemctl status elasticsearch
● elasticsearch.service - Elasticsearch
Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service; disabled; vendor preset: disabled)
Active: active (running) since Thu 2016-07-14 18:18:28 UTC; 1h 23min ago

However, if I then run just the specific id it will enable the service.

[centos@es01.wb-S5-1 ~]$ sudo salt-call state.sls_id start-service elasticsearch

      ID: start-service
Function: service.running
    Name: elasticsearch
  Result: True
 Comment: Service elasticsearch has been enabled, and is in the desired state
 Started: 19:43:47.082646
Duration: 304.342 ms
 Changes:   
          ----------
          elasticsearch:
              True

Summary for local

Succeeded: 4 (changed=1)

Failed: 0

Total states run: 4

And now we see the service is now enabled.

[centos@es01.wb-S5-1 ~]$ systemctl status elasticsearch
● elasticsearch.service - Elasticsearch
Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service; enabled; vendor preset: disabled)
Active: active (running) since Thu 2016-07-14 18:18:28 UTC; 1h 25min ago

[centos@es01.wb-S5-1 ~]$ sudo salt-call --versions-report
Salt Version:
Salt: 2015.8.8.2

Dependency Versions:
Jinja2: 2.7.2
M2Crypto: Not Installed
Mako: Not Installed
PyYAML: 3.11
PyZMQ: 14.7.0
Python: 2.7.5 (default, Nov 20 2015, 02:00:19)
RAET: Not Installed
Tornado: 4.2.1
ZMQ: 4.0.5
cffi: Not Installed
cherrypy: Not Installed
dateutil: 2.5.3
gitdb: Not Installed
gitpython: Not Installed
ioflo: Not Installed
libgit2: Not Installed
libnacl: Not Installed
msgpack-pure: Not Installed
msgpack-python: 0.4.7
mysql-python: Not Installed
pycparser: Not Installed
pycrypto: 2.6.1
pygit2: Not Installed
python-gnupg: Not Installed
smmap: Not Installed
timelib: Not Installed

System Versions:
dist: centos 7.2.1511 Core
machine: x86_64
release: 3.10.0-327.22.2.el7.x86_64
system: CentOS Linux 7.2.1511 Core

jmcgrail · 2016-07-15T14:49:01Z

Under the hood, Wayne's issue is likely:

[INFO    ] Executing command ['systemctl', 'is-enabled', 'elasticsearch.service'] in directory '/root'
[ERROR   ] An exception occurred in this state: Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/salt/state.py", line 1626, in call
    **cdata['kwargs'])
  File "/usr/lib/python2.7/site-packages/salt/loader.py", line 1492, in wrapper
    return f(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/salt/states/service.py", line 325, in running
    ret.update(_enable(name, True, **kwargs))
  File "/usr/lib/python2.7/site-packages/salt/states/service.py", line 119, in _enable
    if __salt__['service.enable'](name, **kwargs):
  File "/usr/lib/python2.7/site-packages/salt/modules/systemd.py", line 695, in enable
    cmd = [_get_service_exec(), '-f', name, 'defaults', '99']
  File "/usr/lib/python2.7/site-packages/salt/modules/systemd.py", line 138, in _get_service_exec
    salt.utils.check_or_die(executable)
  File "/usr/lib/python2.7/site-packages/salt/utils/__init__.py", line 804, in check_or_die
    raise CommandNotFoundError(command)
CommandNotFoundError: update-rc.d

[INFO    ] Completed state [elasticsearch] at time 13:16:32.306074
[INFO    ] Running state [elasticsearch] at time 13:16:32.306337

The above is from the cfn-init.log on an amazon instance we spun up using his rules.

Ch3LL · 2016-08-05T20:58:00Z

I verified that the commit @edgan references does indeed fix this issue. I will see about getting it backported to 2016.3

rallytime · 2016-08-05T21:18:36Z

@Ch3LL Back-port is in #35244

wblair · 2016-08-09T00:55:51Z

Thank you, thank you!
On Aug 5, 2016 4:58 PM, "Megan Wilhite" notifications@github.com wrote:

I verified that the commit @edgan https://github.com/edgan references
does indeed fix this issue. I will see about getting it backported to 2016.3

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
#34267 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AGO53CN0nsbV_7RyS23473QMECzSRfcFks5qc6PrgaJpZM4I9nr-
.

Ch3LL added Bug broken, incorrect, or confusing behavior P3 Priority 3 severity-medium 3rd level, incorrect or bad functionality, confusing and lacks a work around Platform Relates to OS, containers, platform-based utilities like FS, system based apps labels Jun 24, 2016

Ch3LL added this to the Approved milestone Jun 24, 2016

Ch3LL added the State-Module label Jun 24, 2016

This was referenced Aug 5, 2016

Return correct value for services that must be enabled in Systemd #31677

Merged

service.running with enable & watch does not enable #35220

Closed

rallytime added the fixed-pls-verify fix is linked, bug author to confirm fix label Aug 5, 2016

cachedout closed this as completed Aug 9, 2016

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

service.running and service.enabled fail to enable services (Debian and systemd) #34267

service.running and service.enabled fail to enable services (Debian and systemd) #34267

morganwillcock commented Jun 24, 2016

Ch3LL commented Jun 24, 2016

morganwillcock commented Jun 24, 2016

edgan commented Jun 30, 2016

edgan commented Jun 30, 2016

edgan commented Jun 30, 2016

edgan commented Jun 30, 2016

edgan commented Jun 30, 2016

wblair commented Jul 14, 2016 •

edited

Loading

jmcgrail commented Jul 15, 2016 •

edited

Loading

Ch3LL commented Aug 5, 2016

rallytime commented Aug 5, 2016

wblair commented Aug 9, 2016

service.running and service.enabled fail to enable services (Debian and systemd) #34267

service.running and service.enabled fail to enable services (Debian and systemd) #34267

Comments

morganwillcock commented Jun 24, 2016

Versions Report

Ch3LL commented Jun 24, 2016

morganwillcock commented Jun 24, 2016

edgan commented Jun 30, 2016

edgan commented Jun 30, 2016

edgan commented Jun 30, 2016

edgan commented Jun 30, 2016

edgan commented Jun 30, 2016

wblair commented Jul 14, 2016 • edited Loading

[centos@es01.wb-S5-1 ~]$ sudo salt-call state.sls_id start-service elasticsearch

Summary for local

Failed: 0

jmcgrail commented Jul 15, 2016 • edited Loading

Ch3LL commented Aug 5, 2016

rallytime commented Aug 5, 2016

wblair commented Aug 9, 2016

wblair commented Jul 14, 2016 •

edited

Loading

jmcgrail commented Jul 15, 2016 •

edited

Loading