Service Module Fails with Sys V init script on a system that has Systemd

[tkinz27]

Issue Type: Bug Report

Ansible Version:

root@68b23fe4c544:~/Code/devel/tools/python/atlas# ansible --version
ansible 1.9
configured module search path = /usr/share/ansible:./modules

root@68b23fe4c544:/opt/ansible-new# git log -p -1
commit fb717ff
Merge: a68ca90 57c81df
Author: Brian Coca bcoca@ansible.com
Date: Fri Mar 6 11:45:07 2015 -0500

Merge pull request #10345 from emonty/features/openstack-inventory-docs

Add documentation for the new OpenStack Inventory

Environment:

I am running ansible from within a cloudfoundry/lucid64:minbase Docker image. I installed ansible from git.

The system that I am orchestrating is a Fedora 15 machine.

Summary:

Service module tries to use systemctl when it detects the system is using systemd, but the service I am trying to restart is using still using Sys V style init script. Further described ( including a possible fix ) on the ansible developer list ( see https://groups.google.com/forum/#!topic/ansible-devel/0ml1y6gkmEc )

Steps To Reproduce:

On the Fedora 15 system I am managing, auditd and sshd are both Sys V init scripts as well as our custom service init scripts.

- hosts: fedora
  sudo: yes

  tasks:
    - service: name=auditd state=restarted enabled=yes

Expected Results:

I would expect the service module to recognize that the service is not managed by systemd and use rc.local or the init script to manage the service.

Actual Results:

TASK: [service name=auditd enabled=yes] ***************************************
<10.17.217.203> ESTABLISH CONNECTION FOR USER: root on PORT 22 TO 10.17.217.203
<10.17.217.203> REMOTE_MODULE service name=auditd enabled=yes
<10.17.217.203> EXEC /bin/sh -c 'mkdir -p $HOME/.ansible/tmp/ansible-tmp-1425923472.38-164975472934968 && chmod a+rx $HOME/.ansible/tmp/ansible-tmp-1425923472.38-164975472934968 && echo $HOME/.ansible/tmp/ansible-tmp-1425923472.38-164975472934968'
<10.17.217.203> PUT /tmp/tmpAPdmFn TO /home/sgadmin/.ansible/tmp/ansible-tmp-1425923472.38-164975472934968/service
<10.17.217.203> EXEC /bin/sh -c 'LANG=C LC_CTYPE=C /usr/bin/python /home/sgadmin/.ansible/tmp/ansible-tmp-1425923472.38-164975472934968/service'
failed: [10.17.217.203] => {"failed": true}
msg: Error when trying to enable auditd: rc=1 Unit name auditd is not a valid unit name.
Cannot install unit auditd: Invalid argument


FATAL: all hosts have already failed -- aborting

[bcoca]

So I don't think the restart is the issue as I've tested it and it wors, its the enabled=yes, which systemd does not support for init scripts, nor does it run rc.local either unless you have a rc.local.service setup and old sysV init is not available because systemd is in place.

I'm not sure what needs to be done here as systemd docs just tell you to create a systemd service file and does not give alternatives.

[nusenu]

Since the title matches the bug I'm seeing I'm adding this info here.

Due to a bug in debian's systemd (is-enabled won't work), ansible will fail to disable legacy sysv services on systemd hosts because it thinks the service is not enabled in the first place.

Debian bug:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=751638

https://groups.google.com/d/msgid/ansible-project/552AD4BF.4060609%40openmailbox.org

[bcoca]

I don't think this is a debian issue either but a systemd one, it does not allow for enabling/disabling sysV init scripts, it can start/stop them but not enable/disable them.

As such I don't see a possible solution ansible can provide, the 'systemd solution' is to create a systemd config file and I'm in no way comfortable in doing that dynamically for a sysV init script.

[bcoca]

so I was wrong above, it seems systemd should be configured by the distribution to fallback to the old sysV init tools to enable/disable init scripts, the bug report linked above also has the updates that fix this issue on debian.

[mpolden]

It's true that this has been fixed in Debian (https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=751638). It still applies to stable though, and I'm guessing the fix is not important enough to be backported.

Maybe Ansible could provide a workaround so that service: name=foo enabled=yes works as expected on Debian stable?

[nusenu]

It's true that this has been fixed in Debian (https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=751638). It still applies to stable though, and I'm guessing the fix is not important enough to be backported. Maybe Ansible could provide a workaround so that `service: name=foo enabled=yes` works as expected on Debian stable?

I guess it makes sense to ask debian devs whether they plan to backport it before asking ansible devs to provide a workaround. You can also implement your own workaround for this issue, example: nusenu/ansible-relayor@efabec4

[Gillingham]

Shouldn't this be re-opened? Especially since as #1387 was closed as a duplicate and this issue was never resolved.

I'm seeing the same thing on a Oracle Linux 7 machine with ansible 1.9.2. I have a service using an old init script which ansible cannot start. Issuing any state to service gets back "Failed to issue method call: Unit server-service.sh.service failed to load: No such file or directory."

If I just do a simple /sbin/service server-service.sh it works fine, same if I call /etc/init.d/server-service.sh directly.

Oddly doing enabled=true alone works fine, it's just the state functionality that fails.

[leoluk]

Please re-open this issue - it's probably not going to be fixed soon (at least) in Debian Stable, so an Ansible workaround would be nice.

[chriswayg]

+1
"Please re-open this issue - it's probably not going to be fixed soon (at least) in Debian Stable, so an Ansible workaround would be nice."

[LearnLinuxTV]

I also agree about reopening this issue, Debian will never include a version of systemd from unstable into their current branch. A work around would be awesome and would be the only way Debian Jessie can be properly supported throughout its life cycle.

[adriankirchner]

+1 for reopening this issue

[mpolden]

+1. I think it's reasonable to expect service to work properly on a stable version of such a popular distro as Debian.

[leoluk]

There's a pretty good summary by the Puppet guys here: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=775795;msg=81

They added a workaround to Puppet, by the way. Ansible should probably do the same.

[canha]

I've been missing this feature a lot. It doesn't make sense that such a core part of Ansible itself doesn't work for one of the father distributions of Linux.

[lxhunter]

+1

[azaghal]

For what it's worth, I have requested fixing this explicitly in Debian Jessie on Debian bug-tracker, not sure much would happen though - https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=809405

My C is a bit rusty/unused, although I was capable of hacking together something to get a custom build of systemd package working. Perhaps with more interest from Debian people and eventually someone with more C experience we could get a similar solution deployed for Jessie, therefore eliminating the need to hack Ansible to work with broken distro implementation.

[mpolden]

@azaghal Great, hope it gets backported seeing as it affects most configuration managements tools, including Chef and Puppet.

[mpolden]

Looks like a workaround was added back in August: 55e8863

Works fine on jessie now (using Ansible 2.0.0.1).

[leoluk]

Nice! Puppet ended up fixing it in their codebase, too.

[Gillingham]

@martinp This issue does not seem to be resolved at least on RHEL based systems, verified with ansible 2.0.0.2 that I still get systemd could not find the requested service when trying to restart a init.d based script.

..$ ansible --version
ansible 2.0.0.2
  config file = .../ansible/ansible.cfg
  configured module search path = Default w/o overrides
(ansible)worktop:ansible gillingh$ ansible ... -m service -a "name=server-service.sh state=restarted"
... | FAILED! => {
    "changed": false,
    "failed": true,
    "msg": "systemd could not find the requested service \"'server-service.sh'\": "
}

$ cat /etc/redhat-release
Red Hat Enterprise Linux Server release 7.2 (Maipo)
$ sudo /etc/init.d/server-service.sh status
Fiorano Messaging Queue server <FioranoMQ_HA_shared_HASecondary-disa> is running (18040).
$ sudo service server-service.sh status
Fiorano Messaging Queue server <FioranoMQ_HA_shared_HASecondary-disa> is running (18040).

[geerlingguy]

I wanted to note that I ran into this bug in my Apache Solr role on RHEL/CentOS 7.x; I had to shim in a task that uses service solr stop prior to using Ansible's service module to control the service via systemd:

geerlingguy/ansible-role-solr@71d5d56

[ghost]

I just encountered this for the first time using Ansible 2.2 on Debian with a Centos 7.2.1511 target host.

 - name: Start / Enable cassandra
   service: name=cassandra
     state=started
     enabled=yes

Leads to:

TASK [cassandra : Start / Enable cassandra] ***************************
fatal: [centoshost]: FAILED! => {"changed": false, "failed": true, "msg": "Could not find the requested service \"'cassandra'\": "}

Then I ran this command on the server

[root@centoshost /]# systemctl enable cassandra
cassandra.service is not a native service, redirecting to /sbin/chkconfig.
Executing /sbin/chkconfig cassandra on

And if I rerun the Ansible script, it works like a charm.
Maybe the service module can run systemctl or chkconfig when it initially fails.

Thx for your support!

[jctanner]

@bbaassssiiee @tkinz27 If you have found a bug in 2.2, please open a new issue. This ticket was originally for ansible 1.9

[Gillingham]

@jctanner the bug has been outstanding since 1.9, a fix has never been included in ansible so it's migrated from 1.9 -> 2.2 as the versions have incremented. Do you want a duplicate bug opened for every version of ansible?

[geerlingguy]

I've also started hitting this again in 2.2, but only for certain cases (where an init script is installed as part of a package—in this case Solr): https://travis-ci.org/geerlingguy/ansible-role-solr/jobs/178318292#L391-L396 — and only on CentOS 7 (not on Debian 8 or Ubuntu 16.04, which have different setups).

Technically it would be best if I could convince Apache Solr to add a unit file upstream, but as that will take some time, it'd be nice to have this working as it was in 2.0.x and 2.1.x :D

Attempting to fix with: geerlingguy/ansible-role-solr@ccb6141 / geerlingguy/ansible-role-solr@7c410fd

Edit: Also ran into the issue with my Gogs role here: geerlingguy/ansible-role-gogs#10

[gadelkareem]

Got this with Ansible 2.2 as well

- name: Copy mailhog init script into place.
  template:
    src: mailhog.init.j2
    dest: /etc/init.d/mailhog
    owner: root
    group: root
    mode: 0755

- name: Ensure mailhog is enabled and will start on boot.
  service: name=mailhog state=started enabled=yes

Using this works

- name: Ensure mailhog is enabled and will start on boot.
  shell: service mailhog restart && chkconfig mailhog on

[barabaka]

The same with Ansible 2.2 and Ubuntu 16.04, init script is: /etc/init.d/zookeeper

• name: Zookeeper | Start Zookeeper
  service: name=zookeeper state=started

TASK [zookeeper : Zookeeper | Start Zookeeper] *********************************
fatal: [*****]: FAILED! => {"changed": false, "failed": true, "msg": "Could not find the requested service "'zookeeper'": "}

[ksonj]

I am also seeing this with Ansible 2.2, Ubuntu 14.04.5 and a manually installed service script. Manually starting with service supervisor start works fine. Ansible reports:

FAILED! => {"changed": false, "failed": true, "msg": "Error loading unit file 'supervisor': org.freedesktop.DBus.Error.FileNotFound \"No such file or directory\""}

The unit file part sounds like systemd to me. systemd does exist on Ubuntu 14.04 but it's not used as the init system as far as I know. My guess is there's something wrong with the detection of the correct init system.

There appears to be a new use setting for the service module to control which service manager to use http://docs.ansible.com/ansible/service_module.html

The possible values are not documented, but I guess it's those listed down from https://github.com/ansible/ansible/blob/ffb80926cec57125e9e0762dafd414f35d1e2d14/lib/ansible/module_utils/facts.py#L373

Even more unfortunately that setting doesn't seem to have any effect, as I'm still seeing systemd errors. Overwriting the fact ansible_service_mgr doesn't seem to do anything as well. Did anyone have any luck with that?

[ksonj]

The only thing that does work is uninstalling systemd

[bcoca]

it seems that the systemd version on Ubuntu 14 has the same problem that older debian systemd versions had, they cannot deal with init scripts correctly, systemd SHOULD be able to do this but the initial debian implementation was buggy in this respect.

so detection IS working correctly, but the systemd on the OS is not capable of doing what it should

[ksonj]

I see. I'd expect to be able to overwrite the detection result with the use keyword, though.
Wouldn't it also be possible to detect the affected versions of ubuntu/debian and systemd on part of ansible and take the approriate action?

[bcoca]

you can, but i have not added a sysvinit service module yet

[bbaassssiiee]

On centos 7 systemd can deal with init scripts, only ansible's service module seems confused that the host can be so hybrid. A 'service this restart' works in a root shell, but '- service: name=this state=restarted' does not.

[bcoca]

'service' is not part of systemd, systemctl is, which is what Ansible uses in case of systemd being detected.

[bbaassssiiee]

Mind you: 'Service Module Fails with Sys V init script on a system that has Systemd' is the title of this bug. If we don't give a str(4) what the system has to launch a process, 'service' could be an nice abstraction if it would be capable to deal with all the differences between init, systemd, launchd, upstart, and whatnot.

[bcoca]

if it could, we would, sadly init systems are all over the place, differences on the systemd implementation alone require many special cases to deal with.

That is why i decided to actually split it so users can use each system as they see fit as the module that tries to handle all cases is becoming unmaintainable.

[axsantos]

@bocca, I'm running a chroot with sysvinit on top of host with jessie. I run my playbooks directly inside the chroot via sshd running on a different port of the host system. My problem is that service module is always detecting systemd.

TASK [Start server] ************************************************************
fatal: [col01-axtract]: FAILED! => {"changed": false, "failed": true, "msg": "No ActiveState value in systemctl show output for 'cron'"}

I took a look into the code and it seems that the code checks "/proc/1/comm" which in a chroot is typically a bind mount to the host system. I tried to use the use param to force it to sysvinit but it does not work.

- hosts: fe-nodes
  gather_facts: False
  pre_tasks:
    - include: common_tasks/install_tmp_ssh_key.yml
  tasks:
    - name: Start server
      service: name=cron state=started use="sysvinit"
  post_tasks:
    - include: common_tasks/remove_tmp_ssh_key.yml

The only valid workaround I found for this was to rename systemctl tool (probably uninstalling systemd will also work, just like @ksonj), but honestly I think the use param should skip the auto-detection mechanism. Am I missing the point of the use parameter?

[bcoca]

@axsantos no, except sysvinit is not implemented yet as it's own module

[tomdeblende]

Having the same issue with ansible 2.1.1.0:

FAILED! => {"changed": false, "failed": true, "msg": "systemd could not find the requested service "'site24x7monagent'": "}

[dmcd]

I'm seeing this too when using the https://github.com/elastic/ansible-elasticsearch playbook, using Ansible 2.2 on RHEL 7.3

- name: restart elasticsearch
  service: name=elasticsearch state=restarted enabled=yes

Leads to:

   elasticsearch: RUNNING HANDLER [roles/elasticsearch : restart elasticsearch] ******************
   elasticsearch: fatal: [127.0.0.1]: FAILED! => {"changed": false, "failed": true, "msg": "Service is in unknown state", "status": {}}
   elasticsearch: to retry, use: --limit @/tmp/packer-provisioner-ansible-local/main.retry
   elasticsearch:
   elasticsearch: PLAY RECAP *********************************************************************
   elasticsearch: 127.0.0.1                  : ok=39   changed=19   unreachable=0    failed=1

Then I ran this command on the server

$ systemctl is-enabled elasticsearch
elasticsearch.service is not a native service, redirecting to /sbin/chkconfig.
Executing /sbin/chkconfig elasticsearch --level=5
enabled

[kakabomba]

+1
- name: Enable logstash service
service:
name: logstash
enabled: yes

ansible: 2.3.0.0
server: debian jessie

[jacobweber]

Help! This is still an issue in Ansible 2.2 with CentOS 7. Is there a workaround? I created ansible/ansible#25440 since this one is still closed.