/etc/init.d/logstash: 99: /etc/init.d/logstash: [[: not found

[abcfy2]

When I use service logstash restart on Ubuntu 14.04 LTS, will raise this error.

Logstash version is 2.2.3, and installed from official repo.

This because Ubuntu does not use sh alias for bash, and sh does not support built-in [[ term.

[andrewvc]

@abcfy2 interesting, I tried this on a fresh 14.04 install. I couldn't repro it, even running the init script with dash and sh directly. Do you have any other relevant info?

I do agree that the init script should probably use /usr/bin/env bash instead

[andrewvc]

@abcfy2 did you perform an upgrade? I'm wondering if you somehow have an old init script.

[abcfy2]

Yes, I do upgrade by using apt-get dist-upgrade. Upgrade logstash from an early 2.2 version

[purbon]

will test now the upgrade with a VM and Ubuntu 14.04.4 LTS (Trusty Tahr), will report here if I'm able to reproduce.

@abcfy2 are you able to reproduce in a fresh installation?

[purbon]

I tested this out of a fresh installation of Ubuntu 14.04.4 LTS (Trusty Tahr), running apt-get update as a first thing, so everything is up to date.

• Install Logstash 2.2 (from deb repository)
• Add a dummy config
• start the service
• stop the service
• update to Logstash 2.3
• start the service

and everything work as expected.

@abcfy2 can you provide probably a few more hints? might be test this in a fresh installation (with all updated)

[abcfy2]

The stop() function code:

stop() {
  # Try a few times to kill TERM the program
  if status ; then
    pid=`cat "$pidfile"`
    echo "Killing $name (pid $pid) with SIGTERM"
    kill -TERM $pid
    # Wait for it to exit.
    for i in 1 2 3 4 5 6 7 8 9 ; do
      echo "Waiting $name (pid $pid) to die..."
      status || break
      sleep 1
    done
    if status ; then
      if [[ $KILL_ON_STOP_TIMEOUT -eq 1 ]] ; then
        echo "Timeout reached. Killing $name (pid $pid) with SIGKILL. This may result in data loss."
        kill -KILL $pid
        echo "$name killed with SIGKILL."
      else
        echo "$name stop failed; still running."
      fi
    else
      echo "$name stopped."
    fi
  fi
}

Only timeout reached could get this error.

[abcfy2]

I find when I stop the elasticsearch, will easy to reproduce this issue.

sinoiot@sinoiot-172-16-250-3:~$ sudo service redis-server start
Starting redis-server: redis-server.
sinoiot@sinoiot-172-16-250-3:~$ sudo service elasticsearch stop
 * Stopping Elasticsearch Server                                                                                                                                                      [ OK ] 
sinoiot@sinoiot-172-16-250-3:~$ sudo service logstash restart
Killing logstash (pid 21829) with SIGTERM
Waiting logstash (pid 21829) to die...
Waiting logstash (pid 21829) to die...
Waiting logstash (pid 21829) to die...
Waiting logstash (pid 21829) to die...
Waiting logstash (pid 21829) to die...
Waiting logstash (pid 21829) to die...
Waiting logstash (pid 21829) to die...
Waiting logstash (pid 21829) to die...
Waiting logstash (pid 21829) to die...
/etc/init.d/logstash: 99: /etc/init.d/logstash: [[: not found
logstash stop failed; still running.
logstash started.

Maybe better to use upstart script or systemd script in Ubuntu. Because both upstart and systemd have the built-in way to force a service stop.

[ThomasdOtreppe]

I got the same issue after updating. Replacing the double brackets with a single one on each side (one opening and one closing) fixes the issue.

[abcfy2]

Yes, that's right. Because Ubuntu does not use sh -> bash, while CentOS/RHEL does.

[[ is a bash built-in command, but sh does not support. So replace [[ ]] to [ ] could fix this issue.

But I wonder why logstash/elasticsearch/kibana not use upstart or systemd?

The init.d script not do well with the stop() function. I often meet that when I want logstash to restart, but start two processes(the old process stop failed, while start a new one).

sudo service logstash restart
Killing logstash (pid 21829) with SIGTERM
Waiting logstash (pid 21829) to die...
Waiting logstash (pid 21829) to die...
Waiting logstash (pid 21829) to die...
Waiting logstash (pid 21829) to die...
Waiting logstash (pid 21829) to die...
Waiting logstash (pid 21829) to die...
Waiting logstash (pid 21829) to die...
Waiting logstash (pid 21829) to die...
Waiting logstash (pid 21829) to die...
logstash stop failed; still running.
logstash started.

But both upstart and systemd could handle this problem easily. For example, upstart has a built-in kill timeout stanza that could force kill a service if still running after timeout.

[purbon]

@abcfy2 @ThomasdOtreppe thanks a lot for the help (#4940 (comment)) here, this shows that I'm not definetly a packaging wizard. I was able to reproduce the issue and will work toward a fix for this. Much appreciate.

To also answer your questions about upstart and systemd isn't init.d much more compatible cross distros and versions than the first two? At my understanding it is, what would be in your thoughts the best way to handle this cross versions?

[abcfy2]

Maybe prepare 3 scripts for logstash is better. CentOS 7 or later, Ubuntu 16.04 or later, Debian 8 or later use systemd as default service manager. CentOS 6 and Ubuntu 12.04/14.04 support upstart. Others that not support systemd and upstart could use init.d as the last choice.

In Ubuntu 14.04(I'm not sure about Ubuntu 12.04), if both upstart and init.d script provide, the upstart will be used prefer.

[purbon]

Fixed in #4992

[purbon]

@abcfy2 would you mind opening an issue with this proposal, please include benefits of doing it 👍 thanks a lot for your time and contribution!

[abcfy2]

OK. New issue about the service script: #5012