Lack of systemd init files in deb package

[abravorus]

Hello,

Is it possible to add files which are needed to start the service with systemd in elasticsearch deb package?
This will allow to start "normally" elasticsearch under latest Debian-based Linux distributions which switched recently from system-V to systemd init system. As I understand such files (mostly elasticsearch.service) are included already in RPM packages and they have to be just slightly modified.

Thanks,
Alexandr

[abravorus]

Hello,

I prepared a set of files (based on files in RPM package) needed to start elasticsearch with systemd under Debian jessie (testing) and was able to start it normally.

1. /etc/systemd/system/elasticsearch.service

[Unit]
Description=Starts and stops a single elasticsearch instance on this system
Documentation=http://www.elasticsearch.org

[Service]
Type=forking
EnvironmentFile=/etc/default/elasticsearch
User=elasticsearch
Group=elasticsearch
PIDFile=/var/run/elasticsearch/elasticsearch.pid
ExecStart=/usr/share/elasticsearch/bin/elasticsearch -d -p /var/run/elasticsearch/elasticsearch.pid -Des.default.config=$CONF_FILE -Des.default.path.home=$ES_HOME -Des.default.path.logs=$LOG_DIR -Des.default.path.data=$DATA_DIR -Des.default.path.work=$WORK_DIR -Des.default.path.conf=$CONF_DIR
# See MAX_OPEN_FILES in /etc/default/elasticsearch
LimitNOFILE=65535
# See MAX_LOCKED_MEMORY in /etc/default/elasticsearch, use "infinity" when     MAX_LOCKED_MEMORY=unlimited and using bootstrap.mlockall: true
#LimitMEMLOCK=infinity
# Shutdown delay in seconds, before process is tried to be killed with KILL (if configured)
TimeoutStopSec=20

[Install]
WantedBy=multi-user.target

2. /usr/lib/sysctl.d/elasticsearch.conf

vm.max_map_count=262144

3. /usr/lib/tmpfiles.d/elasticsearch.conf

d    /run/elasticsearch   0755 elasticsearch elasticsearch - -

You should also check the content of /etc/default/elasticsearch for default values for your configuration, for example to change ES_HEAP_SIZE value.

You also need to make all systemd exercises ones (as root or via sudo):

systemd-tmpfiles --create elasticsearch.conf (to created a folder with proper permissions for PID file)
systemctl daemon-reload
systemctl enable elasticsearch.service

After that elasticsearch should start automatically after reboot or you can start it manually via:
systemctl start elasticsearch.service
or
/etc/init.d/elasticsearch start (which actually run the previous command)

Of course, I'm not sure that all above is errors free but it works for me.

Alexandr

[t-lo]

Debian stable ("Wheezy") still uses sysv-init; "Jessie" is still in "testing" (i.e. not released), and thusly will not receive timely security updates until its release in 2015 (see https://www.debian.org/releases/jessie/).

So, does elasticsearch support .deb packages for "testing" (i.e. unreleased) versions?

[abravorus]

Hi,

I can just say that in my experience official elasticsearch .deb package
can be installed without any problems on Debian testing ("Jessie") and
could be run on it smootly. The only question is that "Jessie" switched
already to systemd (instead of sysv-init) and I had to prepare
elasticsearch systemd config files by myself.

I have quite a simple elasticsearch cluster with two non-heavily-loaded
nodes both running under Debian "Jessie". Upgrade from ES 1.4.0 to 1.4.1
was done successfully by standard apt-get update/apt-get upgrade procedure
(having ES systemd files in place already, of course).

Security updates for "Jessie" are done via the following repository record
in /etc/apt/sources.list:
deb http://security.debian.org/ jessie/updates main non-free contrib

There are only 4 weeks till 2015, time goes fast :)

Alexandr

On Wed, Dec 3, 2014 at 1:22 PM, Thilo Fromm notifications@github.com
wrote:

Debian stable ("Wheezy") still uses sysv-init; "Jessie" is still in
"testing" (i.e. not released), and thusly will not receive timely security
updates until its release in 2015 (see
https://www.debian.org/releases/jessie/).

So, does elasticsearch support .deb packages for "testing" (i.e.
unreleased) versions?

—
Reply to this email directly or view it on GitHub
#8493 (comment)
.

[t-lo]

@abravorus Is there a generic way to build a .deb package which provides both a .service file for systemd and a sysv-script, and only installs the correct ones (i.e. init script on Deb < 8, and service file on Deb >= 8)? Or do you think it would be best to provide a .deb w/ init script for Debian < 8, and a separate .deb for Debian >= 8?

Concerning sec updates, quoting from https://www.debian.org/releases/jessie/:

Please note that security updates for testing distribution are not yet managed by the security team.
Hence, testing does not get security updates in a timely manner. You are encouraged to switch
your sources.list entries from testing to wheezy for the time being if you need security support. See
also the entry in the Security Team's FAQ for the testing distribution.

[abravorus]

Hi Thilo,

I do not know is it possible to build such a single .deb package. May be
possible if it will look on installed packages in pre-install phase, but I
never did it by myself. I think that more simple and robust way is to build
two separate packages for Debian < 8 and for Debian >= 8.

Keeping in mind that some other Linux distros are switched already to
systemd and ES has already systemd support for RPM there is a sense to be
prepared for that for Debian too.

You are right about security updates for Debian testing. I've just noted
that they are available in principle.

Alexandr

PS: Looks like GitHub web-site is blocked in Russia, I can answer via
e-mail only.

On Wed, Dec 3, 2014 at 2:37 PM, Thilo Fromm notifications@github.com
wrote:

@abravorus https://github.com/abravorus Is there a generic way to build
a .deb package which provides both a .service file for systemd and a
sysv-script, and only installs the correct ones (i.e. init script on Deb <
8, and service file on Deb >= 8)? Or do you think it would be best to
provide a .deb w/ init script for Debian < 8, and a separate .deb for
Debian >= 8?

Concerning sec updates, quoting from
https://www.debian.org/releases/jessie/:

Please note that security updates for testing distribution are not yet
managed by the security team.
Hence, testing does not get security updates in a timely manner. You are
encouraged to switch
your sources.list entries from testing to wheezy for the time being if you
need security support. See
also the entry in the Security Team's FAQ for the testing distribution.

—
Reply to this email directly or view it on GitHub
#8493 (comment)
.

[t-lo]

Ping @clintongormley does ES want two separate .debs, one for systemd, one for sysv-init?

[t-lo]

@abravorus Currently there are some improvements to the ES .service file in the works. Most notably the ES start script doesn't daemonize (so backgrounding is left to systemd), and we use After=network-online.target to wait for networking before starting ES. I checked with Fedora 21 and it behaves as expected.
Could you please test the new service file w/ Debian 8?

[Unit]
Description=Starts and stops a single elasticsearch instance on this system
Documentation=http://www.elasticsearch.org
Wants=network-online.target
After=network-online.target

[Service]
EnvironmentFile=/etc/sysconfig/elasticsearch
User=elasticsearch
Group=elasticsearch
ExecStart=/usr/share/elasticsearch/bin/elasticsearch            \
                            -Des.default.config=$CONF_FILE      \
                            -Des.default.path.home=$ES_HOME     \
                            -Des.default.path.logs=$LOG_DIR     \
                            -Des.default.path.data=$DATA_DIR    \
                            -Des.default.path.work=$WORK_DIR    \
                            -Des.default.path.conf=$CONF_DIR
# See MAX_OPEN_FILES in sysconfig
LimitNOFILE=65535
# See MAX_LOCKED_MEMORY in sysconfig, use "infinity" when MAX_LOCKED_MEMORY=unlimited and using bootstrap.mlockall: true
#LimitMEMLOCK=infinity
# Shutdown delay in seconds, before process is tried to be killed with KILL (if configured)
TimeoutStopSec=20

[Install]
WantedBy=multi-user.target

Regards,
Thilo

[clintongormley]

@t-lo We'd definitely want to provide just a single .deb package, so supporting both in one would be awesome.

[abravorus]

Hi Thilo,

I will test new .service file tomorrow and let you know.

One note: There is no /etc/sysconfig folder in Debian (both <8 and >=8), I
use /etc/default/elasticsearch instead. Or this folder should be created by
.deb , but I do not see that now.

Alexandr

On Wed, Dec 3, 2014 at 4:10 PM, Thilo Fromm notifications@github.com
wrote:

@abravorus https://github.com/abravorus Currently there are some
improvements to the ES .service file in the works. Most notably the ES
start script doesn't daemonize (so backgrounding is left to systemd), and
we use After=network-online.target to wait for networking before starting
ES. I checked with Fedora 21 and it behaves as expected.
Could you please test the new service file w/ Debian 8?

[Unit]
Description=Starts and stops a single elasticsearch instance on this system
Documentation=http://www.elasticsearch.org
Wants=network-online.target
After=network-online.target

[Service]
EnvironmentFile=/etc/sysconfig/elasticsearch
User=elasticsearch
Group=elasticsearch
ExecStart=/usr/share/elasticsearch/bin/elasticsearch
-Des.default.config=$CONF_FILE
-Des.default.path.home=$ES_HOME
-Des.default.path.logs=$LOG_DIR
-Des.default.path.data=$DATA_DIR
-Des.default.path.work=$WORK_DIR
-Des.default.path.conf=$CONF_DIR

See MAX_OPEN_FILES in sysconfig

LimitNOFILE=65535

See MAX_LOCKED_MEMORY in sysconfig, use "infinity" when MAX_LOCKED_MEMORY=unlimited and using bootstrap.mlockall: true

#LimitMEMLOCK=infinity

Shutdown delay in seconds, before process is tried to be killed with KILL (if configured)

TimeoutStopSec=20

[Install]
WantedBy=multi-user.target

Regards,
Thilo

—
Reply to this email directly or view it on GitHub
#8493 (comment)
.

[electrical]

@abravorus the /etc/default/elasticsearch file should be created by the package. weird that it doesn't in your case.

[t-lo]

@electrical I think @abravorus pointed out a problematic line in my (RPM specific) .service file: EnvironmentFile=/etc/sysconfig/elasticsearch. And I think he's right - for Debianesque systems this needs to be EnvironmentFile=/etc/default/elasticsearch.

[abravorus]

Richard, it's created, no problem. But proposed .service file uses
/etc/sysconfig/elasticsearch
file which doesn't exists in .deb . And I use, in my handmade .service
file /etc/default/elasticsearch
instead.

Alexandr

On Wed, Dec 3, 2014 at 4:24 PM, Richard Pijnenburg <notifications@github.com

wrote:

@abravorus https://github.com/abravorus the /etc/default/elasticsearch
file should be created by the package. weird that it doesn't in your case.

—
Reply to this email directly or view it on GitHub
#8493 (comment)
.

[electrical]

Ahh i see. sorry. yeah, we will need to create a separate service file indeed.

[abravorus]

Thilo, exactly. Probably this is due to the difference in RPM and .deb
folder structure.

Alexandr

On Wed, Dec 3, 2014 at 4:34 PM, Thilo Fromm notifications@github.com
wrote:

@electrical https://github.com/electrical I think @abravorus
https://github.com/abravorus pointed out a problematic line in the (RPM
specific) .service file: EnvironmentFile=/etc/sysconfig/elasticsearch.
And I think he's right - for Debianesque systems this needs to be
EnvironmentFile=/etc/default/elasticsearch.

—
Reply to this email directly or view it on GitHub
#8493 (comment)
.

[t-lo]

@abravorus thanks for pointing this out; I missed that :)

[abravorus]

Hi Thilo,

I checked your .service file on one of my ES nodes under Debian 8. It
works, service starts/stops as expected.
But I didn't check it with full machine reboot because I can't do it now
(and probably two weeks more). If this is not urgently I will let you know
about full reboot test as soon as I'll be able to do it.

Alexandr

On Wed, Dec 3, 2014 at 4:10 PM, Thilo Fromm notifications@github.com
wrote:

@abravorus https://github.com/abravorus Currently there are some
improvements to the ES .service file in the works. Most notably the ES
start script doesn't daemonize (so backgrounding is left to systemd), and
we use After=network-online.target to wait for networking before starting
ES. I checked with Fedora 21 and it behaves as expected.
Could you please test the new service file w/ Debian 8?

[Unit]
Description=Starts and stops a single elasticsearch instance on this system
Documentation=http://www.elasticsearch.org
Wants=network-online.target
After=network-online.target

[Service]
EnvironmentFile=/etc/sysconfig/elasticsearch
User=elasticsearch
Group=elasticsearch
ExecStart=/usr/share/elasticsearch/bin/elasticsearch
-Des.default.config=$CONF_FILE
-Des.default.path.home=$ES_HOME
-Des.default.path.logs=$LOG_DIR
-Des.default.path.data=$DATA_DIR
-Des.default.path.work=$WORK_DIR
-Des.default.path.conf=$CONF_DIR

See MAX_OPEN_FILES in sysconfig

LimitNOFILE=65535

See MAX_LOCKED_MEMORY in sysconfig, use "infinity" when MAX_LOCKED_MEMORY=unlimited and using bootstrap.mlockall: true

#LimitMEMLOCK=infinity

Shutdown delay in seconds, before process is tried to be killed with KILL (if configured)

TimeoutStopSec=20

[Install]
WantedBy=multi-user.target

Regards,
Thilo

—
Reply to this email directly or view it on GitHub
#8493 (comment)
.

[t-lo]

@abravorus Thanks for testing!

[jpountz]

Fixed via #8765

[abravorus]

Great, thanks!

Alexandr

On Fri, Dec 5, 2014 at 2:30 PM, Adrien Grand notifications@github.com
wrote:

Closed #8493 #8493.

—
Reply to this email directly or view it on GitHub
#8493 (comment)
.

[abravorus]

Hi Thilo,

Just to let you know.
.service file works as expected during full machine reboot too. Thanks to our electricians for the occasional 4-hours blackout :)

Alexandr

[t-lo]

@abravorus Heh :D Thanks again for testing!