Elasticsearch startup failure / missing symlink: java.io.IOException: Could not create directory /usr/share/elasticsearch/logs

[minfrin]

Elasticsearch version (bin/elasticsearch --version):

Version: 6.0.0, Build: 8f0685b/2017-11-10T18:41:22.859Z, JVM: 1.8.0_151

Plugins installed: []

JVM version (java -version):

openjdk version "1.8.0_151"
OpenJDK Runtime Environment (build 1.8.0_151-8u151-b12-0ubuntu0.16.04.2-b12)
OpenJDK 64-Bit Server VM (build 25.151-b12, mixed mode)

OS version (uname -a if on a Unix-like system):

Linux elastic01 4.4.0-1041-aws #50-Ubuntu SMP Wed Nov 15 22:18:17 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

Description of the problem including expected versus actual behavior:

Steps to reproduce:

• Install elasticsearch v6.0.0 from the debian package on an Ubuntu Xenial machine.

• Start elasticsearch.

• Elasticsearch fails to start with the log message below:

Provide logs (if relevant):

Nov 28 13:47:32 elastic01 elasticsearch[2231]: 2017-11-28 13:47:32,103 main ERROR Unable to create file /usr/share/elasticsearch/logs/black_index_indexing_slowlog.log java.io.IOException: Could not create directory /usr/share/elasticsearch/logs

The package seems to have a missing symbolic link pointing /usr/share/elasticsearch/logs at /var/log/elasticsearch.

ln -s /var/log/elasticsearch/ /usr/share/elasticsearch/logs

With the symlink in place as follows a new error occurs:

java.lang.IllegalStateException: Unable to access 'path.data' (/usr/share/elasticsearch/data)

ln -s /var/lib/elasticsearch /usr/share/elasticsearch/data

With the symlink in place, the server starts.

[jasontedor]

When you install Elasticsearch via the Debian or RPM package, we have intended for Elasticsearch to be run as a service only using Sys V init or systemd. You are attempting to start Elasticsearch using bin/elasticsearch directly for which the package is not configured. That leads to the problem that you are experiencing. Instead, use the service or reinstall from an archive distribution.

[minfrin]

I am starting elasticsearch using systemd as follows:

systemctl start elasticsearch

[jasontedor]

Something is wrong with your configuration, please check your elasticsearch.yml that you have path.logs set to /var/log/elasticsearch. It is simply not the case that a clean install will try to log to /usr/share/elasticsearch/logs.

[minfrin]

I followed the instructions at https://www.elastic.co/guide/en/elasticsearch/reference/current/deb.html, there is no mention of the need for any special configuration to put things in the right place.

I see what happened here - I had used an elasticsearch.yaml file from a previous attempt at installation, and this one didn't contain the values for path.data and path.logs that are hardcoded into the default file here:

https://github.com/elastic/elasticsearch/blob/master/distribution/src/main/resources/config/elasticsearch.yml

If the symlinks are created as per this ticket, these two config lines are no longer necessary.

[jasontedor]

I don't think you should use symlinks here, instead set the configuration for path.data and path.logs.

[ArSa-Apps]

Hey there, trying to start an instance, which a colleague has configured, but get the same error.
ERROR RollingFileManager (${sys:es.logs}.log) java.io.FileNotFoundException: ${sys:es.logs}.log (Permission denied
Elasticsearch 6.2.2
"path.logs" and "path.date" are been defined in /etc/elasticsearch/elasticsearch.yml and in systemd-script is already an environment variable "Environment=ES_PATH_CONF=/etc/elasticsearch"
User "elasticsearch" owns the logs- and index-folders and has the group ownership on /etc/elasticsearch.

[ArSa-Apps]

Hey there, trying to start an instance, which a colleague has configured, but get the same error.
ERROR RollingFileManager (${sys:es.logs}.log) java.io.FileNotFoundException: ${sys:es.logs}.log (Permission denied
Elasticsearch 6.2.2
"path.logs" and "path.date" are been defined in /etc/elasticsearch/elasticsearch.yml and in systemd-script is already an environment variable "Environment=ES_PATH_CONF=/etc/elasticsearch"
User "elasticsearch" owns the logs- and index-folders and has the group ownership on /etc/elasticsearch.

Ok, I got it. I should've used ${sys:es.logs.base_path}... now is working!

[bhartishradha]

Hi
i am also getting the same error
and we have not set any path to log also
could you tell me how to fix this