New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unable to access 'path.data' (/var/lib/elasticsearch/elasticsearch) #69

Closed
a1exus opened this Issue Sep 26, 2016 · 3 comments

Comments

Projects
None yet
2 participants
@a1exus

a1exus commented Sep 26, 2016

I'm trying to follow Snapshot And Restore | Elasticsearch Reference [2.4] | Elastic, and in order to proceed, I have to define path.repo inside of elasticsearch.yml, like this:

# grep path.repo elasticsearch.yml 
path.repo: /run/elasticsearch
# 

I added 3 following lines in addition to existing lines of docker-compose.yml:

# tail -3 docker-compose.yml 
  container_name: elk
  volumes:
    - elk-data:/var/lib/elasticsearch
# 

then:

# docker-compose up -d
Creating elk
# docker-compose ps
         Name                   Command                   State                    Ports          
-------------------------------------------------------------------------------------------------
elk                      /usr/local/bin/start.s   Up                       0.0.0.0:5000->5000/tcp 
                         h                                                 , 0.0.0.0:5044->5044/t 
                                                                           cp, 0.0.0.0:5601->5601 
                                                                           /tcp, 0.0.0.0:9200->92 
                                                                           00/tcp, 9300/tcp       
# docker volume ls
DRIVER              VOLUME NAME
local               elk-data
local               643a8d459d5c81062f41129dc46c7b0e4ba1b064a2ba7c6eb451f38575bb433e
# 

yet, I'm unable to connect to elasticsearch (localhost:9200)

# curl 0:9200
curl: (56) Recv failure: Connection reset by peer
# 

last 21 lines of /var/log/elasticsearch/elasticsearch.log:

# docker exec -it elk tail -21 /var/log/elasticsearch/elasticsearch.log 
[2016-09-26 19:00:26,927][ERROR][bootstrap                ] Exception
java.lang.IllegalStateException: Unable to access 'path.data' (/var/lib/elasticsearch/elasticsearch)
    at org.elasticsearch.bootstrap.Security.addPath(Security.java:316)
    at org.elasticsearch.bootstrap.Security.addFilePermissions(Security.java:259)
    at org.elasticsearch.bootstrap.Security.createPermissions(Security.java:212)
    at org.elasticsearch.bootstrap.Security.configure(Security.java:118)
    at org.elasticsearch.bootstrap.Bootstrap.setupSecurity(Bootstrap.java:212)
    at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:183)
    at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:286)
    at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:35)
Caused by: java.nio.file.AccessDeniedException: /var/lib/elasticsearch/elasticsearch
    at sun.nio.fs.UnixException.translateToIOException(UnixException.java:84)
    at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
    at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
    at sun.nio.fs.UnixFileSystemProvider.createDirectory(UnixFileSystemProvider.java:384)
    at java.nio.file.Files.createDirectory(Files.java:674)
    at java.nio.file.Files.createAndCheckIsDirectory(Files.java:781)
    at java.nio.file.Files.createDirectories(Files.java:767)
    at org.elasticsearch.bootstrap.Security.ensureDirectoryExists(Security.java:337)
    at org.elasticsearch.bootstrap.Security.addPath(Security.java:314)
    ... 7 more
# 

however, if I follow Persisting log data and run it like this:

docker run -p 5601:5601 -p 9200:9200 -p 5044:5044 -p 5000:5000 -v elk-data:/var/lib/elasticsearch --name elk sebp/elk

Everything works as expect it!

# docker version
Client:
 Version:         1.10.3
 API version:     1.22
 Package version: docker-common-1.10.3-46.el7.14.x86_64
 Go version:      go1.6.2
 Git commit:      5206701-unsupported
 Built:           Mon Aug 29 14:00:01 2016
 OS/Arch:         linux/amd64

Server:
 Version:         1.10.3
 API version:     1.22
 Package version: docker-common-1.10.3-46.el7.14.x86_64
 Go version:      go1.6.2
 Git commit:      5206701-unsupported
 Built:           Mon Aug 29 14:00:01 2016
 OS/Arch:         linux/amd64
# docker-compose version
docker-compose version 1.8.0, build f3628c7
docker-py version: 1.9.0
CPython version: 2.7.9
OpenSSL version: OpenSSL 1.0.1e 11 Feb 2013
# cat /etc/redhat-release 
Red Hat Enterprise Linux Server release 7.2 (Maipo)
# rpm -q docker
docker-1.10.3-46.el7.14.x86_64
# 

Please advice.
Thank you!

@spujadas

This comment has been minimized.

Show comment
Hide comment
@spujadas

spujadas Sep 26, 2016

Owner

Unfortunately I can't seem to reproduce your issue: your set-up is working absolutely fine on my side, both with docker-compose and docker, and both by creating the elk-data volume manually (docker volume create --name elk-data) and by letting docker* create the volume themselves.

You could try to destroy the volume and all the containers, and then try again.

Additionally, if the volume is bind-mounted to a directory on the host, you may want to check the permissions (which is what Elasticsearch appears to be complaining about according to your logs): the (mounted) directory on the host should be writable by Elasticsearch (UID and GID 991).

Hope that helps, because other than that, nothing springs to mind. 😕

Owner

spujadas commented Sep 26, 2016

Unfortunately I can't seem to reproduce your issue: your set-up is working absolutely fine on my side, both with docker-compose and docker, and both by creating the elk-data volume manually (docker volume create --name elk-data) and by letting docker* create the volume themselves.

You could try to destroy the volume and all the containers, and then try again.

Additionally, if the volume is bind-mounted to a directory on the host, you may want to check the permissions (which is what Elasticsearch appears to be complaining about according to your logs): the (mounted) directory on the host should be writable by Elasticsearch (UID and GID 991).

Hope that helps, because other than that, nothing springs to mind. 😕

@a1exus

This comment has been minimized.

Show comment
Hide comment
@a1exus

a1exus Sep 27, 2016

SELinux was causing this behavior, as soon as I disabled it issue went away...

# getenforce 
Enforcing
# setenforce 0
# getenforce 
Permissive
#

a1exus commented Sep 27, 2016

SELinux was causing this behavior, as soon as I disabled it issue went away...

# getenforce 
Enforcing
# setenforce 0
# getenforce 
Permissive
#

@a1exus a1exus closed this Sep 27, 2016

@spujadas

This comment has been minimized.

Show comment
Hide comment
@spujadas

spujadas Sep 27, 2016

Owner

How unexpected!
Thanks for the update, will make a note to update the documentation to add that in.

Owner

spujadas commented Sep 27, 2016

How unexpected!
Thanks for the update, will make a note to update the documentation to add that in.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment