New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fail to start on linux #1

Closed
ericlemerdy opened this Issue Apr 5, 2017 · 1 comment

Comments

Projects
None yet
1 participant
@ericlemerdy

ericlemerdy commented Apr 5, 2017

Hello,

Thank you again and @Gregon for your Devoxx session at #DevoxxFR this morning.

We experienced some trouble with the docker image you provided. Let me show you the logs:

Output from console:

$ docker run -p 9200:9200 -p 5601:5601 ibeauvais/elasticsearch-kibana:5.2
/usr/lib/python2.7/site-packages/supervisor/options.py:296: UserWarning: Supervisord is running as root and it is searching for its configuration file in default locations (including its current working directory); you probably want to specify a "-c" argument specifying an absolute path to a configuration file for improved security.
  'Supervisord is running as root and it is searching '
2017-04-05 20:44:46,658 CRIT Supervisor running as root (no user in config file)
2017-04-05 20:44:46,661 INFO supervisord started with pid 1
2017-04-05 20:44:47,663 INFO spawned: 'elasticsearch' with pid 9
2017-04-05 20:44:47,668 INFO spawned: 'kibana' with pid 10
2017-04-05 20:44:48,669 INFO success: elasticsearch entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2017-04-05 20:44:48,669 INFO success: kibana entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2017-04-05 20:44:52,304 INFO exited: elasticsearch (exit status 78; not expected)
2017-04-05 20:44:53,307 INFO spawned: 'elasticsearch' with pid 103
2017-04-05 20:44:54,341 INFO success: elasticsearch entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2017-04-05 20:44:56,669 INFO exited: elasticsearch (exit status 78; not expected)
2017-04-05 20:44:57,671 INFO spawned: 'elasticsearch' with pid 186
2017-04-05 20:44:58,673 INFO success: elasticsearch entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2017-04-05 20:45:01,457 INFO exited: elasticsearch (exit status 78; not expected)
2017-04-05 20:45:01,523 INFO spawned: 'elasticsearch' with pid 269
...

After entering the running container, with :
$ docker exec -i -t xenodochial_bassi /bin/bash

We can see the standard output of elasticsearch and kibana processes as captured by supervisord in /tmp:

bash-4.3# head -n 30 /tmp/elasticsearch-stdout---supervisor-oxnBUU.log
[2017-04-05T20:44:49,149][INFO ][o.e.n.Node               ] [] initializing ...
[2017-04-05T20:44:49,221][INFO ][o.e.e.NodeEnvironment    ] [frrGrAX] using [1] data paths, mounts [[/usr/share/elasticsearch/data (/dev/sda5)]], net usable_space [111gb], net total_space [163.6gb], spins? [possibly], types [ext4]
[2017-04-05T20:44:49,221][INFO ][o.e.e.NodeEnvironment    ] [frrGrAX] heap size [1.9gb], compressed ordinary object pointers [true]
[2017-04-05T20:44:49,222][INFO ][o.e.n.Node               ] node name [frrGrAX] derived from node ID [frrGrAXfSQOJU7JUUBc_mA]; set [node.name] to override
[2017-04-05T20:44:49,224][INFO ][o.e.n.Node               ] version[5.2.2], pid[9], build[f9d9b74/2017-02-24T17:26:45.835Z], OS[Linux/3.16.0-4-amd64/amd64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/1.8.0_111-internal/25.111-b14]
[2017-04-05T20:44:50,079][INFO ][o.e.p.PluginsService     ] [frrGrAX] loaded module [aggs-matrix-stats]
[2017-04-05T20:44:50,079][INFO ][o.e.p.PluginsService     ] [frrGrAX] loaded module [ingest-common]
[2017-04-05T20:44:50,079][INFO ][o.e.p.PluginsService     ] [frrGrAX] loaded module [lang-expression]
[2017-04-05T20:44:50,079][INFO ][o.e.p.PluginsService     ] [frrGrAX] loaded module [lang-groovy]
[2017-04-05T20:44:50,079][INFO ][o.e.p.PluginsService     ] [frrGrAX] loaded module [lang-mustache]
[2017-04-05T20:44:50,079][INFO ][o.e.p.PluginsService     ] [frrGrAX] loaded module [lang-painless]
[2017-04-05T20:44:50,080][INFO ][o.e.p.PluginsService     ] [frrGrAX] loaded module [percolator]
[2017-04-05T20:44:50,080][INFO ][o.e.p.PluginsService     ] [frrGrAX] loaded module [reindex]
[2017-04-05T20:44:50,080][INFO ][o.e.p.PluginsService     ] [frrGrAX] loaded module [transport-netty3]
[2017-04-05T20:44:50,080][INFO ][o.e.p.PluginsService     ] [frrGrAX] loaded module [transport-netty4]
[2017-04-05T20:44:50,080][INFO ][o.e.p.PluginsService     ] [frrGrAX] no plugins loaded
[2017-04-05T20:44:52,054][INFO ][o.e.n.Node               ] initialized
[2017-04-05T20:44:52,055][INFO ][o.e.n.Node               ] [frrGrAX] starting ...
[2017-04-05T20:44:52,128][WARN ][i.n.u.i.MacAddressUtil   ] Failed to find a usable hardware address from the network interfaces; using random bytes: ad:04:52:4e:c6:05:fc:ff
[2017-04-05T20:44:52,169][INFO ][o.e.t.TransportService   ] [frrGrAX] publish_address {172.17.0.2:9300}, bound_addresses {[::]:9300}
[2017-04-05T20:44:52,173][INFO ][o.e.b.BootstrapChecks    ] [frrGrAX] bound or publishing to a non-loopback or non-link-local address, enforcing bootstrap checks
ERROR: bootstrap checks failed
max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
[2017-04-05T20:44:52,215][INFO ][o.e.n.Node               ] [frrGrAX] stopping ...
[2017-04-05T20:44:52,234][INFO ][o.e.n.Node               ] [frrGrAX] stopped
[2017-04-05T20:44:52,235][INFO ][o.e.n.Node               ] [frrGrAX] closing ...
[2017-04-05T20:44:52,243][INFO ][o.e.n.Node               ] [frrGrAX] closed
[2017-04-05T20:44:54,340][INFO ][o.e.n.Node               ] [] initializing ...
[2017-04-05T20:44:54,404][INFO ][o.e.e.NodeEnvironment    ] [frrGrAX] using [1] data paths, mounts [[/usr/share/elasticsearch/data (/dev/sda5)]], net usable_space [111gb], net total_space [163.6gb], spins? [possibly], types [ext4]
[2017-04-05T20:44:54,405][INFO ][o.e.e.NodeEnvironment    ] [frrGrAX] heap size [1.9gb], compressed ordinary object pointers [true]
bash-4.3# head -n 15 /tmp/kibana-stdout---supervisor-60Fm22.log 
{"type":"log","@timestamp":"2017-04-05T20:44:51Z","tags":["status","plugin:kibana@5.2.2","info"],"pid":10,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
{"type":"log","@timestamp":"2017-04-05T20:44:51Z","tags":["status","plugin:elasticsearch@5.2.2","info"],"pid":10,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
{"type":"log","@timestamp":"2017-04-05T20:44:51Z","tags":["error","elasticsearch","admin"],"pid":10,"message":"Request error, retrying\nHEAD http://localhost:9200/ => connect ECONNREFUSED 127.0.0.1:9200"}
{"type":"log","@timestamp":"2017-04-05T20:44:51Z","tags":["warning","elasticsearch","admin"],"pid":10,"message":"Unable to revive connection: http://localhost:9200/"}
{"type":"log","@timestamp":"2017-04-05T20:44:51Z","tags":["warning","elasticsearch","admin"],"pid":10,"message":"No living connections"}
{"type":"log","@timestamp":"2017-04-05T20:44:51Z","tags":["status","plugin:console@5.2.2","info"],"pid":10,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
{"type":"log","@timestamp":"2017-04-05T20:44:51Z","tags":["status","plugin:elasticsearch@5.2.2","error"],"pid":10,"state":"red","message":"Status changed from yellow to red - Unable to connect to Elasticsearch at http://localhost:9200.","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
{"type":"log","@timestamp":"2017-04-05T20:44:51Z","tags":["status","plugin:timelion@5.2.2","info"],"pid":10,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
{"type":"log","@timestamp":"2017-04-05T20:44:51Z","tags":["listening","info"],"pid":10,"message":"Server running at http://0.0.0.0:5601"}
{"type":"log","@timestamp":"2017-04-05T20:44:51Z","tags":["status","ui settings","error"],"pid":10,"state":"red","message":"Status changed from uninitialized to red - Elasticsearch plugin is red","prevState":"uninitialized","prevMsg":"uninitialized"}
{"type":"log","@timestamp":"2017-04-05T20:44:54Z","tags":["warning","elasticsearch","admin"],"pid":10,"message":"Unable to revive connection: http://localhost:9200/"}
{"type":"log","@timestamp":"2017-04-05T20:44:54Z","tags":["warning","elasticsearch","admin"],"pid":10,"message":"No living connections"}
{"type":"log","@timestamp":"2017-04-05T20:44:56Z","tags":["warning","elasticsearch","admin"],"pid":10,"message":"Unable to revive connection: http://localhost:9200/"}
{"type":"log","@timestamp":"2017-04-05T20:44:56Z","tags":["warning","elasticsearch","admin"],"pid":10,"message":"No living connections"}
{"type":"log","@timestamp":"2017-04-05T20:44:59Z","tags":["warning","elasticsearch","admin"],"pid":10,"message":"Unable to revive connection: http://localhost:9200/"}

As I understand, there is not enough memory in elasticsearch configuration to let the process start. As a consequence, kibana does not find any connection on port 9200.

What do you think of this analysis ? Can you also show the logs on your mac ?

Regards,

@ericlemerdy

This comment has been minimized.

Show comment
Hide comment
@ericlemerdy

ericlemerdy Apr 5, 2017

According to https://www.elastic.co/guide/en/elasticsearch/reference/current/vm-max-map-count.html and spujadas/elk-docker#92 (comment), the host must execute :

$ sudo sysctl -w vm.max_map_count=262144

It solved the issue for me tonight.

ericlemerdy commented Apr 5, 2017

According to https://www.elastic.co/guide/en/elasticsearch/reference/current/vm-max-map-count.html and spujadas/elk-docker#92 (comment), the host must execute :

$ sudo sysctl -w vm.max_map_count=262144

It solved the issue for me tonight.

@ericlemerdy ericlemerdy closed this Apr 5, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment