ElasticSearch第一篇 安装
-
维基百科使用Elasticsearch提供全文搜索并高亮关键字，以及输入实时搜索(search-as-you-type)和搜索纠错(did-you-mean)等搜索建议功能。
-
英国卫报使用Elasticsearch结合用户日志和社交网络数据提供给他们的编辑以实时的反馈，以便及时了解公众对新发表的文章的回应。
-
StackOverflow结合全文搜索与地理位置查询，以及more-like-this功能来找到相关的问题和答案。
-
Github使用Elasticsearch检索1300亿行的代码。
- 安装方便：没有其他依赖，下载后安装非常方便；只用修改几个参数就可以搭建起来一个集群
- JSON：输入/输出格式为 JSON，意味着不需要定义 Schema，快捷方便
- RESTful：基本所有操作（索引、查询、甚至是配置）都可以通过 HTTP 接口进行
- 分布式：节点对外表现对等（每个节点都可以用来做入口）；加入节点自动均衡
- 多租户：可根据不同的用途分索引；可以同时操作多个索引
es只能用非root用户启动，重新建一个用户：
[root@yyp softpackages]# useradd esuser
[root@yyp softpackages]# passwd esuser
……
passwd: all authentication tokens updated successfully.
[root@yyp softpackages]# su esuser
[esuser@yyp softpackages]$ cd /home/esuser/
[esuser@yyp ~]$ mkdir install
[esuser@yyp ~]$ tar -zxvf elasticsearch-5.0.1.tar.gz
[esuser@yyp ~]$ ll
total 32180
drwxr-xr-x. 7 esuser esuser 4096 Nov 11 17:11 elasticsearch-5.0.1
-rw-r--r--. 1 esuser esuser 32944777 Feb 18 09:54 elasticsearch-5.0.1.tar.gz
drwxrwxr-x. 2 esuser esuser 6 Feb 18 09:54 install
[esuser@yyp ~]$
[esuser@yyp ~]$ mv elasticsearch-5.0.1 install/elasticsearch
修改访问elasticsearch的IP及端口
[esuser@yyp install]$ vi ./elasticsearch/config/elasticsearch.yml
找到
network.host与http.port修改
network.host:192.168.74.129
http.port: 9200
启动elasticsearch
[esuser@yyp install]$ cd elasticsearch/bin/
[esuser@yyp bin]$ ./elasticsearch
Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x0000000085330000, 2060255232, 0) failed; error='Ca
nnot allocate memory' (errno=12)#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (mmap) failed to map 2060255232 bytes for committing reserved memory.
# An error report file with more information is saved as:
# /home/esuser/install/elasticsearch/bin/hs_err_pid38761.log
[esuser@yyp bin]$
- Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x0000000085330000, 2060255232, 0) failed; error='Ca nnot allocate memory' (errno=12)
完整错误信息：
[esuser@yyp elasticsearch]$ ./bin/elasticsearch
Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x0000000085330000, 2060255232, 0) failed; error='Ca
nnot allocate memory' (errno=12)
[esuser@yyp elasticsearch]$
由于elasticsearch5.0默认分配jvm空间大小为2g，修改jvm空间分配
[esuser@yyp elasticsearch]$ pwd
/home/esuser/install/elasticsearch
[esuser@yyp elasticsearch]$ vi config/jvm.options
将
-Xms2g
-Xmx2g
改为
-Xms512m
-Xmx512m
- Could not register mbeans java.security.AccessControlException: access denied ("javax.management.MBeanTrustPermission" "register")
完整错误信息：
[esuser@localhost install]$ elasticsearch-5.2.1/bin/elasticsearch
2017-02-22 03:13:42,718 main ERROR Could not register mbeans java.security.AccessControlException: access denied ("javax.management.MBeanTrustPermission" "register")
at java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
at java.lang.SecurityManager.checkPermission(SecurityManager.java:585)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.checkMBeanTrustPermission(DefaultMBeanServerInterceptor.java:1848)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:322)
at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:522)
at org.apache.logging.log4j.core.jmx.Server.register(Server.java:389)
at org.apache.logging.log4j.core.jmx.Server.reregisterMBeansAfterReconfigure(Server.java:167)
at org.apache.logging.log4j.core.jmx.Server.reregisterMBeansAfterReconfigure(Server.java:140)
at org.apache.logging.log4j.core.LoggerContext.setConfiguration(LoggerContext.java:541)
at org.apache.logging.log4j.core.LoggerContext.start(LoggerContext.java:258)
at org.apache.logging.log4j.core.impl.Log4jContextFactory.getContext(Log4jContextFactory.java:206)
at org.apache.logging.log4j.core.config.Configurator.initialize(Configurator.java:220)
at org.apache.logging.log4j.core.config.Configurator.initialize(Configurator.java:197)
at org.elasticsearch.common.logging.LogConfigurator.configureStatusLogger(LogConfigurator.java:125)
at org.elasticsearch.common.logging.LogConfigurator.configureWithoutConfig(LogConfigurator.java:67)
解决方式：
[root@localhost install]# cd jdk1.8.0_112/jre/lib/security/ [root@localhost security]# ll total 164 -rw-r--r--. 1 10 143 4054 Sep 23 12:20 blacklist -rw-r--r--. 1 10 143 1273 Sep 23 12:20 blacklisted.certs -rw-r--r--. 1 10 143 112860 Sep 23 12:20 cacerts -rw-r--r--. 1 10 143 2529 Feb 22 20:23 java.policy -rw-r--r--. 1 10 143 27358 Sep 23 12:20 java.security -rw-r--r--. 1 10 143 98 Sep 23 12:20 javaws.policy -rw-r--r--. 1 10 143 3527 Sep 23 12:20 local_policy.jar -rw-r--r--. 1 10 143 0 Sep 23 12:20 trusted.libraries -rw-r--r--. 1 10 143 3026 Sep 23 12:20 US_export_policy.jar [root@localhost security]#
在tomcat所用的jre/lib/security/java.policy文件中新增
permission javax.management.MBeanTrustPermission "register";
重启tomcat：
[root@localhost install]# apache-tomcat-8.5.11/bin/shutdown.sh
[root@localhost install]# apache-tomcat-8.5.11/bin/startup.sh
- Exception in thread "main" ElasticsearchParseException[malformed, expected settings to start with 'object', instead was [VALUE_STRING]]
完整错误信息：
[esuser@yyp elasticsearch]$ ./bin/elasticsearch
Exception in thread "main" ElasticsearchParseException[malformed, expected settings to start with 'object', instead was
[VALUE_STRING]] at org.elasticsearch.common.settings.loader.XContentSettingsLoader.load(XContentSettingsLoader.java:70)
at org.elasticsearch.common.settings.loader.XContentSettingsLoader.load(XContentSettingsLoader.java:50)
at org.elasticsearch.common.settings.loader.YamlSettingsLoader.load(YamlSettingsLoader.java:50)
at org.elasticsearch.common.settings.Settings$Builder.loadFromStream(Settings.java:938)
at org.elasticsearch.common.settings.Settings$Builder.loadFromPath(Settings.java:927)
at org.elasticsearch.node.internal.InternalSettingsPreparer.prepareEnvironment(InternalSettingsPreparer.java:10
2) at org.elasticsearch.bootstrap.Bootstrap.initialEnvironment(Bootstrap.java:207)
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:247)
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:112)
at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:103)
at org.elasticsearch.cli.SettingCommand.execute(SettingCommand.java:54)
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:96)
at org.elasticsearch.cli.Command.main(Command.java:62)
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:80)
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:73)
Refer to the log for complete error details.
[esuser@yyp elasticsearch]$
更改端口的时候，冒号与参数之间没有加空格
将
- network.host:192.168.74.129
- http.port: 9200
改为- network.host: 192.168.74.129
- http.port: 9200
- ERROR: bootstrap checks failed max file descriptors [4096] for elasticsearch process is too low, increase to at least [65536] max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
完整错误信息：
[esuser@yyp elasticsearch]$ ./bin/elasticsearch
[2017-02-18T10:20:15,360][INFO ][o.e.n.Node ] [] initializing ...
[2017-02-18T10:20:16,062][INFO ][o.e.e.NodeEnvironment ] [lQ0Agk6] using [1] data paths, mounts [[/ (rootfs)]], net
usable_space [11.8gb], net total_space [17.4gb], spins? [unknown], types [rootfs][2017-02-18T10:20:16,062][INFO ][o.e.e.NodeEnvironment ] [lQ0Agk6] heap size [503.6mb], compressed ordinary object p
ointers [true][2017-02-18T10:20:16,064][INFO ][o.e.n.Node ] [lQ0Agk6] node name [lQ0Agk6] derived from node ID; set [no
de.name] to override[2017-02-18T10:20:16,087][INFO ][o.e.n.Node ] [lQ0Agk6] version[5.0.1], pid[39039], build[080bb47/2016-11
-11T22:08:49.812Z], OS[Linux/3.10.0-123.el7.x86_64/amd64], JVM[Oracle Corporation/Java HotSpot(TM) 64-Bit Server VM/1.8.0_111/25.111-b14][2017-02-18T10:20:20,145][INFO ][o.e.p.PluginsService ] [lQ0Agk6] loaded module [aggs-matrix-stats]
[2017-02-18T10:20:20,159][INFO ][o.e.p.PluginsService ] [lQ0Agk6] loaded module [ingest-common]
[2017-02-18T10:20:20,160][INFO ][o.e.p.PluginsService ] [lQ0Agk6] loaded module [lang-expression]
[2017-02-18T10:20:20,160][INFO ][o.e.p.PluginsService ] [lQ0Agk6] loaded module [lang-groovy]
[2017-02-18T10:20:20,160][INFO ][o.e.p.PluginsService ] [lQ0Agk6] loaded module [lang-mustache]
[2017-02-18T10:20:20,160][INFO ][o.e.p.PluginsService ] [lQ0Agk6] loaded module [lang-painless]
[2017-02-18T10:20:20,160][INFO ][o.e.p.PluginsService ] [lQ0Agk6] loaded module [percolator]
[2017-02-18T10:20:20,160][INFO ][o.e.p.PluginsService ] [lQ0Agk6] loaded module [reindex]
[2017-02-18T10:20:20,160][INFO ][o.e.p.PluginsService ] [lQ0Agk6] loaded module [transport-netty3]
[2017-02-18T10:20:20,160][INFO ][o.e.p.PluginsService ] [lQ0Agk6] loaded module [transport-netty4]
[2017-02-18T10:20:20,161][INFO ][o.e.p.PluginsService ] [lQ0Agk6] no plugins loaded
[2017-02-18T10:20:29,966][INFO ][o.e.n.Node ] [lQ0Agk6] initialized
[2017-02-18T10:20:29,966][INFO ][o.e.n.Node ] [lQ0Agk6] starting ...
[2017-02-18T10:20:30,972][INFO ][o.e.t.TransportService ] [lQ0Agk6] publish_address {192.168.74.129:9300}, bound_addr
esses {192.168.74.129:9300}[2017-02-18T10:20:30,976][INFO ][o.e.b.BootstrapCheck ] [lQ0Agk6] bound or publishing to a non-loopback or non-link
-local address, enforcing bootstrap checksERROR: bootstrap checks failed
max file descriptors [4096] for elasticsearch process is too low, increase to at least [65536]
max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
[2017-02-18T10:20:31,263][INFO ][o.e.n.Node ] [lQ0Agk6] stopping ...
[2017-02-18T10:20:31,322][INFO ][o.e.n.Node ] [lQ0Agk6] stopped
[2017-02-18T10:20:31,322][INFO ][o.e.n.Node ] [lQ0Agk6] closing ...
[2017-02-18T10:20:31,394][INFO ][o.e.n.Node ] [lQ0Agk6] closed
[esuser@yyp elasticsearch]$
解决办法： 解决：切换到root用户，编辑limits.conf 添加类似如下内容
[esuser@yyp elasticsearch]$ su root
Password:
[root@yyp elasticsearch]# vi /etc/security/limits.conf
添加如下内容:
* soft nofile 65536
* hard nofile 131072
* soft nproc 2048
* hard nproc 4096
[root@yyp elasticsearch]# su esuser
[esuser@yyp elasticsearch]$ ./bin/elasticsearch
- ERROR: bootstrap checks failed max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
[esuser@yyp elasticsearch]$ ./bin/elasticsearch
[2017-02-18T10:31:53,617][INFO ][o.e.n.Node ] [] initializing ...
[2017-02-18T10:31:54,046][INFO ][o.e.e.NodeEnvironment ] [lQ0Agk6] using [1] data paths, mounts [[/ (rootfs)]], net
usable_space [11.8gb], net total_space [17.4gb], spins? [unknown], types [rootfs][2017-02-18T10:31:54,047][INFO ][o.e.e.NodeEnvironment ] [lQ0Agk6] heap size [503.6mb], compressed ordinary object p
ointers [true][2017-02-18T10:31:54,048][INFO ][o.e.n.Node ] [lQ0Agk6] node name [lQ0Agk6] derived from node ID; set [no
de.name] to override[2017-02-18T10:31:54,083][INFO ][o.e.n.Node ] [lQ0Agk6] version[5.0.1], pid[39376], build[080bb47/2016-11
-11T22:08:49.812Z], OS[Linux/3.10.0-123.el7.x86_64/amd64], JVM[Oracle Corporation/Java HotSpot(TM) 64-Bit Server VM/1.8.0_111/25.111-b14][2017-02-18T10:31:57,581][INFO ][o.e.p.PluginsService ] [lQ0Agk6] loaded module [aggs-matrix-stats]
[2017-02-18T10:31:57,582][INFO ][o.e.p.PluginsService ] [lQ0Agk6] loaded module [ingest-common]
[2017-02-18T10:31:57,582][INFO ][o.e.p.PluginsService ] [lQ0Agk6] loaded module [lang-expression]
[2017-02-18T10:31:57,582][INFO ][o.e.p.PluginsService ] [lQ0Agk6] loaded module [lang-groovy]
[2017-02-18T10:31:57,582][INFO ][o.e.p.PluginsService ] [lQ0Agk6] loaded module [lang-mustache]
[2017-02-18T10:31:57,582][INFO ][o.e.p.PluginsService ] [lQ0Agk6] loaded module [lang-painless]
[2017-02-18T10:31:57,582][INFO ][o.e.p.PluginsService ] [lQ0Agk6] loaded module [percolator]
[2017-02-18T10:31:57,582][INFO ][o.e.p.PluginsService ] [lQ0Agk6] loaded module [reindex]
[2017-02-18T10:31:57,582][INFO ][o.e.p.PluginsService ] [lQ0Agk6] loaded module [transport-netty3]
[2017-02-18T10:31:57,582][INFO ][o.e.p.PluginsService ] [lQ0Agk6] loaded module [transport-netty4]
[2017-02-18T10:31:57,583][INFO ][o.e.p.PluginsService ] [lQ0Agk6] no plugins loaded
[2017-02-18T10:32:12,435][INFO ][o.e.n.Node ] [lQ0Agk6] initialized
[2017-02-18T10:32:12,450][INFO ][o.e.n.Node ] [lQ0Agk6] starting ...
[2017-02-18T10:32:13,494][INFO ][o.e.t.TransportService ] [lQ0Agk6] publish_address {192.168.74.129:9300}, bound_addr
esses {192.168.74.129:9300}[2017-02-18T10:32:13,498][INFO ][o.e.b.BootstrapCheck ] [lQ0Agk6] bound or publishing to a non-loopback or non-link
-local address, enforcing bootstrap checksERROR: bootstrap checks failed
max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
[2017-02-18T10:32:13,535][INFO ][o.e.n.Node ] [lQ0Agk6] stopping ...
[2017-02-18T10:32:13,642][INFO ][o.e.n.Node ] [lQ0Agk6] stopped
[2017-02-18T10:32:13,642][INFO ][o.e.n.Node ] [lQ0Agk6] closing ...
[2017-02-18T10:32:13,700][INFO ][o.e.n.Node ] [lQ0Agk6] closed
[esuser@yyp elasticsearch]$
解决：切换到root用户修改配置sysctl.conf
[esuser@yyp elasticsearch]$ su root
Password:
[root@yyp elasticsearch]# vi /etc/sysctl.conf
添加下面配置：
vm.max_map_count=655360
并执行命令：sysctl -p
[root@yyp elasticsearch]# sysctl -p
vm.max_map_count = 655360
[root@yyp elasticsearch]#
然后，重新启动elasticsearch
[root@yyp elasticsearch]# su esuser
[esuser@yyp elasticsearch]$ ./bin/elasticsearch
输出：
[2017-02-18T10:43:27,711][INFO ][o.e.n.Node ] [] initializing ...
[2017-02-18T10:43:28,041][INFO ][o.e.e.NodeEnvironment ] [lQ0Agk6] using [1] data paths, mounts [[/ (rootfs)]], net
usable_space [11.8gb], net total_space [17.4gb], spins? [unknown], types [rootfs][2017-02-18T10:43:28,042][INFO ][o.e.e.NodeEnvironment ] [lQ0Agk6] heap size [503.6mb], compressed ordinary object p
ointers [true][2017-02-18T10:43:28,043][INFO ][o.e.n.Node ] [lQ0Agk6] node name [lQ0Agk6] derived from node ID; set [no
de.name] to override[2017-02-18T10:43:28,077][INFO ][o.e.n.Node ] [lQ0Agk6] version[5.0.1], pid[39688], build[080bb47/2016-11
-11T22:08:49.812Z], OS[Linux/3.10.0-123.el7.x86_64/amd64], JVM[Oracle Corporation/Java HotSpot(TM) 64-Bit Server VM/1.8.0_111/25.111-b14][2017-02-18T10:43:31,994][INFO ][o.e.p.PluginsService ] [lQ0Agk6] loaded module [aggs-matrix-stats]
[2017-02-18T10:43:31,995][INFO ][o.e.p.PluginsService ] [lQ0Agk6] loaded module [ingest-common]
[2017-02-18T10:43:31,995][INFO ][o.e.p.PluginsService ] [lQ0Agk6] loaded module [lang-expression]
[2017-02-18T10:43:31,995][INFO ][o.e.p.PluginsService ] [lQ0Agk6] loaded module [lang-groovy]
[2017-02-18T10:43:31,995][INFO ][o.e.p.PluginsService ] [lQ0Agk6] loaded module [lang-mustache]
[2017-02-18T10:43:31,995][INFO ][o.e.p.PluginsService ] [lQ0Agk6] loaded module [lang-painless]
[2017-02-18T10:43:31,995][INFO ][o.e.p.PluginsService ] [lQ0Agk6] loaded module [percolator]
[2017-02-18T10:43:31,995][INFO ][o.e.p.PluginsService ] [lQ0Agk6] loaded module [reindex]
[2017-02-18T10:43:31,995][INFO ][o.e.p.PluginsService ] [lQ0Agk6] loaded module [transport-netty3]
[2017-02-18T10:43:31,995][INFO ][o.e.p.PluginsService ] [lQ0Agk6] loaded module [transport-netty4]
[2017-02-18T10:43:31,996][INFO ][o.e.p.PluginsService ] [lQ0Agk6] no plugins loaded
[2017-02-18T10:43:42,684][INFO ][o.e.n.Node ] [lQ0Agk6] initialized
[2017-02-18T10:43:42,710][INFO ][o.e.n.Node ] [lQ0Agk6] starting ...
[2017-02-18T10:43:46,225][INFO ][o.e.t.TransportService ] [lQ0Agk6] publish_address {192.168.74.129:9300}, bound_addr
esses {192.168.74.129:9300}[2017-02-18T10:43:46,240][INFO ][o.e.b.BootstrapCheck ] [lQ0Agk6] bound or publishing to a non-loopback or non-link
-local address, enforcing bootstrap checks[2017-02-18T10:43:49,506][INFO ][o.e.c.s.ClusterService ] [lQ0Agk6] new_master {lQ0Agk6}{lQ0Agk6eQfeapNExaq_P6g}{fiUn
DJYETGmPTkMuPW9ztQ}{192.168.74.129}{192.168.74.129:9300}, reason: zen-disco-elected-as-master ([0] nodes joined)[2017-02-18T10:43:49,630][INFO ][o.e.h.HttpServer ] [lQ0Agk6] publish_address {192.168.74.129:9200}, bound_addr
esses {192.168.74.129:9200}[2017-02-18T10:43:49,630][INFO ][o.e.n.Node ] [lQ0Agk6] started
[2017-02-18T10:43:49,658][INFO ][o.e.g.GatewayService ] [lQ0Agk6] recovered [0] indices into cluster_state
出现上面结果即表示启动成功。 浏览器输入：http://192.168.74.129:9200 页面返回：
{
"name" : "lQ0Agk6",
"cluster_name" : "elasticsearch",
"cluster_uuid" : "D4CoRTZQS9WBsIKKTGNgag",
"version" : {
"number" : "5.0.1",
"build_hash" : "080bb47",
"build_date" : "2016-11-11T22:08:49.812Z",
"build_snapshot" : false,
"lucene_version" : "6.2.1"
},
"tagline" : "You Know, for Search"
}
当Elasticsearch在前台运行，可以使用Ctrl-C快捷键终止，或者你可以调用shutdown API来关闭：
curl -XPOST 'http://localhost:9200/_shutdown'