-
Notifications
You must be signed in to change notification settings - Fork 0
Elasticsearch Cheat Sheet
ElasticSearch is a highly scalable open source search engine with a REST API
https://github.com/elastic/elasticsearch
- Distributed, scalable, and highly available
- Real-time search and analytics capabilities
- Sophisticated RESTful API
- is build ton top of Lucene
Default port - localhost:9200
https://www.elastic.co/guide/en/elasticsearch/reference/1.3/_basic_concepts.html
Near Real Time, this means is there is a slight latency (normally one second) from the time you index a document until the time it becomes searchable.
A cluster is a collection of one or more nodes (servers) that together holds your entire data and provides federated indexing and search capabilities across all nodes. A cluster is identified by a unique name which by default is "elasticsearch"
A node is a single server that is part of your cluster, stores your data, and participates in the cluster’s indexing and search capabilities. Just like a cluster, a node is identified by a name which by default is a random Marvel character name that is assigned to the node at startup. In a single cluster, you can have as many nodes as you want.
An index is a collection of documents that have somewhat similar characteristics. For example, you can have an index for customer data, another index for a product catalog, and yet another index for order data.
Within an index, you can define one or more types. A type is a logical category/partition of your index whose semantics is completely up to you. In general, a type is defined for documents that have a set of common fields. For example, let’s assume you run a blogging platform and store all your data in a single index. In this index, you may define a type for user data, another type for blog data, and yet another type for comments data.
A document is a basic unit of information that can be indexed.
An index can potentially store a large amount of data that can exceed the hardware limits of a single node. For example, a single index of a billion documents taking up 1TB of disk space may not fit on the disk of a single node or may be too slow to serve search requests from a single node alone. To solve this problem, Elasticsearch provides the ability to subdivide your index into multiple pieces called shards.
Sharding is important for two primary reasons:
- It allows you to horizontally split/scale your content volume
- It allows you distribute and parallelize operations across shards (potentially on multiple nodes) thus increasing performance/throughput
$ elasticsearch # or use window servicebase url - http://localhost:9200
- /_cat --- will use cat api for checking health of cluster, will show all option for _cat command
- /_cat/nodes?v --- list of nodes in our cluster
- /_cat/indices?v --- list of indexes
- /_cat/plugins -- List of plugins
- Green - Everything Ok
- Yellow - Some replica not allocated
- Red - Some data is missing
$ curl -XPUT 'localhost:9200/blog?pretty'
$ curl -XPUT 'http://localhost:9200/blog/user/dilbert' -d '{ "name" : "Dilbert Brown" }'
$ curl -XPUT 'http://localhost:9200/blog/post/1' -d '
{
"user": "dilbert",
"postDate": "2011-12-15",
"body": "Search is hard. Search should be easy." ,
"title": "On search"
}'
$ curl -XPUT 'http://localhost:9200/blog/post/2' -d '
{
"user": "dilbert",
"postDate": "2011-12-12",
"body": "Distribution is hard. Distribution should be easy." ,
"title": "On distributed search"
}'
# Get
$ curl -XGET 'http://localhost:9200/blog/user/dilbert?pretty=true'
$ curl -XGET 'http://localhost:9200/blog/post/1?pretty=true'
$ curl -XPOST 'customer/external/1/_update?pretty&pretty' -H 'Content-Type: application/json' -d'
{
"doc": { "name": "Jane Doe" }
}
'
Elasticsearch provides the abiltiy to perform any of operation ( insert, update, delete) in batches using the bulk api.
POST /customer/internal/_bulk
{ "index": { "id": 1} }
{ "name" : "john" }
{ "index": { "id": 2} }
{ "name" : "doe" }Above example - indexes two document ( Id -1 john & Id-2 - Doe) in one bulk operation.
$ POST /customer/external/_bulk?pretty
{"update":{"_id":"1"}}
{"doc": { "name": "John Doe becomes Jane Doe" } }
{"delete":{"_id":"2"}}Above example - update the first document & then delete the second document.
$ curl 'localhost:9200/blog/_search?q=user:ganesh&pretty=true'
This will return result in
{
"took" : 7,
"timed_out" : false,
"_shards" : {
"total" : 5,
"successful" : 5,
"failed" : 0
},
"hits" : {
"total" : 1,
"max_score" : 0.30685282,
"hits" : [ {
"_index" : "blog",
"_type" : "post",
"_id" : "1",
"_score" : 0.30685282,
"_source":{"user":"ganesh", "title":"on search", "body":"search is hard"}
} ]
}
}We can also use the JSON query language
$ curl 'localhost:9200/blog/_search?pretty' -d '{
"query" : {
"match" : { "user" : "ganesh"}
}
}'GET /bank/_search?q=*&sort=account_number:asc&pretty
GET /bank/_search
{
"query": { "match_all": {} },
"sort" : [
{ "account_number" : "asc" }
]
}
// match term `mill` or `lane`
GET /bank/_search
{
"query": { "match": { "address": "mill lane" } }
}
// match term `mill lane`
GET /bank/_search
{
"query": { "match_phrase": { "address": "mill lane" } }
}
GET /bank/_search
{
"query": { "match_all": {} },
"from": 10,
"size": 10
} GET /bank/_search
{
"query": { "match": { "account_number": 20 } }
} GET /bank/_search
{
"query": { "match_all": {} },
"_source": ["account_number", "balance"]
}// return all accounts containing "mill" and "lane" in address
GET /bank/_search
{
"query": {
"bool": {
"must": [
{ "match": { "address": "mill" } },
{ "match": { "address": "lane" } }
]
}
}
}
GET /bank/_search
{
"query": {
"bool": {
"must": [
{ "match": { "age": "40" } }
],
"must_not": [
{ "match": { "state": "ID" } }
]
}
}
}
GET /bank/_search
{
"size": 0,
"aggs": {
"group_by_state": {
"terms": {
"field": "state.keyword"
}
}
}
}
Above aggregation is similar in concept to - SELECT state, COUNT(*) FROM bank GROUP BY state ORDER BY COUNT(*) DESC.
Note that we set size=0 to not show search hits because we only want to see the aggregation results in the response.
$ sudo bin/elasticsearch_plugin install <plugin-name>
$ ./bin/plugin list
- x-pack
- head
- Marvel - https://www.elastic.co/downloads/marvel
elastic / changeme
https://www.elastic.co/downloads/kibana
$ bin/kibana
Open source centralized logging manager
- Elasticsearch.Net
- NEST
Elasticsearch.Net is a very low level, dependency free.