Laravel Elasticsearch Query Builder

Query Elasticsearch by using Eloquent

This Laravel package is developed to simplify the process of querying Elasticsearch. Eloquent is a powerful tool to access and manipulate data in RDB. However, it is not designed to query no-sql DB like Elasticsearch. This package is made to fill the gap between the most popular ORM in Laravel and Elasticsearch.

The package only relies on the official PHP Elasticsearch package(v6).

PHP version

The package is developed and tested under PHP v7.1. It should be also compatible with v7.*. Please email me if you find any compatibility issue.

Elasticsearch version

The package is developed and tested under Elasticsearch v6.*. It should be also compatible with v5.*. Please email me if you find any compatibility issue. It is confirmed that the package does not support versions before v5.*.

Installation

Add the following to your composer.json file:

"shisun/laravel-elasticsearch-query-builder": "dev-master"

Run composer update in terminal

Add the package to your app.php file in the config folder:

'providers' => [
    ...
    Shisun\LaravelElasticsearchQueryBuilder\LaravelElasticsearchQueryBuilderServiceProvider::class,
]

If your Elasticsearch is not accessible at localhost:9200 then you have to publish the config file by running:

$ php artisan vendor:publish --provider="Shisun\LaravelElasticsearchQueryBuilder\LaravelElasticsearchQueryBuilderServiceProvider"

(Optional) Change the ES_HOSTS in config/laravel-elasticsearch-query-builder.php to the address of your Elasticsearch

Use with Eloquent Model

Add EsTrait to your model

use Shisun\LaravelElasticsearchQueryBuilder\EsTrait;

class User extends Model
{
    use EsTrait;
}

Enable validation(Optional)

use Shisun\LaravelElasticsearchQueryBuilder\EsTrait;
class User extends Model
{
   use EsTrait;
   public $mappingProperties;
   public function __construct($attributes = []) {
       parent::__construct($attributes);
   
       $this->mappingProperties = [
           'id' => [
               'type' => 'integer',
           ],
           'name' => [
               'type' => 'text',
                   'index' => true,
           ],
           'email' => [
               'type' => 'keyword'
           ],
           'address' => [
               'type' => 'text'
           ],
           'float_example' => [
               'type' => 'float'
           ],
           'multi_fields_example' => [
               'type' => 'text',
               'fields' => [
                   'raw' => [
                       'type' => 'keyword'
                   ]
               ]
           ],
           'created_at' => [
               'type' => 'date',
               'format' => 'yyyy-MM-dd HH:mm:ss',
           ],
           'some_relations' => [
               'type' => 'nested',
               'properties' => [
                   ....
               ]
           ]
       ];
   }
}

Set model-level Index and Type name. Add follow two functions to your model.(Optional)

public function getIndexName() {
    return 'users';
}
// type name will be set to index name if this function is not defined.
public function getTypeName() {
    return 'users';
}

Use without Eloquent

use Shisun\LaravelElasticsearchQueryBuilder\LaravelElasticsearchQueryBuilder as Builder;
$query = (new Builder())->setOptions([
    'index_name' => 'users',
    'type_name'  => 'users'
])->where('name', 'Leo')->get()->toArray();
// or if you only want to generate the query without getting results
$query = (new Builder())->where('name', 'Leo')->getBody()

Usage

Init & Configs
- __construct
- setOptions
- getters
  - getIndexName
  - getTypeName
  - getValidation
  - getMappingProperties
- setters
  - setIndexName
  - setTypeName
  - setValidation
  - setMappingProperties
Query Clauses
- where
- orWhere
- whereMatch
- whereDoesntMatch
- orWhereMatch
- whereHas
- whereHasNull
- orWhereHas
- whereIn
- orWhereIn
- whereNotIn
- orWhereNotIn
- whereBetween
- orWhereBetween
Order
- orderBy
Pagination
- page
- limit
- offset
Aggregation
- aggregate
- aggregateAll
- aggregateOn
- groupBy
- min
- max
- avg
- sum
Other Filters
- minScore
- with
- withOut
Execute Query
- get
- scroll
- count
- first
- find
- delete
Results Manipulation
Direct Query Output
- getQuery
- getBody
- getAggs

Init & Configs

__construct

You do not need to use this function if you use Eloquent
Parameters

Name Required Type Default Description

model Eloquent,null null
Output

self

Examples

basic example

use Shisun\LaravelElasticsearchQueryBuilder\LaravelElasticsearchQueryBuilder as Builder;
$builder = new Builder();

setOptions

You do not need to use this function if you use Eloquent
Parameters

Name Required Type Default Description

options Y array
Output

self

Examples

Basic

use Shisun\LaravelElasticsearchQueryBuilder\LaravelElasticsearchQueryBuilder as Builder;
$builder = (new Builder())->setOptions([
    'index_name' => 'users',
    'type_name'  => 'users',
    'validation' => false,
    'mapping_properties' => [check the example in the installation section]
]);

Query Clauses

where

Parameters

Name	Required	Type	Default	Description
column	Y	`callable`,`string`
operator		`string`	`null`	`=`,`>`,`<`,`<=`,`>=`,`!=`,`*`
value		`mixed`	`null`
or		`bool`	`false`
boost		`bool`,`int`	`false`	The weight of the column

Output

self

Examples

= can be ignored
```
User::es()->where('id', 1)->first()
```

* is the wildcard operator

// Find the user whose name starts with 'Leo'
User::es()->where('name', '*', 'Leo*')->first()

column can be a function

User::es()->where(function($q) {
    $q->orWhere(...)->orWhere(...);
})->get()->toArray()

whereNull

Parameters

Name Required Type Default Description

column Y string Column or relation
Output

self

Examples

Find all users with no name
```
User::es()->whereNull('name')->get()
```

Find all users who don't have address

User::es()->whereNull('Addresses')->get()

whereNotNull

Parameters

Name Required Type Default Description

column Y string Column or relation
Output

self

Examples

Find all users with a name
```
User::es()->whereNotNull('name')->get()
```

Find all users who have at least one address

User::es()->whereNotNull('Addresses')->get()

orWhere

Parameters

Name Required Type Default Description

column Y callable,string

operator string null =,>,<,<=,>=,like,!=,*

value mixed null

boost bool,int false The weight of the column
Output

self

Examples

= can be ignored
```
User::es()->orWhere('id', 1)->first()
```

column can be a function

User::es()->orWhere(function($q) {
    $q->where(...)->where(...);
})->limit(1)->get()->toArray()

whereMatch

It is used to make fuzzy text search. This function should only be applied on text fields.
Parameters

Name Required Type Default Description

column Y string

value mixed null

options array [] matchquery options. Check elasticsearch Docs for references
Output

self

Examples

without options

User::es()->whereMatch('email', 'shisun@')->first()

with options

User::es()->whereMatch('email', 'shisun@', [
       'query' => 'this will be overrided by $value',
       'operator' => 'and',
       'zero_terms_query' => 'all'
     ])->first()

whereDoesntMatch

Parameters

Name Required Type Default Description

column Y string

value mixed null

options array [] matchquery options. Check elasticsearch Docs for references
Output

self

Examples

without options

User::es()->whereDoesntMatch('email', 'shisun@')->first()

with options

User::es()->whereDoesntMatch('email', 'shisun@', [
       'query' => 'this will be overrided by $value',
       'operator' => 'and',
       'zero_terms_query' => 'all'
     ])->first()

orWhereMatch

Parameters

Name Required Type Default Description

column Y string

value mixed null

options array [] matchquery options. Check elasticsearch Docs for references
Output

self

Examples

without options

User::es()->orWhereMatch('email', 'shisun@')->first()

with options

User::es()->orWhereMatch('email', 'shisun@', [
       'query' => 'this will be overrided by $value',
       'operator' => 'and',
       'zero_terms_query' => 'all'
     ])->first()

whereHas

Parameters

Name	Required	Type	Default	Description
relation	Y	`string`, `callable`		Must be capitalized
closure		`callable`, `null`	`null`	`from` and `to` cannot be both `null`
or		`boolean`	`null`	for internal use. See orWhereHas
boost		`boolean`,`integer`	`false`	Used to adjust the weight of the condition

Output

self

Examples

basic example

// find all users with active someRelations
User::es()->whereHas('someRelations', function($q) {
    $q->where('status', 'active');
})->first();

first parameter can be a function

// find all users with either active someRelations or red someOtherRelations
// Note: the parent 'whereHas' is interchangable with 'where' clause
User::es()->whereHas(function($q) {
    $q->orWhereHas('someRelations', function($k) {
         $k->where('status', 'active');
    })->orWhereHas('someOtherRelations', function($k) {
         $k->where('color', 'red');
    });
})->first();

orWhereHas

Parameters

Name	Required	Type	Default	Description
relation	Y	`string`, `callable`		Must be capitalized
closure		`callable`, `null`	`null`	`from` and `to` cannot be both `null`
boost		`boolean`,`integer`	`false`	Used to adjust the weight of the condition

Output

self

Examples

basic example

// find all users with either active someRelations or red someOtherRelations
// Note: the parent 'whereHas' is interchangable with 'where' clause
User::es()->whereHas(function($q) {
    $q->orWhereHas('someRelations', function($k) {
         $k->where('status', 'active');
    })->orWhereHas('someOtherRelations', function($k) {
         $k->where('color', 'red');
    });
})->first();

whereHasNull

Parameters

Name Required Type Default Description

relation Y string Must be capitalized

closure null, callable null
Output

self

Examples

basic example

// find all users with no someRelation
User::es()->whereHasNull('someRelations')->first();

with sub query

// find all users with no someRelation with id = 1
User::es()->whereHasNull('someRelations', function($q) {
    $q->where('id', 1);
})->first();

orWhereHasNull

Parameters

Name Required Type Default Description

relation Y string Must be capitalized

closure null, callable null
Output

self

Examples

basic example

// find all users either with no someRelation or named as 'Leo'
User::es()->where(function($q) {
    $q->orWhereHasNull('someRelations')->orWhere('name', 'Leo');
})->first();

whereIn

Parameters

Name Required Type Default Description

column Y string

values Y array
Output

self

Examples

basic example

// find all users with pending or active status
User::es()->whereIn('status', ['active', 'pending'])->get();

orWhereIn

Parameters

Name Required Type Default Description

column Y string

values Y array
Output

self

Examples

basic example

// find all users with either pending/active status or with name Leo
User::es()->where(function($q) {
    $q->orWhereIn('status', ['active', 'pending'])->orWhere('name', 'Leo');
})->get();

whereNotIn

Parameters

Name Required Type Default Description

column Y string

values Y array
Output

self

Examples

basic example

// find all users that are not in pending or active status
User::es()->whereNotIn('status', ['active', 'pending'])->get();

orWhereNotIn

Parameters

Name Required Type Default Description

column Y string

values Y array
Output

self

Examples

basic example

// find all users with either not in pending/active status or with name Leo
User::es()->where(function($q) {
    $q->orWhereNotIn('status', ['active', 'pending'])->orWhere('name', 'Leo');
})->get();

whereBetween

Parameters

Name Required Type Default Description

column Y string

from numeric null from and to cannot be both null

to numeric null
Output

self

Examples

basic example

User::es()->whereBetween('id', 1, 5)->first()

orWhereBetween

Parameters

Name Required Type Default Description

column Y string

from numeric null from and to cannot be both null

to numeric null
Output

self

Examples

basic example

User::es()->orWhereBetween('id', 1, 5)->first()

Order

orderBy

Parameters

Name Required Type Default Description

column Y string

order numeric asc either asc or desc

script array,boolean false
Output

self

Examples

basic example

User::es()->orderBy('id', 'desc')->get()

script

// If the category of the item is Laptop then use discount_price for ordering. Otherwise, use listing_price.
Item::es()->orderBy('id', 'desc', 
            ['lang' => 'painless', 'source' => "if(doc['category'].value == 'Laptops') {return doc['discount_price'].value;} else {return doc['listing_price'].value;}"])
      ->get()

Pagination

page

Parameters

Name Required Type Default Description

page Y integer

per_page Y integer
Output

self

Examples

basic example

// get the first page. 25 users per page.
User::es()->page(1, 25)->get()

limit

Parameters

Name Required Type Default Description

limit Y integer
Output

self

Examples

basic example

// get 25 users.
User::es()->limit(25)->get()

offset

Parameters

Name Required Type Default Description

limit Y integer
Output

self

Examples

basic example

// skip first 25 users.
User::es()->offset(25)->get()

Aggregations

aggregate

Parameters

Name Required Type Default Description

name Y string

agg Y callable
Output

self

Examples

basic example

Item::es()->aggregate('group_items_by_categories', function($q) {
    $q->groupBy('category_id');
})->get()->aggregations();

with filters

// get all active and pending items. But only aggregate on the filtered items that are also red.
// Note: aggregate clause is only applied on the filtered items.
Item::es()->whereIn('status', ['active', 'pending'])->aggregate('categories', function($q) {
    $q->where('color', 'red')->aggregate('group_by', function($k) {
        $k->groupBy('category_id');
    });
})->get()->aggregations();
// this returns
// [
//     'categories' => [
//         'doc_count' => 50,
//         'group_by'  => [
//             'doc_count_error_upper_bound' => 0,
//             'sum_other_doc_count'         => 8,
//             'buckets' => [
//                 [
//                     'key' => 1,
//                     'doc_count' => 15
//                 ],
//                 ...
//             ]
//         ]
//     ]
//  ]
// the 'key' in buckets is one of the category_id

aggregateAll

Parameters

Name Required Type Default Description

name Y string

agg Y callable
Output

self

Examples

with filters

// get all active and pending items. And aggregate on all red items.
// Note: aggregateAll clause is applied on all items regardless other queries.
Item::es()->whereIn('status', ['active', 'pending'])->aggregateAll('categories', function($q) {
    $q->where('color', 'red')->aggregate('group_by', function($k) {
        $k->groupBy('category_id');
    });
})->get()->aggregations();
// this returns
// [
//     'categories' => [
//         'doc_count' => 50,
//         'group_by'  => [
//             'doc_count_error_upper_bound' => 0,
//             'sum_other_doc_count'         => 8,
//             'buckets' => [
//                 [
//                     'key' => 1,
//                     'doc_count' => 15
//                 ],
//                 ...
//             ]
//         ]
//     ]
//  ]
// the 'key' in buckets is one of the category_id

aggregateOn

Parameters

Name Required Type Default Description

relation Y string Must be capitalized

agg Y callable

custom_name string null
Output

self

Examples

basic example

Item::es()->aggregateOn('someRelations', function($q) {
    $q->min('id');
})->get()->aggregations();
// this returns
// [
//     'some_relations' => [
//         'doc_count' => 50,
//         'min_id'  => [
//             'value' => 1
//         ]
//     ]
//  ]
// 'some_relations' will be replaced if custom_name is provided

groupBy

Parameters

Name Required Type Default Description

column Y string

size integer 10 The limit of number of groups. Default to 10 groups
Output

self
Check aggregate clause for examples

min

Parameters

Name Required Type Default Description

column Y string

custom_name string null
Output

self

Examples

basic example

Item::es()->min('id')->get()->aggregations();
// or
Item::es()->aggregate('test', function($q) {
    $q->min('id');
})->get()->aggregations();
// They both return. Note: the 'test' in the second example is ignored.
// [
//     'min_id' => [
//         'value' => 1
//     ]
//  ]
// 'min_id' will be replaced if custom_name is provided. The format of the name is 'min_' + column

max

Parameters

Name Required Type Default Description

column Y string

custom_name string null

missing_value numeric null This value will be used to replace null values
Output

self

Examples

basic example

Item::es()->max('id')->get()->aggregations();
// or
Item::es()->aggregate('test', function($q) {
    $q->max('id');
})->get()->aggregations();
// They both return. Note: the 'test' in the second example is ignored.
// [
//     'max_id' => [
//         'value' => 1
//     ]
//  ]
// 'min_id' will be replaced if custom_name is provided. The format of the name is 'max_' + column

avg

Parameters

Name Required Type Default Description

column Y string

custom_name string null

missing_value numeric null This value will be used to replace null values
Output

self

Examples

basic example

Item::es()->avg('id')->get()->aggregations();
// or
Item::es()->aggregate('test', function($q) {
    $q->avg('id');
})->get()->aggregations();
// They both return. Note: the 'test' in the second example is ignored.
// [
//     'avg_id' => [
//         'value' => 1
//     ]
//  ]
// 'min_id' will be replaced if custom_name is provided. The format of the name is 'avg_' + column

sum

Parameters

Name	Type	Default	Description
column	`string`	`Empty`
custom_name	`string`	`null`
missing_value	`numeric`	`null`	This value will be used to replace null values
script	`array`	`null`	Check official Doc as reference

Output

self

Examples

basic example

Item::es()->sum('id')->get()->aggregations();
// or
Item::es()->aggregate('test', function($q) {
    $q->sum('id');
})->get()->aggregations();
// They both return. Note: the 'test' in the second example is ignored.
// [
//     'sum_id' => [
//         'value' => 1
//     ]
//  ]
// 'min_id' will be replaced if custom_name is provided. The format of the name is 'sum_' + column

Other Filters

minScore

Parameters

Name Required Type Default Description

score Y numeric Only get items with _score greater than this value
Output

self

Examples

basic example

User::es()->whereMatch('name', 'Leo', ['operator' => 'and'])->minScore(5)->get()

with

Parameters

Name Required Type Default Description

relations Y list include these relations in results. All relations are included by default.
Output

self

Examples

basic example

User::es()->with('Addresses', 'Company')->get()

withOut

Parameters

Name Required Type Default Description

relations Y list Exclude these relations in results. All relations are included by default.
Output

self

Examples

basic example

User::es()->withOut('Addresses', 'Company')->get()

Execute Query

get

scroll

Parameters

Name	Type	Default	Description
scroll_alive	`string`	`5m`	This specifies the life span of the scroll session.
scroll_size	`integer`	`500`	This specifies the number of records retrieved from Elasticsearch per batch.
json	`boolean`	`false`	If encode the results from each batch in JSON format. Fetching data and put the data in PHP array can consume a lot of memory. This option can greatly reduce the memory usage.

Output

array
Examples
1. basic example
```
User::es()->scroll()
```

count

Output

integer
Examples
1. basic example
```
User::es()->count()
```

first

Parameters

Name Required Type Default Description

eloquent boolean false get result in array or Eloquent. Returns array if not provided.
Output

array|Eloquent
Examples
1. basic example
```
User::es()->first()
```

find

Parameters

Name Required Type Default Description

key Y mixed
Output

array

Examples

basic example

// get the user with id = 5
// Note: the key_name can be set by setKeyName or setOptions. The key_name is 'id' by default
User::es()->find(5)

delete

Delete a record from elasticsearch. Returns false if the record does not exist.
Parameters

Name Required Type Default Description

key Y mixed
Output

array

Examples

basic example

// delete the user with id = 5
// Note: the key_name can be set by setKeyName or setOptions. The key_name is 'id' by default
User::es()->delete(5)

Results Manipulation

toArray

Output

array

toEloquent

Warning: This function does not work if you don't use the package with Eloquent.

Output

Eloquent

rawResults

Get raw results from Elasticsearch

Output

array

aggregations

Get aggregation results

Output

array

getAggregationBuckets

This is a helper function to get buckets from the aggregation specified by agg_name

Parameters

Name Required Type Default Description

agg_name Y mixed

agg mixed null
Output

array

Examples

basic example

// if the aggregations are
// this returns
// [
//     'categories' => [
//         'doc_count' => 50,
//         'group_by'  => [
//             'doc_count_error_upper_bound' => 0,
//             'sum_other_doc_count'         => 8,
//             'buckets' => [
//                 [
//                     'key' => 1,
//                     'doc_count' => 15
//                 ],
//                 ...
//             ]
//         ]
//     ]
//  ]
// Then you can use getAggregationBuckets('categories') to get the buckets array

paginate

Returns pagination information

Parameters

Name Required Type Default Description

per_page mixed The value can be ignored if per_page is set by limit() or page(). Otherwise, it's required.
Output

array

Example

basic

[
  "pages" => [
    1,
    2,
    3,
    4,
    5,
  ],
  "rows" => 165,
  "active" => 1,
  "totalpages" => 7.0,
  "prev" => false,
  "next" => true,
  "per_page" => 25
]

Name		Name	Last commit message	Last commit date
Latest commit History 49 Commits
config		config
src		src
.gitignore		.gitignore
.styleci.yml		.styleci.yml
LICENSE		LICENSE
README.md		README.md
changelog.md		changelog.md
composer.json		composer.json
contributing.md		contributing.md
phpunit.xml		phpunit.xml
rewriteRules.php		rewriteRules.php
rules.php		rules.php

Name	Required	Type	Default	Description
column	Y	`string`
from		`numeric`	`null`	`from` and `to` cannot be both `null`
to		`numeric`	`null`

Name	Required	Type	Default	Description
column	Y	`string`
size		`integer`	`10`	The limit of number of groups. Default to 10 groups

License

ShisunXia/Laravel-Elasticsearch-Query-Builder

Folders and files

Latest commit

History

Repository files navigation

Laravel Elasticsearch Query Builder

PHP version

Elasticsearch version

Installation

Usage

Table of Contents

Init & Configs

__construct

setOptions

Query Clauses

where

whereNull

whereNotNull

orWhere

whereMatch

whereDoesntMatch

orWhereMatch

whereHas

orWhereHas

whereHasNull

orWhereHasNull

whereIn

orWhereIn

whereNotIn

orWhereNotIn

whereBetween

orWhereBetween

Order

orderBy

Pagination

page

limit

offset

Aggregations

aggregate

aggregateAll

aggregateOn

groupBy

min

max

avg

sum

Other Filters

minScore

with

withOut

Execute Query

get

scroll

count

first

find

delete

Results Manipulation

toArray

toEloquent

rawResults

aggregations

getAggregationBuckets

paginate

Direct Query Output

getQuery

getBody

getAggs

Release History

Meta

About

Topics

Resources

License

Stars

Watchers

Forks

Releases 4

Packages