Skip to content
Permalink
Browse files

Added support for Elasticsearch 7

  • Loading branch information...
ankane committed Apr 12, 2019
1 parent 4054147 commit 7ce0440ab4f2fdd02a52b2bcf42fb4f90e668667
@@ -14,30 +14,24 @@ cache:
directories:
- $HOME/elasticsearch
env:
- ELASTICSEARCH_VERSION=6.7.0
- ELASTICSEARCH_VERSION=7.0.0
jdk: openjdk10
matrix:
include:
- gemfile: Gemfile
- gemfile: test/gemfiles/activerecord51.gemfile
env: ELASTICSEARCH_VERSION=6.0.0
env: ELASTICSEARCH_VERSION=7.0.0
- gemfile: test/gemfiles/activerecord50.gemfile
env: ELASTICSEARCH_VERSION=5.6.10
- gemfile: test/gemfiles/activerecord42.gemfile
env: ELASTICSEARCH_VERSION=5.0.1
- gemfile: test/gemfiles/mongoid6.gemfile
env: ELASTICSEARCH_VERSION=6.7.0
- gemfile: test/gemfiles/mongoid7.gemfile
env: ELASTICSEARCH_VERSION=6.0.0
services:
- mongodb
- redis-server
- gemfile: test/gemfiles/mongoid5.gemfile
- gemfile: test/gemfiles/mongoid6.gemfile
services:
- mongodb
- redis-server
- gemfile: Gemfile
env: ELASTICSEARCH_VERSION=7.0.0-rc1
allow_failures:
- gemfile: Gemfile
env: ELASTICSEARCH_VERSION=7.0.0-rc1
notifications:
email:
on_success: never
@@ -1,3 +1,13 @@
## 4.0.0 [unreleased]

- Added support for Elasticsearch 7
- Added `models` option

Breaking changes

- Removed support for Elasticsearch 5
- Removed support for multi-word synonyms (they no longer work with shingles)

## 3.1.3

- Added support for endless ranges
@@ -53,7 +53,7 @@ Add this line to your application’s Gemfile:
gem 'searchkick'
```

The latest version works with Elasticsearch 5 and 6. For Elasticsearch 2, use version 2.5.0 and [this readme](https://github.com/ankane/searchkick/blob/v2.5.0/README.md).
The latest version works with Elasticsearch 6 and 7. For Elasticsearch 5, use version 3.1.3 and [this readme](https://github.com/ankane/searchkick/blob/v3.1.3/README.md).

Add searchkick to models you want to search.

@@ -312,13 +312,13 @@ A few languages require plugins:

```ruby
class Product < ApplicationRecord
searchkick synonyms: [["scallion", "green onion"], ["qtip", "cotton swab"]]
searchkick synonyms: [["burger", "hamburger"], ["sneakers", "shoes"]]
end
```

Call `Product.reindex` after changing synonyms.

Synonyms cannot be more than two words at the moment.
Synonyms cannot be multiple words at the moment.

To read synonyms from a file, use:

@@ -796,8 +796,6 @@ Script support
Product.search "*", aggs: {color: {script: {source: "'Color: ' + _value"}}}
```

**Note:** Use `inline` instead of `source` before Elasticsearch 5.6

Date histogram

```ruby
@@ -924,9 +922,7 @@ You can also index and search geo shapes.

```ruby
class Restaurant < ApplicationRecord
searchkick geo_shape: {
bounds: {tree: "geohash", precision: "1km"}
}
searchkick geo_shape: [:bounds]
def search_data
attributes.merge(
@@ -959,12 +955,6 @@ Not touching the query shape
Restaurant.search "burger", where: {bounds: {geo_shape: {type: "envelope", relation: "disjoint", coordinates: [{lat: 38, lon: -123}, {lat: 37, lon: -122}]}}}
```

Containing the query shape

```ruby
Restaurant.search "fries", where: {bounds: {geo_shape: {type: "envelope", relation: "contains", coordinates: [{lat: 38, lon: -123}, {lat: 37, lon: -122}]}}}
```

## Inheritance

Searchkick supports single table inheritance.
@@ -1496,21 +1486,15 @@ Then use `products` and `coupons` as typical results.

**Note:** Errors are not raised as with single requests. Use the `error` method on each query to check for errors.

## Multiple Indices
## Multiple Models

Search across multiple models/indices with:
Search across multiple models with:

```ruby
Searchkick.search "milk", index_name: [Product, Category]
Searchkick.search "milk", models: [Product, Category]
```

Specify conditions for different indices

```ruby
where: {_or: [{_type: "product", in_stock: true}, {_type: "category", active: true}]}
```

Boost specific indices with:
Boost specific models with:

```ruby
indices_boost: {Category => 2, Product => 1}
@@ -1657,7 +1641,7 @@ Product.search "milk", includes: [:brand, :stores]
Eager load different associations by model

```ruby
Searchkick.search("*", index_name: [Product, Store], model_includes: {Product => [:store], Store => [:product]})
Searchkick.search("*", models: [Product, Store], model_includes: {Product => [:store], Store => [:product]})
```

Run additional scopes on results
@@ -1900,6 +1884,11 @@ Check out [this great post](https://www.tiagoamaro.com.br/2014/12/11/multi-tenan

See [how to upgrade to Searchkick 3](docs/Searchkick-3-Upgrade.md)

## Elasticsearch 6 to 7 Upgrade

1. Install Searchkick 4
2. Upgrade your Elasticsearch cluster

## Elasticsearch 5 to 6 Upgrade

Elasticsearch 6 removes the ability to reindex with the `_all` field. Before you upgrade, we recommend disabling this field manually and specifying default fields on your models.
@@ -78,16 +78,36 @@ def self.server_below?(version)
Gem::Version.new(server_version.split("-")[0]) < Gem::Version.new(version.split("-")[0])
end

# memoize for performance
def self.server_below7?
unless defined?(@server_below7)
@server_below7 = server_below?("7.0.0")
end
@server_below7
end

def self.search(term = "*", model: nil, **options, &block)
options = options.dup
klass = model

# make Searchkick.search(index_name: [Product]) and Product.search equivalent
# convert index_name into models if possible
# this should allow for easier upgrade
if options[:index_name] && !options[:models] && Array(options[:index_name]).all? { |v| v.respond_to?(:searchkick_index) }
options[:models] = options.delete(:index_name)
end

# make Searchkick.search(models: [Product]) and Product.search equivalent
unless klass
index_name = Array(options[:index_name])
if index_name.size == 1 && index_name.first.respond_to?(:searchkick_index)
klass = index_name.first
options.delete(:index_name)
models = Array(options[:models])
if models.size == 1
klass = models.first
options.delete(:models)
end
end

if klass
if (options[:models] && Array(options[:models]) != [klass]) || Array(options[:index_name]).any? { |v| v.respond_to?(:searchkick_index) && v != klass }
raise ArgumentError, "Use Searchkick.search to search multiple models"
end
end

@@ -17,7 +17,7 @@ def create(body = {})
end

def delete
if !Searchkick.server_below?("6.0.0") && alias_exists?
if alias_exists?
# can't call delete directly on aliases in ES 6
indices = client.indices.get_alias(name: name).keys
client.indices.delete index: indices
@@ -68,7 +68,7 @@ def total_docs
}
)

response["hits"]["total"]
Searchkick::Results.new(nil, response).total_count
end

def promote(new_name, update_refresh_interval: false)
@@ -4,15 +4,13 @@ def index_options
options = @options
language = options[:language]
language = language.call if language.respond_to?(:call)
index_type = options[:_type]
index_type = index_type.call if index_type.respond_to?(:call)

if options[:mappings] && !options[:merge_mappings]
settings = options[:settings] || {}
mappings = options[:mappings]
else
below60 = Searchkick.server_below?("6.0.0")
below62 = Searchkick.server_below?("6.2.0")
below70 = Searchkick.server_below?("7.0.0")

default_type = "text"
default_analyzer = :searchkick_index
@@ -144,15 +142,6 @@ def index_options
}
}

if below60
# ES docs say standard token filter does nothing in ES 5
# (and therefore isn't needed at at), but tests say otherwise
# https://www.elastic.co/guide/en/elasticsearch/reference/5.0/analysis-standard-tokenfilter.html
[default_analyzer, :searchkick_search, :searchkick_search2].each do |analyzer|
settings[:analysis][:analyzer][analyzer][:filter].unshift("standard")
end
end

stem = options[:stem]

case language
@@ -279,8 +268,7 @@ def index_options
# - Only apply the synonym expansion at index time
# - Don't have the synonym filter applied search
# - Use directional synonyms where appropriate. You want to make sure that you're not injecting terms that are too general.
settings[:analysis][:analyzer][default_analyzer][:filter].insert(4, "searchkick_synonym") if below60
settings[:analysis][:analyzer][default_analyzer][:filter] << "searchkick_synonym"
settings[:analysis][:analyzer][default_analyzer][:filter].insert(2, "searchkick_synonym")

%w(word_start word_middle word_end).each do |type|
settings[:analysis][:analyzer]["searchkick_#{type}_index".to_sym][:filter].insert(2, "searchkick_synonym")
@@ -391,10 +379,6 @@ def index_options
"{name}" => keyword_mapping
}

if below60 && all
dynamic_fields["{name}"][:include_in_all] = !options[:searchable]
end

if options.key?(:filterable)
dynamic_fields["{name}"] = {type: default_type, index: index_false_value}
end
@@ -413,25 +397,24 @@ def index_options
multi_field = dynamic_fields["{name}"].merge(fields: dynamic_fields.except("{name}"))

mappings = {
index_type => {
properties: mapping,
_routing: routing,
# https://gist.github.com/kimchy/2898285
dynamic_templates: [
{
string_template: {
match: "*",
match_mapping_type: "string",
mapping: multi_field
}
properties: mapping,
_routing: routing,
# https://gist.github.com/kimchy/2898285
dynamic_templates: [
{
string_template: {
match: "*",
match_mapping_type: "string",
mapping: multi_field
}
]
}
}
]
}

if below60
all_enabled = all && (!options[:searchable] || options[:searchable].to_a.map(&:to_s).include?("_all"))
mappings[index_type][:_all] = all_enabled ? analyzed_field_options : {enabled: false}
if below70
index_type = options[:_type]
index_type = index_type.call if index_type.respond_to?(:call)
mappings = {index_type => mappings}
end

mappings = mappings.symbolize_keys.deep_merge((options[:mappings] || {}).symbolize_keys)
Oops, something went wrong.

0 comments on commit 7ce0440

Please sign in to comment.
You can’t perform that action at this time.