Skip to content

Commit

Permalink
Merge pull request #82 from doximity/MOFONETNAV-168
Browse files Browse the repository at this point in the history
Remove deprecation warnings for Elasticsearch 7.X.X
  • Loading branch information
mrsweaters committed Aug 10, 2022
2 parents acf5416 + 1f10c8d commit 2683353
Show file tree
Hide file tree
Showing 34 changed files with 296 additions and 232 deletions.
9 changes: 9 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,15 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),

## [Unreleased]

## [1.0.0] - 2022-08-09
### Changed
- Consistently use quotations
- Update API to remove deprecation warnings for Elasticsearch 7.X.X in preparation for upgrade to 8.
- Freeze string literals
- Remove `_type` from query/create/update/delete methods
- Remove `use_new_timestamp_format` config
- Remove `include_type_name_on_create` config

## [0.14.1] - 2022-08-09
### Changed
- Don't publish pre-release builds to RubyGems
Expand Down
2 changes: 1 addition & 1 deletion Gemfile.lock
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
PATH
remote: .
specs:
es-elasticity (0.14.1)
es-elasticity (1.0.0)
activemodel (>= 5.2.0, < 7.1)
activerecord (>= 5.2.0, < 7.1)
activesupport (>= 5.2.0, < 7.1)
Expand Down
80 changes: 42 additions & 38 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,9 +29,10 @@ Or install it yourself as:
## Usage

### Version Support
This gem has [elasticsearch-ruby](https://github.com/elastic/elasticsearch-ruby) as a dependency. In order to use different versions of elasticsearch you will need to match your version of elasticsearch-ruby to the version of elasticsearch you want to use ([see here](https://github.com/elastic/elasticsearch-ruby#compatibility). Elasticity should work across all versions of elastisearch-ruby, although they have not all been tested so there are likely edge cases.

Currently tests are run on CirlceCI against elasticsearch 6.8.2 with elasticsearch-ruby 7.2.0.
This gem has [elasticsearch-ruby](https://github.com/elastic/elasticsearch-ruby) as a dependency. In order to use different versions of elasticsearch you will need to match your version of elasticsearch-ruby to the version of elasticsearch you want to use ([see here](https://github.com/elastic/elasticsearch-ruby#compatibility). Elasticity should work across all versions of elastisearch-ruby, although they have not all been tested so there are likely edge cases.

Currently tests are run on CirlceCI against elasticsearch 7.17.3 with elasticsearch-ruby 7.17.1.

### Configuration

Expand Down Expand Up @@ -149,7 +150,6 @@ documents = [
Search::User.bulk_update(documents)
```


### Searching

Class methods have access to the `search` method, which returns a lazy evaluated search. That means that the search will only be performed when the data is necessary, not when the `search` method is called.
Expand Down Expand Up @@ -194,7 +194,8 @@ adults = adults.active_records(User)
#### Search Args

##### explain: true
For `search` definitions we support passing `{ explain: true }` to the search as a second argument in order to surface the reason a search result was returned.

For `search` definitions we support passing `{ explain: true }` to the search as a second argument in order to surface the reason a search result was returned.

```ruby
# example in single search
Expand All @@ -211,6 +212,7 @@ multi = Elasticity::MultiSearch.new do |m|
m.add(:c, search_c, documents: ::SearchDoc::C)
end
```

For more information about the `active_records` method, read [ActiveRecord integration](#activerecord-integration).

### Segmented Documents
Expand Down Expand Up @@ -324,15 +326,15 @@ When the mapping needs to change, a hot remapping can be performed by doing the
1. Create a new index with the new mapping;
2. change the update alias to point to the new index, and change main alias to point to both indexes; at this point it will look something like this:

```
|¯¯¯¯¯¯¯¯¯¯¯¯¯|----------------------> |¯¯¯¯¯¯¯¯¯¯¯¯¯|
| MainAlias | | Old Index |
|_____________|----------| |_____________|
|
|¯¯¯¯¯¯¯¯¯¯¯¯¯| |-----------> |¯¯¯¯¯¯¯¯¯¯¯¯¯|
| UpdateAlias |----------------------> | New Index |
|_____________| |_____________|
```
```
|¯¯¯¯¯¯¯¯¯¯¯¯¯|----------------------> |¯¯¯¯¯¯¯¯¯¯¯¯¯|
| MainAlias | | Old Index |
|_____________|----------| |_____________|
|
|¯¯¯¯¯¯¯¯¯¯¯¯¯| |-----------> |¯¯¯¯¯¯¯¯¯¯¯¯¯|
| UpdateAlias |----------------------> | New Index |
|_____________| |_____________|
```

3. iterate over all documents on the old index, copying them to the new index;
4. change aliases to point only to the new index;
Expand All @@ -344,43 +346,43 @@ This is a simplified version, there are other things that happen to ensure consi

ActiveRecord integration is mainly a set of conventions rather than implementation, with the exception of one method that allows mapping documents back to a relation. Here is the list of conventions:

* have a class method on the document called `from_active_record` that creates a document object from the active record object;
* have a class method on the Document for rebuilding the index from the records;
* have an `after_save` and an `after_destroy` callbacks on the ActiveRecord model;
- have a class method on the document called `from_active_record` that creates a document object from the active record object;
- have a class method on the Document for rebuilding the index from the records;
- have an `after_save` and an `after_destroy` callbacks on the ActiveRecord model;

For example:

```ruby
class User < ActiveRecord::Base
after_save :update_index_document
after_destroy :delete_index_document
```ruby
class User < ActiveRecord::Base
after_save :update_index_document
after_destroy :delete_index_document

def update_index_document
Search::User.from_active_record(self).update
end
def update_index_document
Search::User.from_active_record(self).update
end

def delete_index_document
Search::User.delete(self.id)
end
def delete_index_document
Search::User.delete(self.id)
end
end

class Search::User < Elasticity::Document
# ... configuration
class Search::User < Elasticity::Document
# ... configuration

def self.from_active_record(ar)
new(name: ar.name, birthdate: ar.birthdate)
end
def self.from_active_record(ar)
new(name: ar.name, birthdate: ar.birthdate)
end

def self.rebuild_index
self.recreate_index
def self.rebuild_index
self.recreate_index

User.find_in_batches do |batch|
documents = batch.map { |record| from_active_record(record) }
self.bulk_index(documents)
end
User.find_in_batches do |batch|
documents = batch.map { |record| from_active_record(record) }
self.bulk_index(documents)
end
end
```
end
```

This makes the code very clear in intent, easier to see when and how things happen and under the developer control, keeping both parts very decoupled.

Expand Down Expand Up @@ -417,13 +419,15 @@ Search::User.adults.active_records(User.where(active: true))
```

## Upgrading from 0.7.0 to 0.8.0

The default persistence strategy changed from SingleIndex to AliasIndex in version 0.8.0 Add the following to your Document configuration to maintain the legacy behaviour.

```ruby
c.strategy = Elasticity::Strategies::SingleIndex
```

## Roadmap

- [ ] Define from_active_record interface
- [ ] Write more detailed documentation section for:
- [ ] Model definition
Expand Down
10 changes: 5 additions & 5 deletions bin/rake
Original file line number Diff line number Diff line change
Expand Up @@ -6,11 +6,11 @@
# this file is here to facilitate running it.
#

require 'pathname'
ENV['BUNDLE_GEMFILE'] ||= File.expand_path("../../Gemfile",
require "pathname"
ENV["BUNDLE_GEMFILE"] ||= File.expand_path("../../Gemfile",
Pathname.new(__FILE__).realpath)

require 'rubygems'
require 'bundler/setup'
require "rubygems"
require "bundler/setup"

load Gem.bin_path('rake', 'rake')
load Gem.bin_path("rake", "rake")
10 changes: 5 additions & 5 deletions bin/rspec
Original file line number Diff line number Diff line change
Expand Up @@ -6,11 +6,11 @@
# this file is here to facilitate running it.
#

require 'pathname'
ENV['BUNDLE_GEMFILE'] ||= File.expand_path("../../Gemfile",
require "pathname"
ENV["BUNDLE_GEMFILE"] ||= File.expand_path("../../Gemfile",
Pathname.new(__FILE__).realpath)

require 'rubygems'
require 'bundler/setup'
require "rubygems"
require "bundler/setup"

load Gem.bin_path('rspec-core', 'rspec')
load Gem.bin_path("rspec-core", "rspec")
4 changes: 2 additions & 2 deletions es-elasticity.gemspec
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# coding: utf-8
lib = File.expand_path('../lib', __FILE__)
lib = File.expand_path("../lib", __FILE__)
$LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
require 'elasticity/version'
require "elasticity/version"

Gem::Specification.new do |spec|
spec.name = "es-elasticity"
Expand Down
4 changes: 3 additions & 1 deletion lib/elasticity/base_document.rb
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
# frozen_string_literal: true

module Elasticity
class BaseDocument
include ::ActiveModel::Model
Expand Down Expand Up @@ -60,7 +62,7 @@ def self.index_config_defaults
end

def self.default_document_type
self.name.gsub('::', '_').underscore
self.name.gsub("::", "_").underscore
end

def self.default_index_base_name
Expand Down
38 changes: 20 additions & 18 deletions lib/elasticity/bulk.rb
Original file line number Diff line number Diff line change
@@ -1,20 +1,22 @@
# frozen_string_literal: true

module Elasticity
class Bulk
def initialize(client)
@client = client
@operations = []
end

def index(index_name, type, id, attributes)
@operations << { index: { _index: index_name, _type: type, _id: id, data: attributes }}
def index(index_name, id, attributes)
@operations << { index: { _index: index_name, _id: id, data: attributes }}
end

def update(index_name, type, id, attributes)
@operations << { update: { _index: index_name, _type: type, _id: id, data: attributes }}
def update(index_name, id, attributes)
@operations << { update: { _index: index_name, _id: id, data: attributes }}
end

def delete(index_name, type, id)
@operations << { delete: { _index: index_name, _type: type, _id: id }}
def delete(index_name, id)
@operations << { delete: { _index: index_name, _id: id }}
end

def execute
Expand All @@ -27,16 +29,16 @@ def initialize(client, index_name)
@index_name = index_name
end

def index(type, id, attributes)
super(@index_name, type, id, attributes)
def index(id, attributes)
super(@index_name, id, attributes)
end

def update(type, id, attributes)
super(@index_name, type, id, attributes)
def update(id, attributes)
super(@index_name, id, attributes)
end

def delete(type, id)
super(@index_name, type, id)
def delete(id)
super(@index_name, id)
end
end

Expand All @@ -47,17 +49,17 @@ def initialize(client, update_alias, delete_indexes)
@delete_indexes = delete_indexes
end

def index(type, id, attributes)
super(@update_alias, type, id, attributes)
def index(id, attributes)
super(@update_alias, id, attributes)
end

def update(type, id, attributes)
super(@update_alias, type, id, attributes)
def update(id, attributes)
super(@update_alias, id, attributes)
end

def delete(type, id)
def delete(id)
@delete_indexes.each do |index|
super(index, type, id)
super(index, id)
end
end
end
Expand Down
2 changes: 2 additions & 0 deletions lib/elasticity/config.rb
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
# frozen_string_literal: true

module Elasticity
class Config
def client=(client)
Expand Down
2 changes: 2 additions & 0 deletions lib/elasticity/document.rb
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
# frozen_string_literal: true

module Elasticity
class Document < BaseDocument
IndexMapper.set_delegates(singleton_class, :mapper)
Expand Down
24 changes: 19 additions & 5 deletions lib/elasticity/index_config.rb
Original file line number Diff line number Diff line change
@@ -1,18 +1,21 @@
# frozen_string_literal: true

module Elasticity
class IndexConfig
class SubclassError < StandardError; end

SUBCLASSES_WARNING = "Indices created in Elasticsearch 6.0.0 or later may only contain a single mapping type. " +
SUBCLASSES_WARNING = "Indices created in Elasticsearch 6.0.0 or later may only contain a single mapping type. "\
"Therefore, doument-type based inheritance has been disabled by Elasticity"
SUBCLASSES_ERROR = "Mapping types have been completely removed in Elasticsearch 7.0.0. " +
SUBCLASSES_ERROR = "Mapping types have been completely removed in Elasticsearch 7.0.0. "\
"Therefore, doument-type based inheritance has been disabled by Elasticity"
VERSION_FOR_SUBCLASS_WARNING = "6.0.0".freeze
VERSION_FOR_SUBCLASS_ERROR = "7.0.0".freeze
VERSION_FOR_SUBCLASS_WARNING = "6.0.0"
VERSION_FOR_SUBCLASS_ERROR = "7.0.0"
ATTRS = [
:index_base_name, :document_type, :mapping, :strategy, :subclasses,
:settings, :use_new_timestamp_format, :include_type_name_on_create
].freeze
VALIDATABLE_ATTRS = [:index_base_name, :document_type, :strategy].freeze
DEPRECATED_ATTRS = [:use_new_timestamp_format, :include_type_name_on_create].freeze

attr_accessor(*ATTRS)

Expand All @@ -23,6 +26,7 @@ def initialize(elasticity_config, defaults = {})
@elasticity_config = elasticity_config
yield(self)
subclasses_warning_or_exception
warn_deprecated_config
validate!
end

Expand All @@ -38,9 +42,10 @@ def client

def definition
return @definition if defined?(@definition)

@definition = {
settings: merge_settings,
mappings: { @document_type => @mapping.nil? ? {} : @mapping.deep_stringify_keys }
mappings: @mapping.nil? ? {} : @mapping.deep_stringify_keys
}
subclasses.each do |doc_type, subclass|
@definition[:mappings][doc_type] = subclass.constantize.mapping
Expand Down Expand Up @@ -80,6 +85,15 @@ def merge_settings
@elasticity_config.settings.merge(settings || {})
end

def warn_deprecated_config
DEPRECATED_ATTRS.each do |attr|
ActiveSupport::Deprecation.warn(
"#{attr} is deprecated and will be "\
"removed in the next major release."
) if public_send(attr).present?
end
end

def subclasses_warning_or_exception
return if subclasses.nil? || subclasses.empty?
raise(SubclassError.new(SUBCLASSES_ERROR)) if es_version_meets_or_exceeds?(VERSION_FOR_SUBCLASS_ERROR)
Expand Down

0 comments on commit 2683353

Please sign in to comment.