Skip to content
Branch: master
Find file History
Type Name Latest commit message Commit time
Failed to load latest commit information.
examples/notes Add license headers, LICENSE and NOTICE files (#861) Nov 29, 2018
lib/elasticsearch [STORE] Handle total hits as an object in search response Dec 14, 2018
spec Add license headers, LICENSE and NOTICE files (#861) Nov 29, 2018
LICENSE.txt Add license headers, LICENSE and NOTICE files (#861) Nov 29, 2018 Update to link to migration blog post (#857) Nov 2, 2018


Persistence layer for Ruby domain objects in Elasticsearch, using the Repository pattern.


This library is compatible with Ruby 1.9.3 and higher.

The library version numbers follow the Elasticsearch major versions, and the master branch is compatible with the Elasticsearch master branch, therefore, with the next major version.

Rubygem Elasticsearch
0.1 1.x
2.x 2.x
5.x 5.x
6.x 6.x
master master


Install the package from Rubygems:

gem install elasticsearch-persistence

To use an unreleased version, either add it to your Gemfile for Bundler:

gem 'elasticsearch-persistence', git: 'git://', branch: '6.x'

or install it from a source code checkout:

git clone
cd elasticsearch-rails/elasticsearch-persistence
bundle install
rake install


The library provides the Repository pattern for adding persistence to your Ruby objects.

The Repository Pattern

The Elasticsearch::Persistence::Repository module provides an implementation of the repository pattern and allows you to save, delete, find and search objects stored in Elasticsearch, as well as configure mappings and settings for the index. It's an unobtrusive and decoupled way of adding persistence to your Ruby objects.

Let's have a simple plain old Ruby object (PORO):

class Note
  attr_reader :attributes

  def initialize(attributes={})
    @attributes = attributes

  def to_hash

Let's create a default, "dumb" repository, as a first step:

require 'elasticsearch/persistence'
class MyRepository; include Elasticsearch::Persistence::Repository; end
repository =

We can save a Note instance into the repository...

note = id: 1, text: 'Test'
# PUT http://localhost:9200/repository/_doc/1 [status:201, request:0.210s, query:n/a]
# > {"id":1,"text":"Test"}
# < {"_index":"repository","_type":"note","_id":"1","_version":1,"created":true}

...find it...

n = repository.find(1)
# GET http://localhost:9200/repository/_doc/1 [status:200, request:0.003s, query:n/a]
# < {"_index":"repository","_type":"note","_id":"1","_version":2,"found":true, "_source" : {"id":1,"text":"Test"}}
=> <Note:0x007fcbfc0c4980 @attributes={"id"=>1, "text"=>"Test"}> for it... { match: { text: 'test' } }).first
# GET http://localhost:9200/repository/_search [status:200, request:0.005s, query:0.002s]
# > {"query":{"match":{"text":"test"}}}
# < {"took":2, ... "hits":{"total":1, ... "hits":[{ ... "_source" : {"id":1,"text":"Test"}}]}}
=> <Note:0x007fcbfc1c7b70 @attributes={"id"=>1, "text"=>"Test"}>

...or delete it:

# DELETE http://localhost:9200/repository/_doc/1 [status:200, request:0.014s, query:n/a]
# < {"found":true,"_index":"repository","_type":"note","_id":"1","_version":3}
=> {"found"=>true, "_index"=>"repository", "_type"=>"note", "_id"=>"1", "_version"=>2}

The repository module provides a number of features and facilities to configure and customize the behavior:

  • Configuring the Elasticsearch client being used
  • Setting the index name, document type, and object class for deserialization
  • Composing mappings and settings for the index
  • Creating, deleting or refreshing the index
  • Finding or searching for documents
  • Providing access both to domain objects and hits for search results
  • Providing access to the Elasticsearch response for search results (aggregations, total, ...)
  • Defining the methods for serialization and deserialization

There are two mixins you can include in your Repository class. The first Elasticsearch::Persistence::Repository, provides the basic methods and settings you'll need. The second, Elasticsearch::Persistence::Repository::DSL adds some additional class methods that allow you to set options that instances of the class will share.

Basic Repository mixin

For simple cases, you can just include the Elasticsearch::Persistence::Repository mixin to your class:

class MyRepository
  include Elasticsearch::Persistence::Repository

  # Customize the serialization logic
  def serialize(document)
    super.merge(my_special_key: 'my_special_stuff')

  # Customize the de-serialization logic
  def deserialize(document)
    puts "# ***** CUSTOM DESERIALIZE LOGIC... *****"

client = ENV['ELASTICSEARCH_URL'], log: true)
repository = client, index_name: :my_notes, type: :note, klass: Note)
repository.settings number_of_shards: 1 do
  mapping do
    indexes :text, analyzer: 'snowball'

The custom Elasticsearch client will be used now, with a custom index and type names, as well as the custom serialization and de-serialization logic.

We can create the index with the desired settings and mappings:

repository.create_index! force: true
# PUT http://localhost:9200/my_notes
# > {"settings":{"number_of_shards":1},"mappings":{ ... {"text":{"analyzer":"snowball","type":"string"}}}}}

Save the document with extra properties added by the serialize method:
# PUT http://localhost:9200/my_notes/note/1
# > {"id":1,"text":"Test","my_special_key":"my_special_stuff"}
{"_index"=>"my_notes", "_type"=>"my_note", "_id"=>"1", "_version"=>4, ... }

And deserialize it:

<Note:0x007f9bd782b7a0 @attributes={... "my_special_key"=>"my_special_stuff"}>

The DSL mixin

In some cases, you'll want to set some of the repository configurations at the class level. This makes most sense when the instances of the repository will use that same configuration:

require 'base64'

class NoteRepository
  include Elasticsearch::Persistence::Repository
  include Elasticsearch::Persistence::Repository::DSL

  index_name 'notes'
  document_type 'note'
  klass Note

  settings number_of_shards: 1 do
    mapping do
      indexes :text,  analyzer: 'snowball'
      # Do not index images
      indexes :image, index: false

  # Base64 encode the "image" field in the document
  def serialize(document)
    hash = document.to_hash.clone
    hash['image'] = Base64.encode64(hash['image']) if hash['image']

  # Base64 decode the "image" field in the document
  def deserialize(document)
    hash = document['_source']
    hash['image'] = Base64.decode64(hash['image']) if hash['image'] hash

You can create an instance of this custom class and get each of the configurations.

client = 'http://localhost:9200', log: true)
repository = client)
# => 'notes'

You can also override the default configuration with options passed to the initialize method:

client = 'http://localhost:9250', log: true)
client.transport.logger.formatter = proc { |s, d, p, m| "\e[2m# #{m}\n\e[0m" }
repository = client, index_name: 'notes_development')

repository.create_index!(force: true)

note ='id' => 1, 'text' => 'Document with image', 'image' => '... BINARY DATA ...')
# PUT http://localhost:9200/notes_development/_doc/1
# > {"id":1,"text":"Document with image","image":"Li4uIEJJTkFSWSBEQVRBIC4uLg==\n"}
puts repository.find(1).attributes['image']
# GET http://localhost:9200/notes_development/_doc/1
# < {... "_source" : { ... "image":"Li4uIEJJTkFSWSBEQVRBIC4uLg==\n"}}
# => ... BINARY DATA ...

Functionality Provided by the Repository mixin

Each of the following configurations can be set for a repository instance. If you have included the Elasticsearch::Persistence::Repository::DSL mixin, then you can use the class-level DSL methods to set each value. You can still override the configuration for any instance by passing options to the #initialize method. Even if you don't use the DSL mixin, you can set the instance configuration with options passed the #initialize method.


The repository uses the standard Elasticsearch client.

client = '')
repository = client)
repository.client.transport.logger =
# => Elasticsearch::Client

or with the DSL mixin:

class NoteRepository
  include Elasticsearch::Persistence::Repository
  include Elasticsearch::Persistence::Repository::DSL

  client url: ''

repository =
# => Elasticsearch::Client

The index_name method specifies the Elasticsearch index to use for storage, lookup and search. The default index name is 'repository'.

repository = 'notes_development')
# => 'notes_development'

or with the DSL mixin:

class NoteRepository
  include Elasticsearch::Persistence::Repository
  include Elasticsearch::Persistence::Repository::DSL

  index_name 'notes_development'

repository =
# => 'notes_development'

The document_type method specifies the Elasticsearch document type to use for storage, lookup and search. The default value is '_doc'. Keep in mind that future versions of Elasticsearch will not allow you to set this yourself and will use the type, '_doc'.

repository = 'note')
# => 'note'

or with the DSL mixin:

class NoteRepository
  include Elasticsearch::Persistence::Repository
  include Elasticsearch::Persistence::Repository::DSL

  document_type 'note'

repository =
# => 'note'

The klass method specifies the Ruby class name to use when initializing objects from documents retrieved from the repository. If this value is not set, a Hash representation of the document will be returned instead.

repository = Note)
# => Note

or with the DSL mixin:

class NoteRepository
  include Elasticsearch::Persistence::Repository
  include Elasticsearch::Persistence::Repository::DSL

  klass Note

repository =
# => Note
Index Configuration

The settings and mappings methods, provided by the elasticsearch-model gem, allow you to configure the index properties:

repository.settings number_of_shards: 1
# => {:number_of_shards=>1}

repository.mappings { indexes :title, analyzer: 'snowball' }
# => { :note => {:properties=> ... }}

or with the DSL mixin:

class NoteRepository
  include Elasticsearch::Persistence::Repository
  include Elasticsearch::Persistence::Repository::DSL

  mappings { indexes :title, analyzer: 'snowball' }
  settings number_of_shards: 1

repository =
Create a Repository and set its configuration with a block

You can also use the #create method to instantiate and set the mappings and settings on an instance with a block in one call:

repository = NoteRepository.create(index_name: 'notes_development') do
  settings number_of_shards: 1, number_of_replicas: 0 do
    mapping dynamic: 'strict' do
      indexes :foo do
        indexes :bar
      indexes :baz
Index Management

The convenience methods create_index!, delete_index! and refresh_index! allow you to manage the index lifecycle. These methods can only be called on repository instances and are not implemented at the class level.


The serialize and deserialize methods allow you to customize the serialization of the document when it is persisted to Elasticsearch, and define the initialization procedure when loading it from the storage:

class NoteRepository
  include Elasticsearch::Persistence::Repository

  def serialize(document)
    Hash[ { |k,v|  v.upcase! if k == :title; [k,v] }]

  def deserialize(document)['_source']).deep_symbolize_keys

The save method allows you to store a domain object in the repository:

note = id: 1, title: 'Quick Brown Fox'
# => {"_index"=>"notes_development", "_type"=>"_doc", "_id"=>"1", "_version"=>1, "created"=>true}

The update method allows you to perform a partial update of a document in the repository. Use either a partial document:

repository.update id: 1, title: 'UPDATED',  tags: []
# => {"_index"=>"notes_development", "_type"=>"_doc", "_id"=>"1", "_version"=>2}

Or a script (optionally with parameters):

repository.update 1, script: 'if (!ctx._source.tags.contains(t)) { ctx._source.tags += t }', params: { t: 'foo' }
# => {"_index"=>"notes_development", "_type"=>"_doc", "_id"=>"1", "_version"=>3}

The delete method allows you to remove objects from the repository (pass either the object itself or its ID):


The find method allows you to find one or many documents in the storage and returns them as deserialized Ruby objects: 2, title: 'Fast White Dog')

note = repository.find(1)
# => <MyNote ... QUICK BROWN FOX>

notes = repository.find(1, 2)
# => [<MyNote... QUICK BROWN FOX>, <MyNote ... FAST WHITE DOG>]

When the document with a specific ID isn't found, a nil is returned instead of the deserialized object:

notes = repository.find(1, 3, 2)
# => [<MyNote ...>, nil, <MyNote ...>]

Handle the missing objects in the application code, or call compact on the result.


The search method is used to retrieve objects from the repository by a query string or definition in the Elasticsearch DSL:'fox or dog').to_a
# GET http://localhost:9200/notes_development/_doc/_search?q=fox
# => [<MyNote ... FOX ...>, <MyNote ... DOG ...>] { match: { title: 'fox dog' } }).to_a
# GET http://localhost:9200/notes_development/_doc/_search
# > {"query":{"match":{"title":"fox dog"}}}
# => [<MyNote ... FOX ...>, <MyNote ... DOG ...>]

The returned object is an instance of the Elasticsearch::Persistence::Repository::Response::Results class, which provides access to the results, the full returned response and hits.

results = { match: { title: 'fox dog' } })

# Iterate over the objects
results.each do |note|
  puts "* #{note.attributes[:title]}"

# Iterate over the objects and hits
results.each_with_hit do |note, hit|
  puts "* #{note.attributes[:title]}, score: #{hit._score}"
# * QUICK BROWN FOX, score: 0.29930896
# * FAST WHITE DOG, score: 0.29930896

# Get total results
# => 2

# Access the raw response as a Hashie::Mash instance.
# Note that a Hashie::Mash will only be created if the 'response' method is called on the results. 
# => 0

# Access the raw response
# => {...}

Example Application

An example Sinatra application is available in examples/notes/application.rb, and demonstrates a rich set of features:

  • How to create and configure a custom repository class
  • How to work with a plain Ruby class as the domain object
  • How to integrate the repository with a Sinatra application
  • How to write complex search definitions, including pagination, highlighting and aggregations
  • How to use search results in the application view

The ActiveRecord Pattern

The ActiveRecord pattern has been deprecated as of version 6.0.0 of this gem. Please use the Repository Pattern instead. For more information on migrating 5.x ActiveRecord-based applications to use the Repository Pattern, please see this blog post.


This software is licensed under the Apache 2 license, quoted below.

Copyright (c) 2014 Elasticsearch <>

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
See the License for the specific language governing permissions and
limitations under the License.
You can’t perform that action at this time.