Merge branch 'master' of github.com:karmi/tire

* 'master' of github.com:karmi/tire: (218 commits)
  [PERSISTENCE] Model::Persistence returns true/false for #save and #destroy operations
  [PERSISTENCE] Model::Persistence uses "string" as the default mapping type
  Release 0.5.7
  [FIX] Added missing require of ANSI::Progressbar in tasks
  Allow options for the `terms` query
  Release 0.5.6
  [karmi#656] Refactored model-specific importing into a set of strategies
  [karmi#656] Added adapter specific importing for different ORMs/OHMs/ODMs/OxMs
  Refactored the `import:all` task to not use the manual Tire integration tracking
  Removed the manual Tire integration tracking for models
  Refactored the `import:model` and `import:all` Rake tasks
  [karmi#655] Extracted import Rake task into `import:all` and `import:model`
  [karmi#655] Added `included` tracking to Model::Search
  Use `Thread.current` in the Curb client to prevent `Curl::Err::MultiBadEasyHandle` errors
  [TEST] Do not run percolator tests on TravisCI
  [TEST] Fixed Ruby 1.8 incompatible Hash syntax in Tire::Search::Query
  [TEST] Fixed missing index delete in teardown for the `boosting` query test
  [TEST] Added a test case for the `boosting` query
  [karmi#648] Updated the integration tests for `constant_score` query
  Added support for the `constant_score` query
  ...

.gitignore

@@ -11,4 +11,4 @@ examples/*.html
 *.log
 .rvmrc
 .rbenv-version
-tags
+tmp/

.travis.yml

@@ -2,12 +2,34 @@
 # Configuration file for http://travis-ci.org/#!/karmi/tire
 # ---------------------------------------------------------
 
-script: "bundle exec rake test:unit"
+language: ruby
 
 rvm:
-  - 1.8.7
+  - 2.0.0
   - 1.9.3
+  - 1.8.7
   - ree
+  - jruby-19mode
+
+env:
+  - TEST_COMMAND="rake test:unit"
+  - TEST_COMMAND="rake test:integration"
+
+script: "bundle exec $TEST_COMMAND"
+
+services:
+  - elasticsearch
+  - redis
+  - mongodb
+
+matrix:
+  exclude:
+    - rvm: 1.8.7
+      env: TEST_COMMAND="rake test:integration"
+    - rvm: ree
+      env: TEST_COMMAND="rake test:integration"
+  allow_failures:
+    - rvm: ree
 
 notifications:
   disable: true

.yardopts

@@ -0,0 +1 @@
+--markup markdown

Gemfile

@@ -2,3 +2,9 @@ source "http://rubygems.org"
 
 # Specify your gem's dependencies in tire.gemspec
 gemspec
+
+platform :jruby do
+  gem "jdbc-sqlite3"
+  gem "activerecord-jdbcsqlite3-adapter"
+  gem "json" if defined?(RUBY_VERSION) && RUBY_VERSION < '1.9'
+end

README.markdown

@@ -1,27 +1,26 @@
 Tire
 =========
 
-_Tire_ is a Ruby (1.8 or 1.9) client for the [ElasticSearch](http://www.elasticsearch.org/)
+_Tire_ is a Ruby (1.8 or 1.9) client for the [Elasticsearch](http://www.elasticsearch.org/)
 search engine/database.
 
-_ElasticSearch_ is a scalable, distributed, cloud-ready, highly-available,
+_Elasticsearch_ is a scalable, distributed, cloud-ready, highly-available,
 full-text search engine and database with
-[powerfull aggregation features](http://www.elasticsearch.org/guide/reference/api/search/facets/),
+[powerful aggregation features](http://www.elasticsearch.org/guide/reference/api/search/facets/),
 communicating by JSON over RESTful HTTP, based on [Lucene](http://lucene.apache.org/), written in Java.
 
 This Readme provides a brief overview of _Tire's_ features. The more detailed documentation is at <http://karmi.github.com/tire/>.
 
-Both of these documents contain a lot of information. Please set aside some time to read them thoroughly, before you blindly dive into „somehow making it work“. Just skimming through it **won't work** for you. For more information, please refer to the [integration test suite](https://github.com/karmi/tire/tree/master/test/integration)
-and [issues](https://github.com/karmi/tire/issues).
+Both of these documents contain a lot of information. Please set aside some time to read them thoroughly, before you blindly dive into „somehow making it work“. Just skimming through it **won't work** for you. For more information, please see the project [Wiki](https://github.com/karmi/tire/wiki/_pages), search the [issues](https://github.com/karmi/tire/issues), and refer to the [integration test suite](https://github.com/karmi/tire/tree/master/test/integration).
 
 Installation
 ------------
 
-OK. First, you need a running _ElasticSearch_ server. Thankfully, it's easy. Let's define easy:
+OK. First, you need a running _Elasticsearch_ server. Thankfully, it's easy. Let's define easy:
 
-    $ curl -k -L -o elasticsearch-0.19.0.tar.gz http://github.com/downloads/elasticsearch/elasticsearch/elasticsearch-0.19.0.tar.gz
-    $ tar -zxvf elasticsearch-0.19.0.tar.gz
-    $ ./elasticsearch-0.19.0/bin/elasticsearch -f
+    $ curl -k -L -o elasticsearch-0.20.2.tar.gz http://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-0.20.2.tar.gz
+    $ tar -zxvf elasticsearch-0.20.2.tar.gz
+    $ ./elasticsearch-0.20.2/bin/elasticsearch -f
 
 See, easy. On a Mac, you can also use _Homebrew_:
 
@@ -41,11 +40,11 @@ Of course, you can install it from the source as well:
 Usage
 -----
 
-_Tire_ exposes easy-to-use domain specific language for fluent communication with _ElasticSearch_.
+_Tire_ exposes easy-to-use domain specific language for fluent communication with _Elasticsearch_.
 
 It easily blends with your _ActiveModel_/_ActiveRecord_ classes for convenient usage in _Rails_ applications.
 
-To test-drive the core _ElasticSearch_ functionality, let's require the gem:
+To test-drive the core _Elasticsearch_ functionality, let's require the gem:
 
 ```ruby
     require 'rubygems'
@@ -102,7 +101,7 @@ for a specific document type:
 ```
 
 Of course, we may have large amounts of data, and it may be impossible or impractical to add them to the index
-one by one. We can use _ElasticSearch's_
+one by one. We can use _Elasticsearch's_
 [bulk storage](http://www.elasticsearch.org/guide/reference/api/bulk.html).
 Notice, that collection items must have an `id` property or method,
 and should have a `type` property, if you've set any specific mapping for the index.
@@ -247,7 +246,7 @@ The best thing about `boolean` queries is that we can easily save these partial
 to mix and reuse them later. So, we may define a query for the _tags_ property:
 
 ```ruby
-    tags_query = lambda do
+    tags_query = lambda do |boolean|
       boolean.should { string 'tags:ruby' }
       boolean.should { string 'tags:java' }
     end
@@ -256,7 +255,7 @@ to mix and reuse them later. So, we may define a query for the _tags_ property:
 And a query for the _published_on_ property:
 
 ```ruby
-    published_on_query = lambda do
+    published_on_query = lambda do |boolean|
       boolean.must   { string 'published_on:[2011-01-01 TO 2011-01-02]' }
     end
 ```
@@ -344,7 +343,7 @@ When you set the log level to _debug_:
 the JSON responses are logged as well. This is not a great idea for production environment,
 but it's priceless when you want to paste a complicated transaction to the mailing list or IRC channel.
 
-The _Tire_ DSL tries hard to provide a strong Ruby-like API for the main _ElasticSearch_ features.
+The _Tire_ DSL tries hard to provide a strong Ruby-like API for the main _Elasticsearch_ features.
 
 By default, _Tire_ wraps the results collection in a enumerable `Results::Collection` class,
 and result items in a `Results::Item` class, which looks like a child of `Hash` and `Openstruct`,
@@ -357,15 +356,15 @@ If that seems like a great idea to you, there's a big chance you already have su
 
 One would bet it's an `ActiveRecord` or `ActiveModel` class, containing model of your Rails application.
 
-Fortunately, _Tire_ makes blending _ElasticSearch_ features into your models trivially possible.
+Fortunately, _Tire_ makes blending _Elasticsearch_ features into your models trivially possible.
 
 
 ActiveModel Integration
 -----------------------
 
 If you're the type with no time for lengthy introductions, you can generate a fully working
 example Rails application, with an `ActiveRecord` model and a search form, to play with
-(it even downloads _ElasticSearch_ itself, generates the application skeleton and leaves you with
+(it even downloads _Elasticsearch_ itself, generates the application skeleton and leaves you with
 a _Git_ repository to explore the steps and the code):
 
     $ rails new searchapp -m https://raw.github.com/karmi/tire/master/examples/rails-application-template.rb
@@ -384,7 +383,7 @@ To make it searchable with _Tire_, just `include` it:
 When you now save a record:
 
 ```ruby
-    Article.create :title =>   "I Love ElasticSearch",
+    Article.create :title =>   "I Love Elasticsearch",
                    :content => "...",
                    :author =>  "Captain Nemo",
                    :published_on => Time.now
@@ -529,7 +528,7 @@ The easiest way is to customize the `to_json` serialization support of your mode
     class Article < ActiveRecord::Base
       # ...
 
-      include_root_in_json = false
+      self.include_root_in_json = false
       def to_indexed_json
         to_json :except => ['updated_at'], :methods => ['length']
       end
@@ -574,9 +573,12 @@ control on how the documents are added to or removed from the index:
     end
 ```
 
+When you're integrating _Tire_ with ActiveRecord models, you should use the `after_commit`
+and `after_rollback` hooks to keep the index in sync with your database.
+
 The results returned by `Article.search` are wrapped in the aforementioned `Item` class, by default.
-This way, we have a fast and flexible access to the properties returned from _ElasticSearch_ (via the
-`_source` or `fields` JSON properties). This way, we can index whatever JSON we like in _ElasticSearch_,
+This way, we have a fast and flexible access to the properties returned from _Elasticsearch_ (via the
+`_source` or `fields` JSON properties). This way, we can index whatever JSON we like in _Elasticsearch_,
 and retrieve it, simply, via the dot notation:
 
 ```ruby
@@ -588,18 +590,18 @@ and retrieve it, simply, via the dot notation:
 ```
 
 The `Item` instances masquerade themselves as instances of your model within a _Rails_ application
-(based on the `_type` property retrieved from _ElasticSearch_), so you can use them carefree;
+(based on the `_type` property retrieved from _Elasticsearch_), so you can use them carefree;
 all the `url_for` or `dom_id` helpers work as expected.
 
-If you need to access the “real” model (eg. to access its assocations or methods not
-stored in _ElasticSearch_), just load it from the database:
+If you need to access the “real” model (eg. to access its associations or methods not
+stored in _Elasticsearch_), just load it from the database:
 
 ```ruby
     puts article.load(:include => 'comments').comments.size
 ```
 
 You can see that _Tire_ stays as far from the database as possible. That's because it believes
-you have most of the data you want to display stored in _ElasticSearch_. When you need
+you have most of the data you want to display stored in _Elasticsearch_. When you need
 to eagerly load the records from the database itself, for whatever reason,
 you can do it with the `:load` option when searching:
 
@@ -617,39 +619,47 @@ Instead of simple `true`, you can pass any options for the model's find method:
     end
 ```
 
-Note that _Tire_ search results are fully compatible with [`will_paginate`](https://github.com/mislav/will_paginate),
-so you can pass all the usual parameters to the `search` method in the controller:
+If you would like to access properties returned by Elasticsearch (such as `_score`),
+in addition to model instance, use the `each_with_hit` method:
+
+```ruby
+    results = Article.search 'One', :load => true
+    results.each_with_hit do |result, hit|
+      puts "#{result.title} (score: #{hit['_score']})"
+    end
+
+    # One (score: 0.300123)
+```
+
+Note that _Tire_ search results are fully compatible with [_WillPaginate_](https://github.com/mislav/will_paginate)
+and [_Kaminari_](https://github.com/amatsuda/kaminari), so you can pass all the usual parameters to the
+`search` method in the controller:
 
 ```ruby
     @articles = Article.search params[:q], :page => (params[:page] || 1)
 ```
 
-OK. Chances are, you have lots of records stored in your database. How will you get them to _ElasticSearch_? Easy:
+OK. Chances are, you have lots of records stored in your database. How will you get them to _Elasticsearch_? Easy:
 
 ```ruby
     Article.index.import Article.all
 ```
 
 This way, however, all your records are loaded into memory, serialized into JSON,
-and sent down the wire to _ElasticSearch_. Not practical, you say? You're right.
+and sent down the wire to _Elasticsearch_. Not practical, you say? You're right.
 
-Provided your model implements some sort of _pagination_ — and it probably does —, you can just run:
+When your model is an `ActiveRecord::Base` or `Mongoid::Document` one, or when it implements
+some sort of pagination, you can just run:
 
 ```ruby
     Article.import
 ```
 
-In this case, the `Article.paginate` method is called, and your records are sent to the index
-in chunks of 1000. If that number doesn't suit you, just provide a better one:
-
-```ruby
-    Article.import :per_page => 100
-```
-
-Any other parameters you provide to the `import` method are passed down to the `paginate` method.
+Depending on the setup of your model, either `find_in_batches`, `limit..skip` or pagination is used
+to import your data.
 
 Are we saying you have to fiddle with this thing in a `rails console` or silly Ruby scripts? No.
-Just call the included _Rake_ task on the commandline:
+Just call the included _Rake_ task on the command line:
 
 ```bash
     $ rake environment tire:import CLASS='Article'
@@ -662,7 +672,7 @@ provided by the `mapping` block in your model):
     $ rake environment tire:import CLASS='Article' FORCE=true
 ```
 
-When you'll spend more time with _ElasticSearch_, you'll notice how
+When you'll spend more time with _Elasticsearch_, you'll notice how
 [index aliases](http://www.elasticsearch.org/guide/reference/api/admin-indices-aliases.html)
 are the best idea since the invention of inverted index.
 You can index your data into a fresh index (and possibly update an alias once everything's fine):
@@ -671,13 +681,18 @@ You can index your data into a fresh index (and possibly update an alias once ev
     $ rake environment tire:import CLASS='Article' INDEX='articles-2011-05'
 ```
 
-OK. All this time we have been talking about `ActiveRecord` models, since
-it is a reasonable _Rails_' default for the storage layer.
-
-But what if you use another database such as [MongoDB](http://www.mongodb.org/),
-another object mapping library, such as [Mongoid](http://mongoid.org/)?
+Finally, consider the Rake importing task just a convenient starting point. If you're loading
+substantial amounts of data, want better control on which data will be indexed, etc., use the
+lower-level Tire API with eg. `ActiveRecordBase#find_in_batches` directly:
 
-Well, things stay mostly the same:
+```ruby
+    Article.where("published_on > ?", Time.parse("2012-10-01")).find_in_batches(include: authors) do |batch|
+      Tire.index("articles").import batch
+    end
+```
+If you're using a different database, such as [MongoDB](http://www.mongodb.org/),
+another object mapping library, such as [Mongoid](http://mongoid.org/) or [MongoMapper](http://mongomapper.com/),
+things stay mostly the same:
 
 ```ruby
     class Article
@@ -691,12 +706,12 @@ Well, things stay mostly the same:
       # These Mongo guys sure do get funky with their IDs in +serializable_hash+, let's fix it.
       #
       def to_indexed_json
-        self.as_json
+        self.to_json
       end
 
     end
 
-    Article.create :title => 'I Love ElasticSearch'
+    Article.create :title => 'I Love Elasticsearch'
 
     Article.tire.search 'love'
 ```
@@ -711,9 +726,9 @@ database to store stuff like `{ :name => 'Tire', :tags => [ 'ruby', 'search' ] }
 Because all you need, really, is to just dump a JSON-representation of your data into a database and load it back again.
 Because you've noticed that _searching_ your data is a much more effective way of retrieval
 then constructing elaborate database query conditions.
-Because you have _lots_ of data and want to use _ElasticSearch's_ advanced distributed features.
+Because you have _lots_ of data and want to use _Elasticsearch's_ advanced distributed features.
 
-All good reasons to use _ElasticSearch_ as a schema-free and highly-scalable storage and retrieval/aggregation engine for your data.
+All good reasons to use _Elasticsearch_ as a schema-free and highly-scalable storage and retrieval/aggregation engine for your data.
 
 To use the persistence mode, we'll include the `Tire::Persistence` module in our class and define its properties;
 we can add the standard mapping declarations, set default values, or define casting for the property to create
@@ -747,7 +762,7 @@ and extensions to the core _Tire_ functionality — be sure to check them out.
 Other Clients
 -------------
 
-Check out [other _ElasticSearch_ clients](http://www.elasticsearch.org/guide/appendix/clients.html).
+Check out [other _Elasticsearch_ clients](http://www.elasticsearch.org/guide/clients/).
 
 
 Feedback

Rakefile

@@ -24,42 +24,15 @@ namespace :test do
   end
 end
 
-# Generate documentation
-begin
-  require 'rdoc'
-  require 'rdoc/task'
-  Rake::RDocTask.new do |rdoc|
-    rdoc.rdoc_dir = 'rdoc'
-    rdoc.title = "Tire"
-    rdoc.rdoc_files.include('README.markdown')
-    rdoc.rdoc_files.include('lib/**/*.rb')
-  end
-rescue LoadError
-  task :rdoc do
-    abort "[!] RDoc gem is not available."
-  end
-end
-
-# Generate coverage reports
-begin
-  require 'rcov/rcovtask'
-  Rcov::RcovTask.new do |test|
-    test.libs << 'test'
-    test.rcov_opts = ['--exclude', 'gems/*']
-    test.pattern = 'test/**/*_test.rb'
-    test.verbose = true
-  end
-rescue LoadError
-  task :rcov do
-    abort "[!] RCov gem is not available."
-  end
-end
-
 namespace :web do
 
   desc "Update the Github website"
   task :update => :generate do
-    current_branch = `git branch --no-color`.split("\n").select { |line| line =~ /^\* / }.to_s.gsub(/\* (.*)/, '\1')
+    current_branch = `git branch --no-color`.
+                       split("\n").
+                       select { |line| line =~ /^\* / }.
+                       first.to_s.
+                       gsub(/\* (.*)/, '\1')
     (puts "Unable to determine current branch"; exit(1) ) unless current_branch
     system "git checkout web"
     system "cp examples/tire-dsl.html index.html"