Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP
Browse files

Removing datetime deltas, finishing changes for 1.3.0

  • Loading branch information...
commit 4f1ab0886ea01d45502468ca5b6c82e91388570c 1 parent 8cb1de8
@pat authored
View
2  cucumber.yml
@@ -1 +1 @@
-default: "--require features/support/env.rb --require features/step_definitions/alpha_steps.rb --require features/step_definitions/beta_steps.rb --require features/step_definitions/common_steps.rb --require features/step_definitions/datetime_delta_steps.rb --require features/step_definitions/delayed_delta_indexing_steps.rb --require features/step_definitions/extensible_delta_indexing_steps.rb --require features/step_definitions/facet_steps.rb --require features/step_definitions/find_arguments_steps.rb --require features/step_definitions/gamma_steps.rb --require features/step_definitions/scope_steps.rb --require features/step_definitions/search_steps.rb --require features/step_definitions/sphinx_steps.rb"
+default: "--require features/support/env.rb --require features/step_definitions/alpha_steps.rb --require features/step_definitions/beta_steps.rb --require features/step_definitions/common_steps.rb --require features/step_definitions/extensible_delta_indexing_steps.rb --require features/step_definitions/facet_steps.rb --require features/step_definitions/find_arguments_steps.rb --require features/step_definitions/gamma_steps.rb --require features/step_definitions/scope_steps.rb --require features/step_definitions/search_steps.rb --require features/step_definitions/sphinx_steps.rb"
View
66 features/datetime_deltas.feature
@@ -1,66 +0,0 @@
-Feature: Datetime Delta Indexing
- In order to have delta indexing on frequently-updated sites
- Developers
- Should be able to use an existing datetime column to track changes
-
- Scenario: Delta Index should not fire automatically
- Given Sphinx is running
- And I am searching on thetas
- When I search for one
- Then I should get 1 result
-
- When I change the name of theta one to eleven
- And I wait for Sphinx to catch up
- And I search for one
- Then I should get 1 result
-
- When I search for eleven
- Then I should get 0 results
-
- Scenario: Delta Index should fire when jobs are run
- Given Sphinx is running
- And I am searching on thetas
- When I search for two
- Then I should get 1 result
-
- When I change the name of theta two to twelve
- And I wait for Sphinx to catch up
- And I search for twelve
- Then I should get 0 results
-
- When I index the theta datetime delta
- And I wait for Sphinx to catch up
- And I search for twelve
- Then I should get 1 result
-
- When I search for two
- Then I should get 0 results
-
- Scenario: New records should be merged into the core index
- Given Sphinx is running
- And I am searching on thetas
- When I search for thirteen
- Then I should get 0 results
-
- When I create a new theta named thirteen
- And I search for thirteen
- Then I should get 0 results
-
- When I index the theta datetime delta
- And I wait for Sphinx to catch up
- And I search for thirteen
- Then I should get 1 result
-
- When I search for the document id of theta thirteen in the theta_core index
- Then it should exist
-
- Scenario: Deleting records
- Given Sphinx is running
- And I am searching on thetas
- When I search for three
- Then I should get 1 result
-
- When I delete the theta named three
- And I wait for Sphinx to catch up
- And I search for three
- Then I should get 0 results
View
2  features/searching_across_models.feature
@@ -17,4 +17,4 @@ Feature: Searching across multiple model
Scenario: Retrieving results from multiple models
Given Sphinx is running
When I search for ten
- Then I should get 6 results
+ Then I should get 4 results
View
1  features/step_definitions/common_steps.rb
@@ -1,6 +1,5 @@
Before do
$queries_executed = []
- ThinkingSphinx::Deltas::Job.cancel_thinking_sphinx_jobs
@model = nil
@method = :search
View
15 features/step_definitions/datetime_delta_steps.rb
@@ -1,15 +0,0 @@
-When /^I index the theta datetime delta$/ do
- Theta.sphinx_indexes.first.delta_object.delayed_index(Theta)
-end
-
-When /^I change the name of theta (\w+) to (\w+)$/ do |current, replacement|
- Theta.find_by_name(current).update_attributes(:name => replacement)
-end
-
-When /^I create a new theta named (\w+)$/ do |name|
- Theta.create(:name => name)
-end
-
-When /^I delete the theta named (\w+)$/ do |name|
- Theta.find_by_name(name).destroy
-end
View
10 features/support/db/fixtures/thetas.rb
@@ -1,10 +0,0 @@
-Theta.create :name => "one"
-Theta.create :name => "two"
-Theta.create :name => "three"
-Theta.create :name => "four"
-Theta.create :name => "five"
-Theta.create :name => "six"
-Theta.create :name => "seven"
-Theta.create :name => "eight"
-Theta.create :name => "nine"
-Theta.create :name => "ten"
View
5 features/support/db/migrations/create_thetas.rb
@@ -1,5 +0,0 @@
-ActiveRecord::Base.connection.create_table :thetas, :force => true do |t|
- t.column :name, :string, :null => false
- t.column :created_at, :datetime, :null => false
- t.column :updated_at, :datetime, :null => false
-end
View
7 features/support/models/theta.rb
@@ -1,7 +0,0 @@
-class Theta < ActiveRecord::Base
- define_index do
- indexes :name, :sortable => true
-
- set_property :delta => :datetime, :threshold => 1.hour
- end
-end
View
17 install.rb
@@ -1,18 +1,5 @@
puts <<-MESSAGE
-With the release of Thinking Sphinx 1.1.18, there is one important change to
-note: previously, the default morphology for indexing was 'stem_en'. The new
-default is nothing, to avoid any unexpected behavior. If you wish to keep the
-old value though, you will need to add the following settings to your
-config/sphinx.yml file:
-
-development:
- morphology: stem_en
-test:
- morphology: stem_en
-production:
- morphology: stem_en
-
-To understand morphologies/stemmers better, visit the following link:
-http://www.sphinxsearch.com/docs/manual-0.9.8.html#conf-morphology
+If you're upgrading, you should read this:
+http://freelancing-god.github.com/ts/en/upgrading.html
MESSAGE
View
2  lib/thinking_sphinx/deltas.rb
@@ -1,6 +1,4 @@
require 'thinking_sphinx/deltas/default_delta'
-require 'thinking_sphinx/deltas/delayed_delta'
-require 'thinking_sphinx/deltas/datetime_delta'
module ThinkingSphinx
module Deltas
View
50 lib/thinking_sphinx/deltas/datetime_delta.rb
@@ -1,50 +0,0 @@
-module ThinkingSphinx
- module Deltas
- class DatetimeDelta < ThinkingSphinx::Deltas::DefaultDelta
- attr_accessor :column, :threshold
-
- def initialize(index, options)
- @index = index
- @column = options.delete(:delta_column) || :updated_at
- @threshold = options.delete(:threshold) || 1.day
- end
-
- def index(model, instance = nil)
- # do nothing
- true
- end
-
- def delayed_index(model)
- config = ThinkingSphinx::Configuration.instance
- rotate = ThinkingSphinx.sphinx_running? ? "--rotate" : ""
-
- output = `#{config.bin_path}#{config.indexer_binary_name} --config #{config.config_file} #{rotate} #{delta_index_name model}`
- output += `#{config.bin_path}#{config.indexer_binary_name} --config #{config.config_file} #{rotate} --merge #{core_index_name model} #{delta_index_name model} --merge-dst-range sphinx_deleted 0 0`
- puts output unless ThinkingSphinx.suppress_delta_output?
-
- true
- end
-
- def toggle(instance)
- # do nothing
- end
-
- def toggled(instance)
- instance.send(@column) > @threshold.ago
- end
-
- def reset_query(model)
- nil
- end
-
- def clause(model, toggled)
- if toggled
- "#{model.quoted_table_name}.#{model.connection.quote_column_name(@column.to_s)}" +
- " > #{adapter.time_difference(@threshold)}"
- else
- nil
- end
- end
- end
- end
-end
View
34 lib/thinking_sphinx/tasks.rb
@@ -65,14 +65,12 @@
desc "Index data for Sphinx using Thinking Sphinx's settings"
task :index => :app_env do
- ThinkingSphinx::Deltas::Job.cancel_thinking_sphinx_jobs
-
config = ThinkingSphinx::Configuration.instance
unless ENV["INDEX_ONLY"] == "true"
puts "Generating Configuration to #{config.config_file}"
config.build
end
-
+
FileUtils.mkdir_p config.searchd_file_path
cmd = "#{config.bin_path}#{config.indexer_binary_name} --config \"#{config.config_file}\" --all"
cmd << " --rotate" if sphinx_running?
@@ -86,30 +84,6 @@
Rake::Task["thinking_sphinx:index"].invoke
Rake::Task["thinking_sphinx:start"].invoke
end
-
- namespace :index do
- task :delta => :app_env do
- ThinkingSphinx.indexed_models.select { |model|
- model.constantize.sphinx_indexes.any? { |index| index.delta? }
- }.each do |model|
- model.constantize.sphinx_indexes.select { |index|
- index.delta? && index.delta_object.respond_to?(:delayed_index)
- }.each { |index|
- index.delta_object.delayed_index(index.model)
- }
- end
- end
- end
-
- desc "Process stored delta index requests"
- task :delayed_delta => :app_env do
- require 'delayed/worker'
-
- Delayed::Worker.new(
- :min_priority => ENV['MIN_PRIORITY'],
- :max_priority => ENV['MAX_PRIORITY']
- ).start
- end
end
namespace :ts do
@@ -123,10 +97,6 @@
task :stop => "thinking_sphinx:stop"
desc "Index data for Sphinx using Thinking Sphinx's settings"
task :in => "thinking_sphinx:index"
- namespace :in do
- desc "Index Thinking Sphinx datetime delta indexes"
- task :delta => "thinking_sphinx:index:delta"
- end
task :index => "thinking_sphinx:index"
desc "Restart Sphinx"
task :restart => "thinking_sphinx:restart"
@@ -136,8 +106,6 @@
task :config => "thinking_sphinx:configure"
desc "Stop Sphinx (if it's running), rebuild the indexes, and start Sphinx"
task :rebuild => "thinking_sphinx:rebuild"
- desc "Process stored delta index requests"
- task :dd => "thinking_sphinx:delayed_delta"
end
def sphinx_pid
View
17 tasks/distribution.rb
@@ -33,21 +33,8 @@
gem.add_dependency 'activerecord', '>= 1.15.6'
gem.post_install_message = <<-MESSAGE
-With the release of Thinking Sphinx 1.1.18, there is one important change to
-note: previously, the default morphology for indexing was 'stem_en'. The new
-default is nil, to avoid any unexpected behavior. If you wish to keep the old
-value though, you will need to add the following settings to your
-config/sphinx.yml file:
-
-development:
- morphology: stem_en
-test:
- morphology: stem_en
-production:
- morphology: stem_en
-
-To understand morphologies/stemmers better, visit the following link:
-http://www.sphinxsearch.com/docs/manual-0.9.8.html#conf-morphology
+If you're upgrading, you should read this:
+http://freelancing-god.github.com/ts/en/upgrading.html
MESSAGE
end
View
41 thinking-sphinx.gemspec
@@ -5,11 +5,11 @@
Gem::Specification.new do |s|
s.name = %q{thinking-sphinx}
- s.version = "1.2.13"
+ s.version = "1.3.0"
s.required_rubygems_version = Gem::Requirement.new(">= 0") if s.respond_to? :required_rubygems_version=
s.authors = ["Pat Allan"]
- s.date = %q{2009-11-01}
+ s.date = %q{2009-11-04}
s.description = %q{A concise and easy-to-use Ruby library that connects ActiveRecord to the Sphinx search daemon, managing configuration, indexing and searching.}
s.email = %q{pat@freelancing-gods.com}
s.extra_rdoc_files = [
@@ -37,12 +37,7 @@ Gem::Specification.new do |s|
"lib/thinking_sphinx/core/array.rb",
"lib/thinking_sphinx/core/string.rb",
"lib/thinking_sphinx/deltas.rb",
- "lib/thinking_sphinx/deltas/datetime_delta.rb",
"lib/thinking_sphinx/deltas/default_delta.rb",
- "lib/thinking_sphinx/deltas/delayed_delta.rb",
- "lib/thinking_sphinx/deltas/delayed_delta/delta_job.rb",
- "lib/thinking_sphinx/deltas/delayed_delta/flag_as_deleted_job.rb",
- "lib/thinking_sphinx/deltas/delayed_delta/job.rb",
"lib/thinking_sphinx/deploy/capistrano.rb",
"lib/thinking_sphinx/excerpter.rb",
"lib/thinking_sphinx/facet.rb",
@@ -71,10 +66,6 @@ Gem::Specification.new do |s|
"vendor/after_commit/lib/after_commit/active_record.rb",
"vendor/after_commit/lib/after_commit/connection_adapters.rb",
"vendor/after_commit/test/after_commit_test.rb",
- "vendor/delayed_job/lib/delayed/job.rb",
- "vendor/delayed_job/lib/delayed/message_sending.rb",
- "vendor/delayed_job/lib/delayed/performable_method.rb",
- "vendor/delayed_job/lib/delayed/worker.rb",
"vendor/riddle/lib/riddle.rb",
"vendor/riddle/lib/riddle/client.rb",
"vendor/riddle/lib/riddle/client/filter.rb",
@@ -93,21 +84,8 @@ Gem::Specification.new do |s|
"vendor/riddle/lib/riddle/controller.rb"
]
s.homepage = %q{http://ts.freelancing-gods.com}
- s.post_install_message = %q{With the release of Thinking Sphinx 1.1.18, there is one important change to
-note: previously, the default morphology for indexing was 'stem_en'. The new
-default is nil, to avoid any unexpected behavior. If you wish to keep the old
-value though, you will need to add the following settings to your
-config/sphinx.yml file:
-
-development:
- morphology: stem_en
-test:
- morphology: stem_en
-production:
- morphology: stem_en
-
-To understand morphologies/stemmers better, visit the following link:
-http://www.sphinxsearch.com/docs/manual-0.9.8.html#conf-morphology
+ s.post_install_message = %q{If you're upgrading, you should read this:
+http://freelancing-god.github.com/ts/en/upgrading.html
}
s.rdoc_options = ["--charset=UTF-8"]
@@ -118,8 +96,6 @@ http://www.sphinxsearch.com/docs/manual-0.9.8.html#conf-morphology
"features/alternate_primary_key.feature",
"features/attribute_transformation.feature",
"features/attribute_updates.feature",
- "features/datetime_deltas.feature",
- "features/delayed_delta_indexing.feature",
"features/deleting_instances.feature",
"features/direct_attributes.feature",
"features/excerpts.feature",
@@ -137,8 +113,6 @@ http://www.sphinxsearch.com/docs/manual-0.9.8.html#conf-morphology
"features/step_definitions/alpha_steps.rb",
"features/step_definitions/beta_steps.rb",
"features/step_definitions/common_steps.rb",
- "features/step_definitions/datetime_delta_steps.rb",
- "features/step_definitions/delayed_delta_indexing_steps.rb",
"features/step_definitions/extensible_delta_indexing_steps.rb",
"features/step_definitions/facet_steps.rb",
"features/step_definitions/find_arguments_steps.rb",
@@ -161,7 +135,6 @@ http://www.sphinxsearch.com/docs/manual-0.9.8.html#conf-morphology
"features/support/db/fixtures/categories.rb",
"features/support/db/fixtures/cats.rb",
"features/support/db/fixtures/comments.rb",
- "features/support/db/fixtures/delayed_betas.rb",
"features/support/db/fixtures/developers.rb",
"features/support/db/fixtures/dogs.rb",
"features/support/db/fixtures/extensible_betas.rb",
@@ -170,7 +143,6 @@ http://www.sphinxsearch.com/docs/manual-0.9.8.html#conf-morphology
"features/support/db/fixtures/posts.rb",
"features/support/db/fixtures/robots.rb",
"features/support/db/fixtures/tags.rb",
- "features/support/db/fixtures/thetas.rb",
"features/support/db/migrations",
"features/support/db/migrations/create_alphas.rb",
"features/support/db/migrations/create_animals.rb",
@@ -180,7 +152,6 @@ http://www.sphinxsearch.com/docs/manual-0.9.8.html#conf-morphology
"features/support/db/migrations/create_boxes.rb",
"features/support/db/migrations/create_categories.rb",
"features/support/db/migrations/create_comments.rb",
- "features/support/db/migrations/create_delayed_betas.rb",
"features/support/db/migrations/create_developers.rb",
"features/support/db/migrations/create_extensible_betas.rb",
"features/support/db/migrations/create_gammas.rb",
@@ -189,7 +160,6 @@ http://www.sphinxsearch.com/docs/manual-0.9.8.html#conf-morphology
"features/support/db/migrations/create_robots.rb",
"features/support/db/migrations/create_taggings.rb",
"features/support/db/migrations/create_tags.rb",
- "features/support/db/migrations/create_thetas.rb",
"features/support/db/mysql.rb",
"features/support/db/postgresql.rb",
"features/support/env.rb",
@@ -204,7 +174,6 @@ http://www.sphinxsearch.com/docs/manual-0.9.8.html#conf-morphology
"features/support/models/cat.rb",
"features/support/models/category.rb",
"features/support/models/comment.rb",
- "features/support/models/delayed_beta.rb",
"features/support/models/developer.rb",
"features/support/models/dog.rb",
"features/support/models/extensible_beta.rb",
@@ -214,7 +183,6 @@ http://www.sphinxsearch.com/docs/manual-0.9.8.html#conf-morphology
"features/support/models/robot.rb",
"features/support/models/tag.rb",
"features/support/models/tagging.rb",
- "features/support/models/theta.rb",
"features/support/post_database.rb",
"spec/lib/thinking_sphinx/active_record/delta_spec.rb",
"spec/lib/thinking_sphinx/active_record/has_many_association_spec.rb",
@@ -225,7 +193,6 @@ http://www.sphinxsearch.com/docs/manual-0.9.8.html#conf-morphology
"spec/lib/thinking_sphinx/configuration_spec.rb",
"spec/lib/thinking_sphinx/core/array_spec.rb",
"spec/lib/thinking_sphinx/core/string_spec.rb",
- "spec/lib/thinking_sphinx/deltas/job_spec.rb",
"spec/lib/thinking_sphinx/excerpter_spec.rb",
"spec/lib/thinking_sphinx/facet_search_spec.rb",
"spec/lib/thinking_sphinx/facet_spec.rb",
Please sign in to comment.
Something went wrong with that request. Please try again.