Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

Loading…

added support for mongoid #54

Open
wants to merge 15 commits into from

4 participants

@dcu
dcu commented

please let me know if you need something else

dcu added some commits
@dcu dcu add mongoid adapter
Signed-off-by: David A. Cuadrado <krawek@gmail.com>
8d5e989
@dcu dcu check whether active record or mongoid is included
Signed-off-by: David A. Cuadrado <krawek@gmail.com>
4344a39
@dcu dcu initialize mongoid documents correctly
Signed-off-by: David A. Cuadrado <krawek@gmail.com>
da8647e
@dcu dcu include mongoid adapter
Signed-off-by: David A. Cuadrado <krawek@gmail.com>
31d469c
@dcu dcu make use of <= operator
Signed-off-by: David A. Cuadrado <krawek@gmail.com>
6f44a94
@guilleiguaran guilleiguaran commented on the diff
lib/xapit/client/model_adapters/mongoid_adapter.rb
((15 lines not shown))
+ @model_class.after_destroy do |member|
+ member.class.xapit_index_builder.remove_document(member) if Xapit.config[:enabled]
+ end
+ end
+
+ def index_all
+ @model_class.all.each do |member|
+ member.class.xapit_index_builder.add_document(member)
+ end
+ end
+ end
+ end
+end
+
+if defined?(Mongoid::Document)
+ Mongoid::Document::ClassMethods.send(:include, Xapit::Client::Membership::ClassMethods)
@dcu
dcu added a note

why? I want those methods available on sinatra too, that's not rails exclusive

sounds good, then is a good idea move the AR one to active_record_adapter.rb

then is a good idea move the AR one to the adapter also :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
dcu and others added some commits
@dcu dcu update rack version
Signed-off-by: David A. Cuadrado <krawek@gmail.com>
c4b5006
@kuadrosx kuadrosx Prevent circular references to to_json method
Signed-off-by: Jorge H. Cuadrado <kuadrosxx@gmail.com>
dc6827c
@dcu dcu Merge branch 'rack13' 7fad283
@dcu dcu rack server must use the local database
Signed-off-by: David A. Cuadrado <krawek@gmail.com>
10c5d19
@dcu dcu improve text indexer
Signed-off-by: David A. Cuadrado <krawek@gmail.com>
13681d0
@dcu dcu allow to set the stemmer language by document
Signed-off-by: David A. Cuadrado <krawek@gmail.com>
cb2df2c
@dcu dcu include active support
Signed-off-by: David A. Cuadrado <krawek@gmail.com>
9394651
@dcu dcu indexing all should use the local database otherwise it would be very…
… slow

Signed-off-by: David A. Cuadrado <krawek@gmail.com>
b6a7a40
@patcito patcito added total_count alias f8fbce7
@dcu dcu Merge remote-tracking branch 'upstream/master'
Conflicts:
	lib/xapit.rb
	xapit.gemspec
cb30673
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Sep 28, 2011
  1. @dcu

    add mongoid adapter

    dcu authored
    Signed-off-by: David A. Cuadrado <krawek@gmail.com>
  2. @dcu

    check whether active record or mongoid is included

    dcu authored
    Signed-off-by: David A. Cuadrado <krawek@gmail.com>
  3. @dcu

    initialize mongoid documents correctly

    dcu authored
    Signed-off-by: David A. Cuadrado <krawek@gmail.com>
  4. @dcu

    include mongoid adapter

    dcu authored
    Signed-off-by: David A. Cuadrado <krawek@gmail.com>
  5. @dcu

    make use of <= operator

    dcu authored
    Signed-off-by: David A. Cuadrado <krawek@gmail.com>
Commits on Oct 3, 2011
  1. @dcu

    update rack version

    dcu authored
    Signed-off-by: David A. Cuadrado <krawek@gmail.com>
  2. @kuadrosx

    Prevent circular references to to_json method

    kuadrosx authored
    Signed-off-by: Jorge H. Cuadrado <kuadrosxx@gmail.com>
  3. @dcu

    Merge branch 'rack13'

    dcu authored
Commits on Oct 4, 2011
  1. @dcu

    rack server must use the local database

    dcu authored
    Signed-off-by: David A. Cuadrado <krawek@gmail.com>
Commits on Oct 6, 2011
  1. @dcu

    improve text indexer

    dcu authored
    Signed-off-by: David A. Cuadrado <krawek@gmail.com>
  2. @dcu

    allow to set the stemmer language by document

    dcu authored
    Signed-off-by: David A. Cuadrado <krawek@gmail.com>
  3. @dcu

    include active support

    dcu authored
    Signed-off-by: David A. Cuadrado <krawek@gmail.com>
  4. @dcu

    indexing all should use the local database otherwise it would be very…

    dcu authored
    … slow
    
    Signed-off-by: David A. Cuadrado <krawek@gmail.com>
Commits on Oct 10, 2011
  1. @patcito

    added total_count alias

    patcito authored
Commits on Jan 27, 2012
  1. @dcu

    Merge remote-tracking branch 'upstream/master'

    dcu authored
    Conflicts:
    	lib/xapit.rb
    	xapit.gemspec
This page is out of date. Refresh to see the latest.
View
6 lib/xapit.rb
@@ -3,6 +3,7 @@
require 'rack'
require 'json'
require 'net/http'
+require 'active_support'
require 'time'
module Xapit
@@ -29,9 +30,9 @@ def reload
@config.merge!(@loaded_config) if @loaded_config
end
- def database
+ def database(force_local = false)
raise Disabled, "Unable to access Xapit database because it is disabled in configuration." unless Xapit.config[:enabled]
- if config[:server]
+ if config[:server] && !force_local
@database ||= Xapit::Client::RemoteDatabase.new(config[:server])
elsif config[:read_only]
@database ||= Xapit::Server::ReadOnlyDatabase.new(config[:database_path])
@@ -140,3 +141,4 @@ def import_changes
require 'xapit/client/model_adapters/abstract_model_adapter'
require 'xapit/client/model_adapters/default_model_adapter'
require 'xapit/client/model_adapters/active_record_adapter'
+require 'xapit/client/model_adapters/mongoid_adapter'
View
5 lib/xapit/client/collection.rb
@@ -10,11 +10,11 @@ def initialize(clauses = [])
end
def in_classes(*classes)
- scope(:in_classes, classes)
+ scope(:in_classes, classes.map(&:to_s))
end
def not_in_classes(*classes)
- scope(:not_in_classes, classes)
+ scope(:not_in_classes, classes.map(&:to_s))
end
def search(phrase = nil)
@@ -89,6 +89,7 @@ def equal?(other)
def total_entries
query[:total].to_i
end
+ alias :total_count :total_entries
def current_page
(clause_value(:page) || 1).to_i
View
39 lib/xapit/client/index_builder.rb
@@ -2,8 +2,11 @@ module Xapit
module Client
class IndexBuilder
attr_reader :attributes
+ attr_reader :language
+
def initialize
@attributes = {}
+ @language = Xapit.config[:stemming]
end
def text(*args, &block)
@@ -24,20 +27,26 @@ def facet(name, custom_name = nil, &block)
add_attribute(:facet, name, options, &block)
end
- def add_document(member)
- Xapit.database.add_document(document_data(member))
+ def add_document(member, force_local = false)
+ Xapit.database(force_local).add_document(document_data(member))
end
- def remove_document(member)
- Xapit.database.remove_document(document_data(member))
+ def remove_document(member, force_local = false)
+ Xapit.database(force_local).remove_document(document_data(member))
end
- def update_document(member)
- Xapit.database.update_document(document_data(member))
+ def update_document(member, force_local = false)
+ Xapit.database(force_local).update_document(document_data(member))
end
def document_data(member)
- data = {:class => member.class.name, :id => member.id, :attributes => {}}
+ data = {
+ :class => member.class.name,
+ :id => member.id,
+ :attributes => {},
+ :language => find_language(member)
+ }
+
attributes.each do |name, options|
options = options.dup # so we can remove block without changing original hash
value = member.send(name)
@@ -53,8 +62,24 @@ def facets
end
end
+ def language(lang)
+ @language = lang
+ end
+
private
+ def find_language(member)
+ lang = if @language.kind_of?(Symbol)
+ member.send(@language)
+ elsif @language.kind_of?(Proc)
+ @language.call(member)
+ else
+ @language
+ end
+
+ lang || 'english'
+ end
+
def add_attribute(type, *args, &block)
options = args.last.kind_of?(Hash) ? args.pop : {}
args.each do |attribute|
View
4 lib/xapit/client/model_adapters/active_record_adapter.rb
@@ -2,7 +2,7 @@ module Xapit
module Client
class ActiveRecordAdapter < AbstractModelAdapter
def self.for_class?(model_class)
- model_class <= ActiveRecord::Base
+ defined?(ActiveRecord::Base) && model_class <= ActiveRecord::Base
end
def setup
@@ -19,7 +19,7 @@ def setup
def index_all
@model_class.find_each do |member|
- member.class.xapit_index_builder.add_document(member)
+ member.class.xapit_index_builder.add_document(member, true)
end
end
end
View
32 lib/xapit/client/model_adapters/mongoid_adapter.rb
@@ -0,0 +1,32 @@
+module Xapit
+ module Client
+ class MongoidAdapter < AbstractModelAdapter
+ def self.for_class?(model_class)
+ defined?(Mongoid::Document) && model_class <= Mongoid::Document
+ end
+
+ def setup
+ @model_class.after_create do |member|
+ member.class.xapit_index_builder.add_document(member) if Xapit.config[:enabled]
+ end
+ @model_class.after_update do |member|
+ member.class.xapit_index_builder.update_document(member) if Xapit.config[:enabled]
+ end
+ @model_class.after_destroy do |member|
+ member.class.xapit_index_builder.remove_document(member) if Xapit.config[:enabled]
+ end
+ end
+
+ def index_all
+ @model_class.all.each do |member|
+ member.class.xapit_index_builder.add_document(member, true)
+ end
+ end
+ end
+ end
+end
+
+if defined?(Mongoid::Document)
+ Mongoid::Document::ClassMethods.send(:include, Xapit::Client::Membership::ClassMethods)
@dcu
dcu added a note

why? I want those methods available on sinatra too, that's not rails exclusive

sounds good, then is a good idea move the AR one to active_record_adapter.rb

then is a good idea move the AR one to the adapter also :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
+end
+
View
2  lib/xapit/server/app.rb
@@ -21,7 +21,7 @@ def authorized_action(command, params)
def action(command, json)
data = Xapit.symbolize_keys(JSON.parse(json))
- render :content => Xapit.database.send(command, data).to_json
+ render :content => Xapit.database(true).send(command, data).to_json
end
def render(options = {})
View
46 lib/xapit/server/indexer.rb
@@ -43,19 +43,35 @@ def values
values
end
- # TODO refactor with stemmed_text_terms
def text_terms
+ terms = []
each_attribute(:text) do |name, value, options|
- value = value.to_s.split(/\s+/u).map { |w| w.gsub(/[^\w]/u, "") } unless value.kind_of? Array
- value.map(&:to_s).map(&:downcase).map do |term|
- [term, options[:weight] || 1] unless term.empty?
+ words = value
+ words = value.to_s.split(/\s+/u) unless value.kind_of? Array
+
+ weight = options[:weight] || 1
+ words.each do |word|
+ next if word.empty? || word.bytesize >= 245
+
+ word.downcase!
+
+ terms << [word, weight]
+ terms << stemmed_term(word, weight)
+
+ clean_word = ActiveSupport::Multibyte::Chars.new(word).normalize(:kd).gsub(/[^\w]/u, "").to_s
+ if !clean_word.empty? && clean_word != word
+ terms << [clean_word, weight]
+ terms << stemmed_term(clean_word, weight)
+ end
end
- end.flatten(1).compact
+ end
+
+ terms.uniq
end
# TODO refactor with stemmed_text_terms
def stemmed_text_terms
- if stemmer
+ if stemmer = build_stemmer
each_attribute(:text) do |name, value, options|
value = value.to_s.split(/\s+/u).map { |w| w.gsub(/[^\w]/u, "") } unless value.kind_of? Array
value.map(&:to_s).map(&:downcase).map do |term|
@@ -94,8 +110,22 @@ def save_facets
private
- def stemmer
- @stemmer ||= Xapian::Stem.new(Xapit.config[:stemming]) if Xapit.config[:stemming]
+ def build_stemmer
+ begin
+ Xapian::Stem.new(@data[:language])
+ rescue ArgumentError
+ return nil
+ end
+ end
+
+ def stemmed_term(word, weight = 1)
+ term = [word, weight]
+ if stemmer = build_stemmer
+ stemmed = stemmer.call(word)
+ term = ["Z#{word}", weight] if stemmed != word
+ end
+
+ term
end
def base_terms
View
2  xapit.gemspec
@@ -10,7 +10,7 @@ Gem::Specification.new do |s|
s.files = Dir["{lib,spec,features,rails_generators,tasks}/**/*", "[A-Z]*", "init.rb", "install.rb", "uninstall.rb"] - ["Gemfile.lock"]
s.require_path = "lib"
- s.add_dependency 'rack'
+ s.add_dependency 'rack', '~> 1.3.4'
s.add_development_dependency 'rspec', '~> 2.7.0'
s.add_development_dependency 'cucumber', '~> 0.10.2'
Something went wrong with that request. Please try again.