Browse files

sphinx? did i hear a noise?

  • Loading branch information...
0 parents commit 582d15ce66db6bcc9f26e3caefe8f4ec84178c76 @evan committed May 8, 2007
Showing with 1,409 additions and 0 deletions.
  1. +184 −0 LICENSE
  2. +18 −0 README
  3. +2 −0 examples/app.multi
  4. +6 −0 init.rb
  5. +12 −0 lib/autoload.rb
  6. +67 −0 lib/fields.rb
  7. +22 −0 lib/is_indexed.rb
  8. +295 −0 lib/search.rb
  9. +27 −0 lib/spell.rb
  10. +218 −0 lib/ultrasphinx.rb
  11. +68 −0 tasks/ultrasphinx.rake
  12. +33 −0 vendor/README
  13. +457 −0 vendor/sphinx.rb
@@ -0,0 +1,184 @@
+Academic Free License (AFL) v. 3.0
+This Academic Free License (the "License") applies to any original work
+of authorship (the "Original Work") whose owner (the "Licensor") has
+placed the following licensing notice adjacent to the copyright notice
+for the Original Work:
+Licensed under the Academic Free License version 3.0
+1) Grant of Copyright License. Licensor grants You a worldwide,
+royalty-free, non-exclusive, sublicensable license, for the duration of
+the copyright, to do the following:
+a) to reproduce the Original Work in copies, either alone or as part of
+a collective work;
+b) to translate, adapt, alter, transform, modify, or arrange the
+Original Work, thereby creating derivative works ("Derivative Works")
+based upon the Original Work;
+c) to distribute or communicate copies of the Original Work and
+Derivative Works to the public, under any license of your choice that
+does not contradict the terms and conditions, including Licensor's
+reserved rights and remedies, in this Academic Free License;
+d) to perform the Original Work publicly; and
+e) to display the Original Work publicly.
+2) Grant of Patent License. Licensor grants You a worldwide,
+royalty-free, non-exclusive, sublicensable license, under patent claims
+owned or controlled by the Licensor that are embodied in the Original
+Work as furnished by the Licensor, for the duration of the patents, to
+make, use, sell, offer for sale, have made, and import the Original Work
+and Derivative Works.
+3) Grant of Source Code License. The term "Source Code" means the
+preferred form of the Original Work for making modifications to it and
+all available documentation describing how to modify the Original Work.
+Licensor agrees to provide a machine-readable copy of the Source Code of
+the Original Work along with each copy of the Original Work that
+Licensor distributes. Licensor reserves the right to satisfy this
+obligation by placing a machine-readable copy of the Source Code in an
+information repository reasonably calculated to permit inexpensive and
+convenient access by You for as long as Licensor continues to distribute
+the Original Work.
+4) Exclusions From License Grant. Neither the names of Licensor, nor the
+names of any contributors to the Original Work, nor any of their
+trademarks or service marks, may be used to endorse or promote products
+derived from this Original Work without express prior permission of the
+Licensor. Except as expressly stated herein, nothing in this License
+grants any license to Licensor's trademarks, copyrights, patents, trade
+secrets or any other intellectual property. No patent license is granted
+to make, use, sell, offer for sale, have made, or import embodiments of
+any patent claims other than the licensed claims defined in Section 2.
+No license is granted to the trademarks of Licensor even if such marks
+are included in the Original Work. Nothing in this License shall be
+interpreted to prohibit Licensor from licensing under terms different
+from this License any Original Work that Licensor otherwise would have a
+right to license.
+5) External Deployment. The term "External Deployment" means the use,
+distribution, or communication of the Original Work or Derivative Works
+in any way such that the Original Work or Derivative Works may be used
+by anyone other than You, whether those works are distributed or
+communicated to those persons or made available as an application
+intended for use over a network. As an express condition for the grants
+of license hereunder, You must treat any External Deployment by You of
+the Original Work or a Derivative Work as a distribution under section
+6) Attribution Rights. You must retain, in the Source Code of any
+Derivative Works that You create, all copyright, patent, or trademark
+notices from the Source Code of the Original Work, as well as any
+notices of licensing and any descriptive text identified therein as an
+"Attribution Notice." You must cause the Source Code for any Derivative
+Works that You create to carry a prominent Attribution Notice reasonably
+calculated to inform recipients that You have modified the Original
+7) Warranty of Provenance and Disclaimer of Warranty. Licensor warrants
+that the copyright in and to the Original Work and the patent rights
+granted herein by Licensor are owned by the Licensor or are sublicensed
+to You under the terms of this License with the permission of the
+contributor(s) of those copyrights and patent rights. Except as
+expressly stated in the immediately preceding sentence, the Original
+Work is provided under this License on an "AS IS" BASIS and WITHOUT
+WARRANTY, either express or implied, including, without limitation, the
+warranties of non-infringement, merchantability or fitness for a
+WORK IS WITH YOU. This DISCLAIMER OF WARRANTY constitutes an essential
+part of this License. No license to the Original Work is granted by this
+License except under this disclaimer.
+8) Limitation of Liability. Under no circumstances and under no legal
+theory, whether in tort (including negligence), contract, or otherwise,
+shall the Licensor be liable to anyone for any indirect, special,
+incidental, or consequential damages of any character arising as a
+result of this License or the use of the Original Work including,
+without limitation, damages for loss of goodwill, work stoppage,
+computer failure or malfunction, or any and all other commercial damages
+or losses. This limitation of liability shall not apply to the extent
+applicable law prohibits such limitation.
+9) Acceptance and Termination. If, at any time, You expressly assented
+to this License, that assent indicates your clear and irrevocable
+acceptance of this License and all of its terms and conditions. If You
+distribute or communicate copies of the Original Work or a Derivative
+Work, You must make a reasonable effort under the circumstances to
+obtain the express assent of recipients to the terms of this License.
+This License conditions your rights to undertake the activities listed
+in Section 1, including your right to create Derivative Works based upon
+the Original Work, and doing so without honoring these terms and
+conditions is prohibited by copyright law and international treaty.
+Nothing in this License is intended to affect copyright exceptions and
+limitations (including "fair use" or "fair dealing"). This License shall
+terminate immediately and You may no longer exercise any of the rights
+granted to You by this License upon your failure to honor the conditions
+in Section 1(c).
+10) Termination for Patent Action. This License shall terminate
+automatically and You may no longer exercise any of the rights granted
+to You by this License as of the date You commence an action, including
+a cross-claim or counterclaim, against Licensor or any licensee alleging
+that the Original Work infringes a patent. This termination provision
+shall not apply for an action alleging patent infringement by
+combinations of the Original Work with other software or hardware.
+11) Jurisdiction, Venue and Governing Law. Any action or suit relating
+to this License may be brought only in the courts of a jurisdiction
+wherein the Licensor resides or in which Licensor conducts its primary
+business, and under the laws of that jurisdiction excluding its
+conflict-of-law provisions. The application of the United Nations
+Convention on Contracts for the International Sale of Goods is expressly
+excluded. Any use of the Original Work outside the scope of this License
+or after its termination shall be subject to the requirements and
+penalties of copyright or patent law in the appropriate jurisdiction.
+This section shall survive the termination of this License.
+12) Attorneys' Fees. In any action to enforce the terms of this License
+or seeking damages relating thereto, the prevailing party shall be
+entitled to recover its costs and expenses, including, without
+limitation, reasonable attorneys' fees and costs incurred in connection
+with such action, including any appeal of such action. This section
+shall survive the termination of this License.
+13) Miscellaneous. If any provision of this License is held to be
+unenforceable, such provision shall be reformed only to the extent
+necessary to make it enforceable.
+14) Definition of "You" in This License. "You" throughout this License,
+whether in upper or lower case, means an individual or a legal entity
+exercising rights under, and complying with all of the terms of, this
+License. For legal entities, "You" includes any entity that controls, is
+controlled by, or is under common control with you. For purposes of this
+definition, "control" means (i) the power, direct or indirect, to cause
+the direction or management of such entity, whether by contract or
+otherwise, or (ii) ownership of fifty percent (50%) or more of the
+outstanding shares, or (iii) beneficial ownership of such entity.
+15) Right to Use. You may use the Original Work in all ways not
+otherwise restricted or conditioned by this License or by law, and
+Licensor promises not to interfere with or be responsible for such uses
+by You.
+16) Modification of This License. This License is Copyright (c) 2005
+Lawrence Rosen. Permission is granted to copy, distribute, or
+communicate this License without modification. Nothing in this License
+permits You to modify this License as applied to the Original Work or to
+Derivative Works. However, You may modify the text of this License and
+copy, distribute or communicate your modified version (the "Modified
+License") and apply it to other original works of authorship subject to
+the following conditions: (i) You may not indicate in any way that your
+Modified License is the "Academic Free License" or "AFL" and you may not
+use those names in the name of your Modified License; (ii) You must
+replace the notice specified in the first paragraph above with the
+notice "Licensed under <insert your license name here>" or with a notice
+of your own that is not confusingly similar to the notice in this
+License; and (iii) You may not claim that your original works are open
+source software unless your Modified License has been approved by Open
+Source Initiative (OSI) and You comply with its license review and
+certification process.
@@ -0,0 +1,18 @@
+Ruby on Rails interface to the Sphinx full text search engine.
+Copyright 2007 Cloudburst, LLC. Licensed under the AFL 3. See included
+LICENSE file. Portions copyright Dmytro Shteflyuk and Alexey Kovyrin;
+used with permission.
@@ -0,0 +1,2 @@
+add en_US-w_accents.multi
+add custom.rws
@@ -0,0 +1,6 @@
+require 'fields'
+require 'ultrasphinx'
+require 'autoload'
+require 'is_indexed'
+require 'search'
@@ -0,0 +1,12 @@
+require 'initializer'
+class Rails::Initializer
+ def after_initialize_with_ultrasphinx_configuration
+ after_initialize_without_ultrasphinx_configuration
+ Ultrasphinx.load_constants
+ end
+ alias_method_chain :after_initialize, :ultrasphinx_configuration
@@ -0,0 +1,67 @@
+module Ultrasphinx
+ class Fields < Hash
+ def initialize
+ self["class_id"] = "numeric"
+ self["class"] = "text"
+ end
+ def slam(field, new_type)
+ field, new_type = field.to_s, COLUMN_TYPES[new_type.to_sym]
+ if existing_type = self[field]
+ raise ConfigurationError, "Column type mismatch for #{field.inspect}" unless new_type == existing_type
+ else
+ self[field] = new_type
+ end
+ end
+ def cast(source_string, field)
+ if self[field] == "date"
+ "UNIX_TIMESTAMP(#{source_string})"
+ elsif source_string =~ /GROUP_CONCAT/
+ "CAST(#{source_string} AS CHAR)"
+ else
+ source_string
+ end + " AS #{field}"
+ end
+ def null(field)
+ case self[field]
+ when 'text'
+ "''"
+ when 'numeric'
+ "0"
+ when 'date'
+ "UNIX_TIMESTAMP('1970-01-01 00:00:00')"
+ end + " AS #{field}"
+ end
+ def configure(configuration)
+ configuration.each do |model, options|
+ klass = model.constantize
+ begin
+ # fields are from the model, renames are not allowed right now... use concat (vinegar)
+ options[:fields].to_a.each do |field|
+ klass.columns_hash[field] and slam(field, klass.columns_hash[field].type) or ActiveRecord::Base.logger.warn "ultrasphinx: WARNING: field #{field} is not present in #{model}"
+ end
+ # joins are whatever they are in the target
+ options[:includes].to_a.each do |join|
+ slam(join[:as] || join[:field], join[:model].constantize.columns_hash[join[:field]].type)
+ end
+ # regular concats are CHAR (I think), group_concats are BLOB and need to be cast to CHAR, e.g. :text
+ options[:concats].to_a.each do |concats|
+ slam(concats[:as], :text)
+ end
+ rescue ActiveRecord::StatementInvalid
+ ActiveRecord::Base.logger.warn "ultrasphinx: WARNING: model #{model} does not exist in the database yet"
+ end
+ end
+ self
+ end
+ end
@@ -0,0 +1,22 @@
+require 'ultrasphinx'
+require 'yaml'
+module ActiveRecord
+ class Base
+ def self.is_indexed opts = {}
+ opts.assert_valid_keys [:fields, :concats, :conditions, :includes, :nulls]
+ Array(opts[:concats]).each do |concat|
+ concat.assert_valid_keys [:model, :conditions, :field, :as, :fields, :association_name]
+ raise Ultrasphinx::ConfigurationError, "You can't mix regular concat and group concats" if concat[:fields] and (concat[:field] or concat[:model] or concat[:association_name])
+ raise Ultrasphinx::ConfigurationError, "Group concats must not have multiple fields" if concat[:field].is_a? Array
+ raise Ultrasphinx::ConfigurationError, "Regular concats should have multiple fields" if concat[:fields] and !concat[:fields].is_a?(Array)
+ end
+ Array(opts[:joins]).each do |join|
+ join.assert_valid_keys [:model, :field, :as]
+ end
+ Ultrasphinx::MODELS_CONF[] = opts
+ end
+ end
Oops, something went wrong. Retry.

0 comments on commit 582d15c

Please sign in to comment.