Every repository with this icon (
Every repository with this icon (
| Description: | Sphinx plugin for Rails and Merb edit |
-
0 comments Created about 1 month ago by martinemdedefine_index before a association is defined causes invalid sql but no errorbugxIf you have a define_index that runs before an indexed association is defined, TS quietly sets the index as if the association didn't exist and it was instead set on the model itself.
class Comment < AR:B define_index do indexes post.name end belongs_to :post endThat will result in comment_core sql that loads "comment.name" instead of "post.name". It would be good if this failed immediately.
I found a reference to this (I think) at this link: http://www.mail-archive.com/thinking-sphinx@googlegroups.com/msg02013.html
Comments
-
1 comment Created 25 days ago by akeiaabstract model class leads to sql-join with non-existent tablebugxif a model class has a parent class that is abstract (self.abstract_class = true) ts sees the class as full model class and tries to join with non-existent table in generated sql.
Comments
freelancing-god
Mon Nov 16 19:56:21 -0800 2009
| link
Might be able to look at this issue soon, so I just want to clarify: in which model is the define_index statement, what does it look like, and which associations used by the define_index block (if any) refer to models with abstract parents?
-
0 comments Created 18 days ago by matchuhas_many :through not interpreted correctlybugxThis is a re-post of a previous issue. I just now realized that the previous fix successfully removed the error, but still doesn't seem to return correct results when scoped.
# Model code... class Publication < ActiveRecord::Base has_many :issues has_many :articles, :through => :issues end class Issue < ActiveRecord::Base belongs_to :publication has_many :articles end class Article < ActiveRecord::Base belongs_to :issue define_index do has issue(:publication_id), :as => :publication_id end sphinx_scope(:web_content) { { :order => 'release_date DESC', :without => {:release => 0} # sphinx stores null as 0 } } endWhat I've observed is the following:
publication = Publication.find(1) publication.articles.search 'foobar' # returns only results in publication - correct behavior publication.articles.web_content # returns web content articles from both publications - incorrect behavior Article.web_content.search :with => {:publication_id => 1} # returns web content articles from publication 1 - correct behavior, and a functional workaround, but no funComments
-
Site.find(1).jobs.search_count('test) != Site.find(1).jobs.search('test').size
but
Site.find(1).jobs.search_count('test) = Job.search('test').sizeComments
-
It drops out with:
`require': no such file to load -- action_controller/dispatcher (MissingSourceFile)The problem appears in init.rb where you require it.
Comments
-
0 comments Created about 1 month ago by myronmarstonfeaturexIn a define_index block, provide a way to force a join in the generated SQLrestructurexThis is needed when you want to define an attribute using a SQL fragment that references a column in another table. Current work around is to define an attribute on a column in the other table using the association in your model, but this isn't a very elegant solution, especially when you don't want the additional attribute.
An example of a real-world need for this can be found in this post from the google group.
Comments
-
1 comment Created 4 months ago by seanabrahamsbugx:with time range not workingintermittentxI have some items that are updated periodically and I use the :with functionality to only show results from updated items. However, the behavior has been unpredictable and not working.
At times it returns no results when it should. Other times it returns results when it shouldn't.
define_index do indexes title, :sortable => :true indexes description indexes retailer(:name), :as => :retailer_name # attributes that are not strings has retailer_id, created_at, updated_at set_property :delta => true endand
Item.search 'search terms', :match_mode => :all, :field_weights => { :title => 20, :description => 5 }, :with => { :updated_at => 1.day.ago..Time.now }Comments
This may very well be time issue with sphinx and mysql. Sphinx converts all date times into a unix timestamp integer. Sphinx does this by using the UNIX_TIMESTAMP function in MySQL.
The caveat is that if your application is storing all times in UTC. The UNIX_TIMESTAMP function (http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html#function_unix-timestamp) interprets any date given in its current timezone (which is usually SYSTEM).
Long story short, it is taking an already converted UTC time and converting it again. The way to get around this is to specify a sql_query_pre option in the sphinx.yml file to set the timezone to UTC during indexing. That way the UNIX_TIMESTAMP function will make no UTC offset adjustments
sql_query_pre: - "SET TIME_ZONE = '+0:00'"
-
9 comments Created about 1 month ago by jamesefeaturexCalling Model.facets results in numerous SQL database queriesrestructurexCalling Model.facets results in SQL database queries even though it returns a hash of facets. Thinking Sphinx should be able to return the facets as stored in the Sphinx database.
Comments
Sorry, I just noticed issue #17. Maybe you could store the crc32, as well store a field with the value? Long shot I'm sure...
freelancing-god
Thu Oct 01 15:42:18 -0700 2009
| link
Not sure what you mean by storing the field... Sphinx doesn't return any field information, and the CRC is what is being returned, but we can't reverse that, so there's no way (that I can think of) for avoiding database calls.
Ideally, it should only make the call when it needs to, so there is room for improvement. I'll leave this ticket open as a reminder for that.
Ya, sorry that didn't make a lot of sense. I imagine this problem will be solved when Sphinx can store string attributes as strings (this is planned right?).
What I was thinking about and didn't write clearly is that you have this sphinx document data, so why not make it available. Then you could use the document data and not hit the database unless you really need to. So, say you search using the CRC, but then display another indexed string associated with the docid.
freelancing-god
Fri Oct 02 03:00:22 -0700 2009
| link
Yeah, string attributes are planned for 0.9.10, from what I've read - although plenty of people are still using 0.9.8 at the moment.
Still unclear on your suggestion though... what do you mean by 'another indexed string associated with the docid'?
I'm horrible at explaining things :) I'll try again...
Say you have the following document data in your sphinx database (using a Hash
to represent document fields/attributes):
[{:color_crc => "3632233996", :color_text => "Blue"}, {:color_crc => "3632233997", :color_text => "Indigo Blue"}, {:color_crc => "3632233998", :color_text => "Baby Blue"}]When you search or retrieve the facets, you do:
@results = Model.search(:with => "Blue".to_crc32) @facets = Model.search(@results.options)
In this case, you'd match the first document, which has {:color_text => "Blue"}
associated with it. What I'm wondering is, would it be possible to use
the associated document data instead of pulling everything from MySQL,
PostgreSQL, etc? So instead of performing a query to get "Blue", you could
just get it from the sphinx document by accessing :color_text?Maybe when defining your indexes you could do something like:
define_index do indexes colors.name, :as => :color_text has "CRC32(colors.name)", :as => :color, :type => :integer, :text => :color_text end
Am I totally mis-understanding how Sphinx works? Or, does this make any more
sense?
freelancing-god
Fri Oct 02 08:17:17 -0700 2009
| link
Okay, makes sense now :)
However, Sphinx doesn't return field information, it just returns attributes - and thus, no string data. Otherwise, would definitely have no need for any database queries in facet searching.
Interesting... so Sphinx is purposefully very dependent on SQL, both to index, and to provide the actual document data?
Well, I guess when Sphinx gets string attributes, it'll be a good day! Or, you could always try figuring out how to reverse CRC32 ;)
http://www.woodmann.com/fravia/crctut1.htm
freelancing-god
Fri Oct 02 09:03:52 -0700 2009
| link
Ouch, assembly code. Still, maybe I'll find the time and patience to convert it to Ruby :) thanks for the link.
And yeah, Sphinx is very much focused on returning a minimal data set for each result, and letting you decide what you want to do with it.
Just a different take on this. Since Thinking Sphinx is spending the time to query the database for the string data, could it make the model instance available instead of just a string? This may be desireable even when Sphinx does get string attributes. Take a Category model for example. If the instance is available, then you could get depth information. You could also store the id, but display something totally different.
-
Is there a way to reuse the same scope for search and for
facet search?Here is example what I mean (from online tutorial)..
scope = Article.latest_first.by_name('Puck')
articles = scope.search
facets = scope.facetsComments
datenimperator
Mon Aug 17 07:07:02 -0700 2009
| link
+1
That'd be great and very useful. In the meantime, the following seems to work without too much fuss:
_projects = Project.latest_first _projects = _projects.by_location(params[:location_name]) if params[:location_name] @facets = Project.facets @query, _projects.options @projects = _projects.search @query, :page => @page, :per_page => 10 -
Xapit has a very convenient solution for facet url creation, which allows you to get your faceting up and running extremely fast (it just works):
From http://github.com/ryanb/xapit
BEGIN QUOTE
Facets allow you to further filter the result set based on certain attributes.
<% for facet in @articles.facets %> <%= facet.name %> <% for option in facet.options %> <%= link_to option.name, :overwrite_params => { :facets => option } %> (<%= option.count %>) <% end %> <% end %>The to_param method is defined on option to return an identifier which will be passed through the URL. Use this in the search.
Article.search("phone", :facets => params[:facets])You can also list the applied facets along with a remove link.
<% for option in @articles.applied_facet_options %> <%=h option.name %> <%= link_to "remove", :overwrite_params => { :facets => option } %> <% end %>END QUOTE
Their to_param implementation allows it to work for both adding the facet if it doesn't exist, and if it does exist, then depending on breadcrumb_facets? will either remove just that facet, or remove all back to a given facet (breadcrumb_facets? == true):
http://github.com/ryanb/xapit/blob/master/lib/xapit/facet_option.rbOne downside to this implementation is that it doesn't allow you to have breadcrumb navigation links and a remove facet link.
Shop > Shoes (X) > Blue[breadcrumb mode] (X)[remove mode]
It would be nice to be able to configure this "on the fly":
<% for option in @articles.applied_facet_options %> <%= link_to option.name, :overwrite_params => { :facets => option } %> <%= link_to "remove", :overwrite_params => { :facets => option.remove } %> <% end %>Having the facet params encoded as a single param enables the breadcrumbs to be displayed in the order the user selects them. This means you don't have to use cookies, etc. to track this ordering. It also means that the controller code is very simple and just has to pass a single param into the Model.search options.
What do you think? :D
Comments
I don't know if this would provide the flexibility people would need in the long run. I'm beginning to think that maybe it would be better to just add more documentation to give people ideas, or a jumpstart. If you'd like to close this, I won't be offended. :)
-
2 comments Created 2 months ago by damianhamundefined method sphinx_scopeintermittentxI am getting an undefined method error for sphinx_scope
I am using ruby enterprise version 1.8.6-20090421this is what is in my environment.rb
config.gem( 'freelancing-god-thinking-sphinx', :lib => 'thinking_sphinx', :version => '1.2.9', :source => 'http://gems.github.com' )
config.gem 'mislav-will_paginate', :version => '~> 2.3.11', :lib => 'will_paginate',
:source => 'http://gems.github.com'here's my gem list actionmailer (2.3.3, 2.3.2)
actionpack (2.3.3, 2.3.2)
activerecord (2.3.3, 2.3.2)
activeresource (2.3.3, 2.3.2)
activesupport (2.3.3, 2.3.2)
capistrano (2.5.8)
cassandra (0.5.6.2)
cgi_multipart_eof_fix (2.5.0)
daemons (1.0.10)
fastthread (1.0.7)
freelancing-god-thinking-sphinx (1.2.9)
gem_plugin (0.2.3)
haml (2.2.3)
highline (1.5.1)
hpricot (0.8.1)
htmldoc (0.2.1)
httpclient (2.1.5.2)
image_science (1.2.1)
json (1.1.9)
memcached (0.16.3)
mislav-will_paginate (2.3.11)
mms2r (2.3.0)
mongrel (1.1.5)
mysql (2.7)
net-scp (1.0.2)
net-sftp (2.0.2)
net-ssh (2.0.15)
net-ssh-gateway (1.0.1)
passenger (2.2.5, 2.2.1)
postgres (0.7.9.2008.01.28)
rack (1.0.0, 0.9.1)
rails (2.3.3, 2.3.2)
rake (0.8.4)
RubyInline (3.8.3)
soap4r (1.5.8)
sqlite3-ruby (1.2.4)
thrift (0.0.810255.1)
tmail (1.2.3.1)
ZenTest (4.1.4)
and here's the stack trace
=> Booting Mongrel => Rails 2.3.3 application starting on http://0.0.0.0:3000 /opt/ruby-enterprise/lib/ruby/gems/1.8/gems/activerecord-2.3.3/lib/active_record/base.rb:1959:in
method_missing_without_paginate': undefined methodsphinx_scope' for #<Class:0x3913590> (NoMethodError)from /opt/ruby-enterprise/lib/ruby/gems/1.8/gems/mislav-will_paginate-2.3.11/lib/will_paginate/finder.rb:168:in `method_missing' from /media/disk/var/www/xoonoo.com/current/app/models/user.rb:98 from /opt/ruby-enterprise/lib/ruby/gems/1.8/gems/activesupport-2.3.3/lib/active_support/dependencies.rb:380:in `load_without_new_constant_marking' from /opt/ruby-enterprise/lib/ruby/gems/1.8/gems/activesupport-2.3.3/lib/active_support/dependencies.rb:380:in `load_file' from /opt/ruby-enterprise/lib/ruby/gems/1.8/gems/activesupport-2.3.3/lib/active_support/dependencies.rb:521:in `new_constants_in' from /opt/ruby-enterprise/lib/ruby/gems/1.8/gems/activesupport-2.3.3/lib/active_support/dependencies.rb:379:in `load_file' from /opt/ruby-enterprise/lib/ruby/gems/1.8/gems/activesupport-2.3.3/lib/active_support/dependencies.rb:259:in `require_or_load' from /opt/ruby-enterprise/lib/ruby/gems/1.8/gems/activesupport-2.3.3/lib/active_support/dependencies.rb:425:in `load_missing_constant' ... 35 levels... from /opt/ruby-enterprise/lib/ruby/gems/1.8/gems/rails-2.3.3/lib/commands/server.rb:84 from /opt/ruby-enterprise/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `gem_original_require' from /opt/ruby-enterprise/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `require' from script/server:3I had this working before but I have just moved to a new workstation and I am trying to setup my environment and now I get this error. Any ideas ?
Comments
freelancing-god
Fri Sep 11 07:55:28 -0700 2009
| link
Not sure what the cause is - is the new workstation the only change? There hasn't been any code changes? I've just tried on my relatively new Snow Leopard setup on Rails 2.3.3 and Mongrel, and it works fine, as far as I can tell.
Completely unrelated, but upgrading to 2.3.4 is recommended, there's a vulnerability in Rails in earlier 2.x versions.
-
It looks like TS 1.3.4 / Riddle 1.0.2 changed Riddle's escape pattern from:
/[()\|-!@~"&\/]/
to
/[()\|-!@~"&\/\\^\$=]/
... Why isn't "/" escaped anymore? Also, riddle.rb still has a line that has the old pattern too:
Thread.current[:riddle_escape_pattern] ||= /[()\|-!@~"&\/]/
This change is breaking all my searches that contain "/."
Comments
agibralter
Fri Nov 20 00:18:44 -0800 2009
| link
Err sorry, it does look from the regexp that it's still escaped... but for some reason it doesn't seem to be in my queries...
agibralter
Fri Nov 20 00:31:09 -0800 2009
| link
So when I search with a query that contains "/" TS tells me "Found results" (note the missing 0). If I search anything else "hello\goodbye", "hello&goodbye", etc. it works and says "Found 0 results"...
agibralter
Fri Nov 20 12:11:23 -0800 2009
| link
Sorry that was all late at night... I don't think my problem here is TS... Rather I need to figure out if I want to use Riddle.escape(). The one weird thing though is that for some reason Sphinx doesn't handle the "/" character nicely. All the other characters like "-", "+", etc. work correctly to remove search keywords and add search keywords (respectively), but "/" always returns a nil result set.











