This repository has been archived by the owner. It is now read-only.

trying to add filters and sorts #101

Closed
michelson opened this Issue Sep 12, 2011 · 7 comments

Comments

Projects
None yet
3 participants
@michelson

michelson commented Sep 12, 2011

Hi , im trying to add filters and sorts,

im doing this in an activerecord model

Cause.rb

mapping do
  indexes :user, :type => 'string'
  indexes :civil do
    indexes :rol, :type => 'string'
    indexes :tribunal,  :type => 'string', :boost => 100
  end
end

def to_indexed_json
  {
    :user => user.full_name,
    :civil  => {
      :tribunal => causable.tribunal.name,
      :rol         => causable.rol
    }
  }.to_json
end

if search this way it works

Cause.search do
   query  { string terms }
end

but if i try to filter by user i get 0 records

Cause.search do
  query  { string "1234" }
  filter :terms, :user => 'michelson'
end

and if i try to sort by user i get an error

 sort   { by :user, 'desc' }

 [REQUEST FAILED] curl -X GET "http://localhost:9200/causes/cause/_search?pretty=true" -d '{"sort"[{"user":"desc"}],"query":  {"query_string":{"query":"c"}}}'
 NoMethodError: undefined method `results' for false:FalseClass

im doing somthing wrong here?

regards

@karmi

This comment has been minimized.

Show comment
Hide comment
@karmi

karmi Sep 12, 2011

Owner

Hi, yes, there's several things going on :)

  1. You can sort only on "not analyzed" fields. So, either set the field to "not analyzed", or, if that would negatively impact searchability, use a multifield type.
  2. The terms filter expects an array of values. Either pass an Array with single value (['michelson']) or use the term filter.

I can sorta replicate your use-case -- note that I'm using the "Persistence" feature, just ignore the property stuff:

require 'rubygems'
require 'tire'

Tire.configure { logger STDERR }

Tire.index('causes') { delete }

class Cause
  include Tire::Model::Persistence
  include Tire::Model::Search
  include Tire::Model::Callbacks

  mapping do
    indexes :user, :index => 'not_analyzed'
    indexes :civil do
      indexes :rol
      indexes :tribunal, :boost => 100
    end
  end

  property :user
  property :civil

end

Cause.create :id => 1, :user => 'Michelson',  :civil => { :rol => 'Foo', :tribunal => 'Bar' }
Cause.create :id => 2, :user => 'Richardson', :civil => { :rol => 'Foo', :tribunal => 'Bar' }
Cause.index.refresh

s = Cause.search do
  query  { string "foo" }
  filter :term, :user => 'Michelson'
  sort { by :user, 'desc' }
end

p s.results

Let me know if this helps.

Owner

karmi commented Sep 12, 2011

Hi, yes, there's several things going on :)

  1. You can sort only on "not analyzed" fields. So, either set the field to "not analyzed", or, if that would negatively impact searchability, use a multifield type.
  2. The terms filter expects an array of values. Either pass an Array with single value (['michelson']) or use the term filter.

I can sorta replicate your use-case -- note that I'm using the "Persistence" feature, just ignore the property stuff:

require 'rubygems'
require 'tire'

Tire.configure { logger STDERR }

Tire.index('causes') { delete }

class Cause
  include Tire::Model::Persistence
  include Tire::Model::Search
  include Tire::Model::Callbacks

  mapping do
    indexes :user, :index => 'not_analyzed'
    indexes :civil do
      indexes :rol
      indexes :tribunal, :boost => 100
    end
  end

  property :user
  property :civil

end

Cause.create :id => 1, :user => 'Michelson',  :civil => { :rol => 'Foo', :tribunal => 'Bar' }
Cause.create :id => 2, :user => 'Richardson', :civil => { :rol => 'Foo', :tribunal => 'Bar' }
Cause.index.refresh

s = Cause.search do
  query  { string "foo" }
  filter :term, :user => 'Michelson'
  sort { by :user, 'desc' }
end

p s.results

Let me know if this helps.

@vhyza

This comment has been minimized.

Show comment
Hide comment
@vhyza

vhyza Sep 12, 2011

Collaborator

Hello,

here is example how can be multifield type used:

require 'rubygems'
require 'tire'

Tire.configure { logger STDERR }
Tire.index("causes").delete

class Cause
  # Only for this test
  include Tire::Model::Persistence

  include Tire::Model::Search

  mapping do
    indexes :user, :type => 'multi_field', :fields => {
      :user => { :type => "string", :analyzer => "snowball" },
      :"user.exact" => { :type => "string", :index => :not_analyzed }
    }
    indexes :civil do
      indexes :rol, :type => 'string'
      indexes :tribunal,  :type => 'string', :boost => 100
    end
  end


  # Only for this test
  property :user
  property :civil

end

Cause.create :user => "first name"
Cause.create :user => "last name"
Cause.index.refresh

response = Cause.search do
   query  { string 'name' }
   filter :term, :"user.exact" => 'last name'
   sort   { by "user.exact", 'desc' }
end

p response.results

# returns nothing because term filter is not analyzed
response = Cause.search do
   query  { string 'name' }
   filter :term, :user => 'last name'
   sort   { by "user.exact", 'desc' }
end

p response.results

# can't sort by analyzed property
response = Cause.search do
   query  { string 'name' }
   filter :term, :user => 'last name'
   sort   { by "user", 'desc' }
end
Collaborator

vhyza commented Sep 12, 2011

Hello,

here is example how can be multifield type used:

require 'rubygems'
require 'tire'

Tire.configure { logger STDERR }
Tire.index("causes").delete

class Cause
  # Only for this test
  include Tire::Model::Persistence

  include Tire::Model::Search

  mapping do
    indexes :user, :type => 'multi_field', :fields => {
      :user => { :type => "string", :analyzer => "snowball" },
      :"user.exact" => { :type => "string", :index => :not_analyzed }
    }
    indexes :civil do
      indexes :rol, :type => 'string'
      indexes :tribunal,  :type => 'string', :boost => 100
    end
  end


  # Only for this test
  property :user
  property :civil

end

Cause.create :user => "first name"
Cause.create :user => "last name"
Cause.index.refresh

response = Cause.search do
   query  { string 'name' }
   filter :term, :"user.exact" => 'last name'
   sort   { by "user.exact", 'desc' }
end

p response.results

# returns nothing because term filter is not analyzed
response = Cause.search do
   query  { string 'name' }
   filter :term, :user => 'last name'
   sort   { by "user.exact", 'desc' }
end

p response.results

# can't sort by analyzed property
response = Cause.search do
   query  { string 'name' }
   filter :term, :user => 'last name'
   sort   { by "user", 'desc' }
end
@michelson

This comment has been minimized.

Show comment
Hide comment
@michelson

michelson Sep 13, 2011

thanks for your replies,

i can get the query to work with a filter but i got some problems with
sorting

i hve tried many ways but i get an error

in the search

a = Cause.search do
query { string terms }
filter :term, :user_id => "1"
#sort { by "tribunal.exact", 'desc' }
#sort { by "user.exact", 'desc' }
end

a.results

it return the collection but when i call a.results it throws an AR exeption
ActiveRecord::UnknownAttributeError: unknown attribute: caratulado

also if i try the sort methods above i get a curl request fail,
NoMethodError: undefined method `results' for false:FalseClass

i have this mapping in the model

mapping do
indexes :user_id, :type => 'string', :index => :not_analyzed

indexes :user, :type => 'multi_field', :fields => {
  :user => { :type => "string", :analyzer => "snowball" },
  :"user.exact" => { :type => "string", :index => :not_analyzed }
}

indexes :tribunal, :type => 'multi_field', :fields => {
  :tribunal => { :type => "string", :analyzer => "snowball" },
  :"tribunal.exact" => { :type => "string", :index => :not_analyzed }
}

indexes :civil do
  indexes :rol, :type => 'string'
  indexes :caratulado ,  :type => 'string', :boost => 100
end

end

really lost here

regards

Atte.
Miguel Michelson Martinez
www.artenlinea.com

On Mon, Sep 12, 2011 at 5:36 AM, Vojtěch Hýža <
reply@reply.github.com>wrote:

Hello,

here is example how can be multifield type used:

require 'rubygems'
require 'tire'

Tire.configure { logger STDERR }
Tire.index("causes").delete

class Cause
 include Tire::Model::Persistence
 include Tire::Model::Search

  mapping do
   indexes :user, :type => 'multi_field', :fields => {
     :user => { :type => "string", :analyzer => "snowball" },
     :"user.exact" => { :type => "string", :index => :not_analyzed }
   }
   indexes :civil do
     indexes :rol, :type => 'string'
     indexes :tribunal,  :type => 'string', :boost => 100
   end
 end


 # Only for this test
  property :user
 property :civil

end

Cause.create :user => "first name"
Cause.create :user => "last name"
Cause.index.refresh

response = Cause.search do
  query  { string 'name' }
  filter :term, :"user.exact" => 'last name'
  sort   { by "user.exact", 'desc' }
end

p response.results

# returns nothing because term filter is not analyzed
response = Cause.search do
  query  { string 'name' }
  filter :term, :user => 'last name'
  sort   { by "user.exact", 'desc' }
end

p response.results

# can't sort by analyzed property
response = Cause.search do
  query  { string 'name' }
  filter :term, :user => 'last name'
   sort   { by "user", 'desc' }
end

Reply to this email directly or view it on GitHub:
#101 (comment)

michelson commented Sep 13, 2011

thanks for your replies,

i can get the query to work with a filter but i got some problems with
sorting

i hve tried many ways but i get an error

in the search

a = Cause.search do
query { string terms }
filter :term, :user_id => "1"
#sort { by "tribunal.exact", 'desc' }
#sort { by "user.exact", 'desc' }
end

a.results

it return the collection but when i call a.results it throws an AR exeption
ActiveRecord::UnknownAttributeError: unknown attribute: caratulado

also if i try the sort methods above i get a curl request fail,
NoMethodError: undefined method `results' for false:FalseClass

i have this mapping in the model

mapping do
indexes :user_id, :type => 'string', :index => :not_analyzed

indexes :user, :type => 'multi_field', :fields => {
  :user => { :type => "string", :analyzer => "snowball" },
  :"user.exact" => { :type => "string", :index => :not_analyzed }
}

indexes :tribunal, :type => 'multi_field', :fields => {
  :tribunal => { :type => "string", :analyzer => "snowball" },
  :"tribunal.exact" => { :type => "string", :index => :not_analyzed }
}

indexes :civil do
  indexes :rol, :type => 'string'
  indexes :caratulado ,  :type => 'string', :boost => 100
end

end

really lost here

regards

Atte.
Miguel Michelson Martinez
www.artenlinea.com

On Mon, Sep 12, 2011 at 5:36 AM, Vojtěch Hýža <
reply@reply.github.com>wrote:

Hello,

here is example how can be multifield type used:

require 'rubygems'
require 'tire'

Tire.configure { logger STDERR }
Tire.index("causes").delete

class Cause
 include Tire::Model::Persistence
 include Tire::Model::Search

  mapping do
   indexes :user, :type => 'multi_field', :fields => {
     :user => { :type => "string", :analyzer => "snowball" },
     :"user.exact" => { :type => "string", :index => :not_analyzed }
   }
   indexes :civil do
     indexes :rol, :type => 'string'
     indexes :tribunal,  :type => 'string', :boost => 100
   end
 end


 # Only for this test
  property :user
 property :civil

end

Cause.create :user => "first name"
Cause.create :user => "last name"
Cause.index.refresh

response = Cause.search do
  query  { string 'name' }
  filter :term, :"user.exact" => 'last name'
  sort   { by "user.exact", 'desc' }
end

p response.results

# returns nothing because term filter is not analyzed
response = Cause.search do
  query  { string 'name' }
  filter :term, :user => 'last name'
  sort   { by "user.exact", 'desc' }
end

p response.results

# can't sort by analyzed property
response = Cause.search do
  query  { string 'name' }
  filter :term, :user => 'last name'
   sort   { by "user", 'desc' }
end

Reply to this email directly or view it on GitHub:
#101 (comment)

@karmi

This comment has been minimized.

Show comment
Hide comment
@karmi

karmi Sep 13, 2011

Owner

Hi,

it is really hard to debug what's going on unless I can re-create your data and model. Could you at least gist the complete model? (http://gist.github.com)

Owner

karmi commented Sep 13, 2011

Hi,

it is really hard to debug what's going on unless I can re-create your data and model. Could you at least gist the complete model? (http://gist.github.com)

@michelson

This comment has been minimized.

Show comment
Hide comment
@michelson

michelson Sep 13, 2011

Hi Karel ,

i ve created the gist,

https://gist.github.com/875756b52b3af05511e9

if you need more information about it just ask

thanks for your help

Atte.
Miguel Michelson Martinez
www.artenlinea.com

On Tue, Sep 13, 2011 at 5:45 AM, Karel Minarik <
reply@reply.github.com>wrote:

Hi,

it is really hard to debug what's going on unless I can re-create your data
and model. Could you at least gist the complete model? (<
http://gist.github.com>)

Reply to this email directly or view it on GitHub:
#101 (comment)

michelson commented Sep 13, 2011

Hi Karel ,

i ve created the gist,

https://gist.github.com/875756b52b3af05511e9

if you need more information about it just ask

thanks for your help

Atte.
Miguel Michelson Martinez
www.artenlinea.com

On Tue, Sep 13, 2011 at 5:45 AM, Karel Minarik <
reply@reply.github.com>wrote:

Hi,

it is really hard to debug what's going on unless I can re-create your data
and model. Could you at least gist the complete model? (<
http://gist.github.com>)

Reply to this email directly or view it on GitHub:
#101 (comment)

@karmi

This comment has been minimized.

Show comment
Hide comment
@karmi

karmi Sep 13, 2011

Owner

Hi,

well, thanks -- but unless I painstakingly create your app (ie. the models Tribunal etc), I cannot test the same thing you're having. I have some advice, though:

1/ Make sure you delete and create the index fresh, when you change the mapping. The Rake task (see Readme) has the FORCE=true option for that. Check the mapping in ES at http://localhost:9200/causes/_mapping.

2/ Try loading your models from database with Cause.search :load => true do ... when searching. I've got the impression the relation between your models is complicated, and I'm not sure how everything clicks together based on your to_indexed_json.

3/ Have a look into the ES directly (http://localhost:9200/causes/_search?q=*), to see how your documents are added.

Owner

karmi commented Sep 13, 2011

Hi,

well, thanks -- but unless I painstakingly create your app (ie. the models Tribunal etc), I cannot test the same thing you're having. I have some advice, though:

1/ Make sure you delete and create the index fresh, when you change the mapping. The Rake task (see Readme) has the FORCE=true option for that. Check the mapping in ES at http://localhost:9200/causes/_mapping.

2/ Try loading your models from database with Cause.search :load => true do ... when searching. I've got the impression the relation between your models is complicated, and I'm not sure how everything clicks together based on your to_indexed_json.

3/ Have a look into the ES directly (http://localhost:9200/causes/_search?q=*), to see how your documents are added.

@karmi

This comment has been minimized.

Show comment
Hide comment
@karmi

karmi Sep 29, 2011

Owner

Closing until there's a way to recreate the issue....

Owner

karmi commented Sep 29, 2011

Closing until there's a way to recreate the issue....

@karmi karmi closed this Sep 29, 2011

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.