special characters ñ or áéíóú and multi-word facets

[hyperrjas]

I have a problem with special characters ñ or áéíóú.

my model:

class Car
  ac_field :name, :description, :city, :skip_settings => true

  def self.ac_search(params, options={})
    tire.search load: true, page: params[:page], per_page: 9 do
      query do
        boolean do
          must { string params[:query], default_operator: "AND" } if params[:query].present?
          must { term :city, params[:city] } if params[:city].present?
        end
      end
      filter :term, city: params[:city] if params[:city].present?
      facet "city" do
        terms :city
      end
    end
  end

end

This version works fine with special characters e.g.:

Query with Martin I get all results with Martín, martín, martin, Martin

With this approach this is the problem:

Now what results is individual words. e.g. A city tagged ["San Francisco", "Madrid"] will end up having three separate tags. Similarly, if I do a query to search on "san francisco" (must { term 'city', params[:city] }), that will fail, while a query on "San" or "Francisco" will succeed. The desired behaviour here is that the tag should be atomic, so only a "San Francisco" (or "Madrid") tag search should succeed.

To fix this problem I create my custom mapping:

model = self
  settings ElasticsearchAutocomplete::Analyzers::AC_BASE do
    mapping _source: {enabled: true, includes: %w(name description city)} do
      indexes :name, model.ac_index_config(:name)
      indexes :description, model.ac_index_config(:description)
      indexes :city, :type => 'string', :index => :not_analyzed 
    end
  end

With this mapping the problem with multi-words is fixed, and now facets with city field works fine:

Instead of getting the type facets San and Francisco Now I get San Francisco

Now, the problem is that with this mapping inside of the model the search doesn't find results with special characters e.g.:

Query with Martin I get only results with Martin martin

I'm using mongoid instead active record.

How can I fix this problem?

Thanks!

[leschenko]

Hi, sorry for delay - I was too busy with my job.

Query with Martin I get all results with Martín, martín, martin, Martin

this is because elasticsearch_autocomplete use asciifolding filter http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/analysis-asciifolding-tokenfilter.html
I think you should define your custom analyzers for your case

elasticsearch recently released suggester for autocomplete, may be you will find this internal solution better http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-suggesters-completion.html

I'm using mongoid instead active record.

doesn't metter

[hyperrjas]

I have the same problem with completion suggester. I have tried in many ways. I have checked with this way:

Model user:

class User
  include Mongoid::Document
  field :city, :type => String
  has_one: car
end

Model car:

class Car
  include Mongoid::Document
  ac_field :name, :user_city, :skip_settings => true

  belongs_to: user
  field: :name, :type => String

  def ac_self.search(param)
    tire.ac_search :load => true do
    query do
      boolean do
        must { term :user_city, params[:user_city] } if params[:user_city].present?
      end
    end
    filter :term, user_city: params[:user_city] if params[:user_city].present?
    facet "cities" do
      terms :user_city
    end
  end

  def to_indexed_json
    to_json(methods: [:user_city])
  end

  def user_city
   user.city
  end

end

This is my filter with facet user_city

 <% @cars.facets['cities']['terms'].each do |facet| %>
  <li>
    <%= link_to_unless_current Car.find_or_initialize_by(city: facet['term']).city, params.merge(user_city: facet['term']) %>
    <% if params[:car_city] == facet['term'].to_s %>
     (<%= link_to "Remove", params.merge(car_city: nil) %>)
    <% else %>
      (<%= facet['count'] %>)
    <% end %>
  </li>
<% end %>

Once I'd made a search, I'd like to filter by the city of the vehicle owner.

If I add my own mapping, asciifolding filter doesn't works. This is the main problem. How can I use asciifolding filter with my own mapping.

Thanks!

[hyperrjas]

Ok I fixed the error with:

model = self
  settings ElasticsearchAutocomplete::Analyzers::AC_BASE do
    mapping _source: {enabled: true, includes: %w(car_city name)} do
      indexes :car_city, :type => 'string', :index => :not_analyzed 
    end
  end

Thanks!