Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

Loading…

Models with Associations #10

Closed
russ opened this Issue · 8 comments

3 participants

@russ

I am trying to create an index with associations and running into errors.

For example, I have a Video model that has many categories. I want to be able to search on the video information as well as the category name.

def to_indexed_json
    { :title => title,
      :description => description,
      :categories => categories.collect { |c| c.name },
      :released_at => released_at,
    }.to_json
end

Though when the object gets reloaded I get an unknown attribute error from Rails being caused by this line in Results::Collection

object = Configuration.wrapper.new(document)

I see what the problem is, so my question is, are there plans on changing on how this is implemented in the future or am I just going about this the wrong way.

@karmi
Owner

Hi, there's no elegant solution for this problem with associations, in the current version.

The wrapper is set to the model, and obviously, the model inicialization fails with NoMethodError (or other) error when being passed the hash, which was returned from ElasticSearch.

First, I have anticipated the issue. As you can see in commit karmi/retire@5d89dd1, Tire was initializing the collection from the records retrieved from the database, originally (via MyModel.find [list of IDs]). This, however, had many issues. The performance was unnecesarilly dumped by roundtrips to the database. The sort order was not preserved (a huge bug). So I decided against it.

Second, the problem can be immediately solved. Let's simplify your situation (because that's really a M to N relation, with videos and categories).

The Article model has many Comment associations:

class Article < ActiveRecord::Base
  include Tire::Model::Search
  include Tire::Model::Callbacks

  index_name 'articles-with-comments'

  has_many :comments
end

The Comment is simple:

class Comment < ActiveRecord::Base
  belongs_to :article
end

Now, obviously, when you have the Article indexed as:

def to_indexed_json
  { :id      => id,
    :title   => title,
    :content => content,
    :comments => comments.map { |c| { :id => id, :author => c.author, :content => c.content }  },
  }.to_json
end

it would break on Article#comments= being given wrong type of objects on initialization.

You can temporarily, and in a quite ugly way, solve it like this:

class Article < ActiveRecord::Base
  # ...
  alias :original_comments= :comments=
  def comments=(comments)
    # Are the objects returned from ElasticSearch?
    if comments.all? { |c| c.is_a? Hash }
      comment_ids = comments.map { |c| c['id'] }
    else
      send :original_comments=, comments
    end
  end
end

So, when we are initializing the comments from ElasticSearch, set the ID (an retrieve the records from database), otherwise, perform the original method.

Of course, this could be isolated into completely different model, like this:

class SearchedArticle < Article
  index_name 'articles-with-comments'

  def comments=(comments)
    comment_ids = comments.map { |c| c['id'] }
  end
end

You'll then search via this fake model: SearchedArticle.search 'love'. Of course, this is an ugly, and temporary way of dealing with the issue. The underlying issue is, of course, blocking for advanced production usage.

When I have discussed this with another Tire users, the real, and elegant, solution seems to me like this:

  • Do not wrap the results in the real model class, but wrap them in Tire::Results::Item, as usual
  • Add a proxy object to every result, let's say object, which would point, via the ID, to the underlying record (and model instance)

In this way, we would keep the awesome performance of ElasticSearch, for most cases. Where you'd need or like to get the original model instance, you'd just write result.object.comments.first.my_complicated_method, instead of result.comments.my_complicated_method.

Thank you for the report, and I'll definitely have a look at this issue.

@russ

Cool. I'll give that a try. Thanks for the detailed response.

@russ russ closed this
@karmi karmi reopened this
@karmi karmi was assigned
@karmi
Owner

@russ, any luck with the proposed solution?

@russ

The solution did work. But for the time being I went back to sunspot. I will definitely give Tire another try in the future.

@karmi
Owner

Understood. Thanks for the report!

@karmi karmi closed this issue from a commit
@karmi [ACTIVEMODEL] Added the `Results::Item#load` method, to load the "rea…
…l" model instance from database [#12]

In accord with issue karmi/retire#11 (see also 05a1331, be7621a, 90de105), this change provides a way to load
the "real" model instance from the database/storage.

All results are wrapped in Tire::Results::Item, by default, which provides fast and flexible access
to the properties returned from ElasticSearch (`_source` or `fields` JSON properties).

This way, we can index whatever we like for ElasticSearch and retrieve it. If, and only if we
need the access to the underlying model (eg. to its methods), we call item.load.<my_special_method>.

An example would be an article with comments:

    >> a = Article.first
      Article Load (4.4ms)  SELECT "articles".* FROM "articles" LIMIT 1
    => #<Article id: 3, title: "Three", content: "Consectetur...", published_on: "2011-07-11", ...>

    >> a.comments
      Comment Load (0.7ms)  SELECT "comments".* FROM "comments" WHERE ("comments".article_id = 3)
    => [#<Comment id: 1, author: "john", body: "Awesome!", article_id: 3, created_at: "2011-07-14 07:04:54", ..., #<Comment id: 2, author: "mary", body: "Yeah!", article_id: 3, created_at: "2011-07-14 07:06:15", ...>]

Now, we have the Article class like this:

    class Article < ActiveRecord::Base
      include Tire::Model::Search
      include Tire::Model::Callbacks

      has_many :comments

      mapping do
        indexes :title
        indexes :content
        indexes :published_on, :type => 'date'

        indexes :comments do
          indexes :author
          indexes :body
        end
      end

      def to_indexed_json
        {
          :title        => title,
          :content      => content,
          :published_on => published_on,
          :length       => length,

          :comments     => comments.map { |c| { :author => c.author, :body => c.body } },
        }.to_json
      end

      def length
        content.length
      end

      def comment_authors
        comments.map(&:author).to_sentence
      end
    end

We will search for articles:

    >> @articles = Article.search('three')
    => #<Tire::Results::Collection:0x10378f948 ...>

    @articles.each do |article|
      puts article.title
      puts article.load(:include => 'comments').comment_authors
    end

Now, we can access the *indexed* properties, as well as the *model* properties:

    >> @articles.each do |article|
    >>   puts article.title
    >>   puts article.load(:include => 'comments').comment_authors
    >> end
    Three
      Article Load (0.6ms)  SELECT "articles".* FROM "articles" WHERE "articles"."id" = 3 LIMIT 1
      Comment Load (0.3ms)  SELECT "comments".* FROM "comments" WHERE ("comments".article_id = 3)
    john and mary

Closes #12. Closes #10.
84f015d
@karmi karmi closed this in 84f015d
@karmi
Owner

Hi @russ, I've tried to solve this issue with ActiveRecord associations -- see the closing commit.

Could you test it against your use-case, if you still have the code handy? The only thing you need is to define Git as endpoint in the gemfile:

gem "tire", :git => "git://github.com/karmi/tire.git", :branch => "activerecord"
@aaronchi

This looks very cool :D

@karmi
Owner

@aaronchi: It looks, but let's see how it works in real world :)

@johnthethird johnthethird referenced this issue from a commit
@karmi [ACTIVEMODEL] Added the `Results::Item#load` method, to load the "rea…
…l" model instance from database [#12]

In accord with issue karmi/retire#11 (see also 05a1331, be7621a, 90de105), this change provides a way to load
the "real" model instance from the database/storage.

All results are wrapped in Tire::Results::Item, by default, which provides fast and flexible access
to the properties returned from ElasticSearch (`_source` or `fields` JSON properties).

This way, we can index whatever we like for ElasticSearch and retrieve it. If, and only if we
need the access to the underlying model (eg. to its methods), we call item.load.<my_special_method>.

An example would be an article with comments:

    >> a = Article.first
      Article Load (4.4ms)  SELECT "articles".* FROM "articles" LIMIT 1
    => #<Article id: 3, title: "Three", content: "Consectetur...", published_on: "2011-07-11", ...>

    >> a.comments
      Comment Load (0.7ms)  SELECT "comments".* FROM "comments" WHERE ("comments".article_id = 3)
    => [#<Comment id: 1, author: "john", body: "Awesome!", article_id: 3, created_at: "2011-07-14 07:04:54", ..., #<Comment id: 2, author: "mary", body: "Yeah!", article_id: 3, created_at: "2011-07-14 07:06:15", ...>]

Now, we have the Article class like this:

    class Article < ActiveRecord::Base
      include Tire::Model::Search
      include Tire::Model::Callbacks

      has_many :comments

      mapping do
        indexes :title
        indexes :content
        indexes :published_on, :type => 'date'

        indexes :comments do
          indexes :author
          indexes :body
        end
      end

      def to_indexed_json
        {
          :title        => title,
          :content      => content,
          :published_on => published_on,
          :length       => length,

          :comments     => comments.map { |c| { :author => c.author, :body => c.body } },
        }.to_json
      end

      def length
        content.length
      end

      def comment_authors
        comments.map(&:author).to_sentence
      end
    end

We will search for articles:

    >> @articles = Article.search('three')
    => #<Tire::Results::Collection:0x10378f948 ...>

    @articles.each do |article|
      puts article.title
      puts article.load(:include => 'comments').comment_authors
    end

Now, we can access the *indexed* properties, as well as the *model* properties:

    >> @articles.each do |article|
    >>   puts article.title
    >>   puts article.load(:include => 'comments').comment_authors
    >> end
    Three
      Article Load (0.6ms)  SELECT "articles".* FROM "articles" WHERE "articles"."id" = 3 LIMIT 1
      Comment Load (0.3ms)  SELECT "comments".* FROM "comments" WHERE ("comments".article_id = 3)
    john and mary

Closes #12. Closes #10.
f9273bc
@sakrafd sakrafd referenced this issue from a commit
@karmi [ACTIVEMODEL] Added the `Results::Item#load` method, to load the "rea…
…l" model instance from database [#12]

In accord with issue karmi/retire#11 (see also 05a1331, be7621a, 90de105), this change provides a way to load
the "real" model instance from the database/storage.

All results are wrapped in Tire::Results::Item, by default, which provides fast and flexible access
to the properties returned from ElasticSearch (`_source` or `fields` JSON properties).

This way, we can index whatever we like for ElasticSearch and retrieve it. If, and only if we
need the access to the underlying model (eg. to its methods), we call item.load.<my_special_method>.

An example would be an article with comments:

    >> a = Article.first
      Article Load (4.4ms)  SELECT "articles".* FROM "articles" LIMIT 1
    => #<Article id: 3, title: "Three", content: "Consectetur...", published_on: "2011-07-11", ...>

    >> a.comments
      Comment Load (0.7ms)  SELECT "comments".* FROM "comments" WHERE ("comments".article_id = 3)
    => [#<Comment id: 1, author: "john", body: "Awesome!", article_id: 3, created_at: "2011-07-14 07:04:54", ..., #<Comment id: 2, author: "mary", body: "Yeah!", article_id: 3, created_at: "2011-07-14 07:06:15", ...>]

Now, we have the Article class like this:

    class Article < ActiveRecord::Base
      include Tire::Model::Search
      include Tire::Model::Callbacks

      has_many :comments

      mapping do
        indexes :title
        indexes :content
        indexes :published_on, :type => 'date'

        indexes :comments do
          indexes :author
          indexes :body
        end
      end

      def to_indexed_json
        {
          :title        => title,
          :content      => content,
          :published_on => published_on,
          :length       => length,

          :comments     => comments.map { |c| { :author => c.author, :body => c.body } },
        }.to_json
      end

      def length
        content.length
      end

      def comment_authors
        comments.map(&:author).to_sentence
      end
    end

We will search for articles:

    >> @articles = Article.search('three')
    => #<Tire::Results::Collection:0x10378f948 ...>

    @articles.each do |article|
      puts article.title
      puts article.load(:include => 'comments').comment_authors
    end

Now, we can access the *indexed* properties, as well as the *model* properties:

    >> @articles.each do |article|
    >>   puts article.title
    >>   puts article.load(:include => 'comments').comment_authors
    >> end
    Three
      Article Load (0.6ms)  SELECT "articles".* FROM "articles" WHERE "articles"."id" = 3 LIMIT 1
      Comment Load (0.3ms)  SELECT "comments".* FROM "comments" WHERE ("comments".article_id = 3)
    john and mary

Closes #12. Closes #10.
10efad2
@tklee tklee referenced this issue from a commit in tklee/tire_shiphawk
@karmi [ACTIVEMODEL] [¡BREAKING!] Wrap search results in Item class, not the…
… actual model class [Closes #11]

Due to the problem with indexing and retrieving ActiveRecord models with associations (such as Article has_many :comments),
do not wrap the results in the model class, but in the Tire::Configuration.wrapper class.

See the karmi/retire#10 issue for explanation.

This commit breaks the old behaviour and usage in Rails, since Results::Item is not (yet) ActiveModel compatible.
2295135
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.