Preloading of through associations is sub-optimal #2020

Closed
jonleighton opened this Issue Jul 9, 2011 · 7 comments

Projects

None yet

3 participants

@jonleighton
Member

Given:

class Post
  has_many :taggables
  has_many :tags, :through => :taggables
end

def includes
  Post.includes(:tags).all.map { |p| p.tags }
end

def no_includes
  Post.all.map { |p| p.tags }
end

The includes method results in more CPU time spent in Rails than the no_includes method (though it will hit the database with significantly fewer queries).

The reason for this is that it will preload all the through records, and then iterate over the through records and preload the source records on the through records. This results in two queries rather than one query with a join, and means that all the through records are unnecessarily loaded into memory with the CPU overhead that that incurs.

The solution will be to make preloading happen using joins as necessary. To do this, I want to adapt AssociationScope to be more generic such that it can be used by the preloader as well as by the normal associations code.

@jonleighton jonleighton was assigned Jul 9, 2011
Contributor

Wasn't that the design choice to have two queries instead of JOINs? I must say, I wondered about it when switched from 2.x.

Member

The query for posts will still be separate. At the moment it's like this:

SELECT * FROM posts
SELECT * FROM taggables WHERE ...
SELECT * FROM tags WHERE ...

After this is done, it will be:

SELECT * FROM posts
SELECT * FROM tags INNER JOIN taggables ON taggables.tag_id = tags.id WHERE ...
Member

And yeah, this does mean that taggables won't get preloaded. But if you want that to happen, you can do:

Post.includes(:taggables => :tags)
Member

Is this still a problem?

Member

yes

Member

Awesome, thanks for confirming.

Member

So, @jonleighton , this feels like a feature request to me, since it's not changing behavior, just an optimization. And since it's been two years with no forward motion, I think we can safely close it as stale.

If someone wants to optimize #includes, please submit a pull request doing so.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment