Eager loading with 'includes' overrides 'select' in SQL query

[kommen]

When creating a query with select and includes like

Post.select('author_id, COUNT(author_id) AS num_posts, authors.*').group('author_id').includes(:author)

the select part is ignored when querying the database.

However, if to_sql is called on the Relation, the query gets constructed directly.

to_sql output:

SELECT author_id, COUNT(author_id) AS num_posts, authors.* FROM "posts"  GROUP BY author_id

query sent to the db:

SELECT "posts"."id" AS t0_r0, "posts"."author_id" AS t0_r1, "posts"."title" AS t0_r2, "posts"."body" AS t0_r3, "posts"."type" AS t0_r4, "posts"."comments_count" AS t0_r5, "posts"."taggings_count" AS t0_r6, "posts"."taggings_with_delete_all_count" AS t0_r7, "posts"."taggings_with_destroy_count" AS t0_r8, "posts"."tags_count" AS t0_r9, "posts"."tags_with_destroy_count" AS t0_r10, "posts"."tags_with_nullify_count" AS t0_r11, "authors"."id" AS t1_r0, "authors"."name" AS t1_r1, "authors"."author_address_id" AS t1_r2, "authors"."author_address_extra_id" AS t1_r3, "authors"."organization_id" AS t1_r4, "authors"."owned_essay_id" AS t1_r5 FROM "posts" LEFT OUTER JOIN "authors" ON "authors"."id" = "posts"."author_id" GROUP BY author_id

See the attached patch for a complete testcase which currently fails.

[dipth]

+1

[the8472]

Wouldn'g using preload instead of includes suffice in most cases?

[isaacsanders]

This seems similar to #6132.

[steveklabnik]

As a note, this no longer merges cleanly, so a rebase will be needed.

[joevandyk]

Man, I'd love to see this fixed. So confusing to have #to_sql output something, but the query that runs is completely different.

[nijikon]

Had a similar problem lately, wanted to run something like this:

select(s).joins(:rule, :campaign => :customer).joins(j).includes(:rule, :campaign, :customer).where(w.join(' AND ')).having(h.join(' AND ')).order('rule_resource.priority ASC').all

https://github.com/rails/rails/blob/3-2-stable/activerecord/lib/active_record/relation/finder_methods.rb#L230

You need to replace

relation = except(:includes, :eager_load, :preload, :select).select(join_dependency.columns)

with

relation = except(:includes, :eager_load, :preload).select(join_dependency.columns)

Can anybody point out why there is a select there in the first place?
If it's there for a reason then having will never work in my case.

[steveklabnik]

@kommen I mentioned 6 months ago that this needed rebased, are you still interested in following-up with this pull request?

[kommen]

@steveklabnik Thanks for the reminder. I've rebased the commit onto the 3-1-stable branch. If you want me to open another pull request for 3-2-stable or master, let me know.

[steveklabnik]

Oh! yes, 3-1-stable isn't getting any non-security fixes these days. The way we prefer it to is to open a pull against master, and then fixes get backported as necessary. So can you please re-submit against master? Thanks!

[kommen]

@steveklabnik Turns out, this is fixed in 3.2 and also master. I've rebased onto master anyway, just in case: #8258

[jyr]

This was added in 3.2.12? I fail to override the select

[jyr]

I have same problem, @the8472 can you show me a example using preload?

I'm use

Property.group("bla").where("bla").select("DISTINCT ON(table.created_at) table.id, *").includes(bla)

[the8472]

Property.group("bla").where("bla").select("DISTINCT ON(table.created_at) table.id, *").joins(:bla).preload(:bla)

[Ferroman]

I have the same issue in 3.2.12.

Dish.select("dishes.*, ordered_dishes.id as dish_ordered"). includes(:ordered_dishes, :ordered_dishes => :user_menu). where("dishes.menu_section_id in (?) AND (user_menus.user_id = ? OR user_menus.user_id IS NULL)", menu_section_ids, user_id).

Result:

SELECT "dishes"."id" AS t0_r0, "dishes"."title" AS t0_r1, "dishes"."created_at" AS t0_r2, "dishes"."updated_at" AS t0_r3, "dishes"."menu_section_id" AS t0_r4, "ordered_dishes"."id" AS t1_r0, "ordered_dishes"."user_menu_id" AS t1_r1, "ordered_dishes"."dish_id" AS t1_r2, "ordered_dishes"."created_at" AS t1_r3, "ordered_dishes"."updated_at" AS t1_r4, "user_menus"."id" AS t2_r0, "user_menus"."weekly_menu_id" AS t2_r1, "user_menus"."user_id" AS t2_r2, "user_menus"."created_at" AS t2_r3, "user_menus"."updated_at" AS t2_r4, "user_menus"."ordered" AS t2_r5 FROM "dishes" LEFT OUTER JOIN "ordered_dishes" ON "ordered_dishes"."dish_id" = "dishes"."id" LEFT OUTER JOIN "user_menus" ON "user_menus"."id" = "ordered_dishes"."user_menu_id" WHERE (dishes.menu_section_id in (16,16,16) AND (user_menus.user_id = 2 OR user_menus.user_id IS NULL))

[gamov]

Yes, select is still overridden by includes in 3.2.22