Is it possible to populate without `_id` ? #2562

Closed
mocheng opened this Issue Dec 26, 2014 · 82 comments

Comments

Projects
None yet
@mocheng
Contributor

mocheng commented Dec 26, 2014

I checked the API http://mongoosejs.com/docs/populate.html and tried out. It seems that populate can only works on foreign _id. Even there is a match option, it just add more filtering condition on _id query condition. There is no way to just populate with fields not involved with _id.

In my project, we keep the _id as ObjectId and has another field as auto-increment Number. For example, customers model has _id as Object and cuid as incremental number. Other collections just reference to cuid not to _id.

Is would be great to be able to customize this?
Is there any design limitation causing this feature missing?

@vkarpov15

This comment has been minimized.

Show comment
Hide comment
@vkarpov15

vkarpov15 Jan 1, 2015

Collaborator

It would be great to be able to customize this, but right now we enforce _id. I can't think of a good reason off the top of my head why not though. I'll have to investigate further.

Collaborator

vkarpov15 commented Jan 1, 2015

It would be great to be able to customize this, but right now we enforce _id. I can't think of a good reason off the top of my head why not though. I'll have to investigate further.

@taxilian

This comment has been minimized.

Show comment
Hide comment
@taxilian

taxilian Feb 19, 2015

Contributor

The ability to populate not just on a single other field but even on a set of fields is almost required to properly support sharding, which is a huge use case for us.

For example, we have a collection "students" which may have a shard key for "subscription"; when we populate student data from our sections model (which contains an array of students) we have the subscription on the document but it will only look up based on the _id stored in "student" rather than allowing us to specify that it should also map the documents "assignment" field to "assignment" on the student.

Since we'd then be doing a lookup without the shard key when doing populate it renders sharding useless, requiring all shards to be hit to retrieve the data instead of just one.

This is something we would be willing to fund development for or even do ourselves if we could get some guidance of where to do the work. It is a major need for our company (GradeCam).

Contributor

taxilian commented Feb 19, 2015

The ability to populate not just on a single other field but even on a set of fields is almost required to properly support sharding, which is a huge use case for us.

For example, we have a collection "students" which may have a shard key for "subscription"; when we populate student data from our sections model (which contains an array of students) we have the subscription on the document but it will only look up based on the _id stored in "student" rather than allowing us to specify that it should also map the documents "assignment" field to "assignment" on the student.

Since we'd then be doing a lookup without the shard key when doing populate it renders sharding useless, requiring all shards to be hit to retrieve the data instead of just one.

This is something we would be willing to fund development for or even do ourselves if we could get some guidance of where to do the work. It is a major need for our company (GradeCam).

@vkarpov15

This comment has been minimized.

Show comment
Hide comment
@vkarpov15

vkarpov15 Feb 20, 2015

Collaborator

@taxilian are you setting the shardKey schema option? AFAIK that's what that option is for.

Collaborator

vkarpov15 commented Feb 20, 2015

@taxilian are you setting the shardKey schema option? AFAIK that's what that option is for.

@taxilian

This comment has been minimized.

Show comment
Hide comment
@taxilian

taxilian Feb 20, 2015

Contributor

I haven't actually moved to start doing it yet because I've been looking for a solution to this issue; does populate take into account shardKey? My understanding was that shardKey only affected updates and inserts

Contributor

taxilian commented Feb 20, 2015

I haven't actually moved to start doing it yet because I've been looking for a solution to this issue; does populate take into account shardKey? My understanding was that shardKey only affected updates and inserts

@arliber

This comment has been minimized.

Show comment
Hide comment
@arliber

arliber Apr 19, 2015

It would be great to be able to specify the field which is to be "joined by", for example I'd like to be able to populate not only by userId (i.e. _id) field but also by userType, userGroup etc.
Thanks!

arliber commented Apr 19, 2015

It would be great to be able to specify the field which is to be "joined by", for example I'd like to be able to populate not only by userId (i.e. _id) field but also by userType, userGroup etc.
Thanks!

@podoler

This comment has been minimized.

Show comment
Hide comment

podoler commented Apr 19, 2015

+1

@nadavsinai

This comment has been minimized.

Show comment
Hide comment
@nadavsinai

nadavsinai Apr 30, 2015

+1 would be very useful

+1 would be very useful

@xr

This comment has been minimized.

Show comment
Hide comment
@xr

xr May 9, 2015

+1 same here!

xr commented May 9, 2015

+1 same here!

@prakash122

This comment has been minimized.

Show comment
Hide comment
@prakash122

prakash122 May 21, 2015

+1 love to have this

+1 love to have this

@ricolo

This comment has been minimized.

Show comment
Hide comment

ricolo commented May 24, 2015

+1

@vlapo

This comment has been minimized.

Show comment
Hide comment

vlapo commented Jun 4, 2015

+1

@prescottprue

This comment has been minimized.

Show comment
Hide comment
@prescottprue

prescottprue Jun 8, 2015

+1

It would be especially nice to be able to select the variable you would like to populate with by adding a parameter such as key:

For example if had posts that I wanted to populate the author and comments parameters of, but the parameters are holding non ObjectId parameters (username of user and title of comments) before population:

{
author: "user1",
comments:["comment5", "comment6"],
title:"Post1"
}

It would be nice to be able to add a parameter like "key" to let mongo know which parameter to lookup by:

Posts.find().populate([{path:"author", model:"Users", key:"username"}, {path:"comments", model:"Comments", key:"title"}])

This would fill the author parameter with a User found by username as well as fill the comments parameter with comments found by their title (I realize the comments by title is a little unrealistic, but you get the point).

+1

It would be especially nice to be able to select the variable you would like to populate with by adding a parameter such as key:

For example if had posts that I wanted to populate the author and comments parameters of, but the parameters are holding non ObjectId parameters (username of user and title of comments) before population:

{
author: "user1",
comments:["comment5", "comment6"],
title:"Post1"
}

It would be nice to be able to add a parameter like "key" to let mongo know which parameter to lookup by:

Posts.find().populate([{path:"author", model:"Users", key:"username"}, {path:"comments", model:"Comments", key:"title"}])

This would fill the author parameter with a User found by username as well as fill the comments parameter with comments found by their title (I realize the comments by title is a little unrealistic, but you get the point).

@apankov

This comment has been minimized.

Show comment
Hide comment

apankov commented Jun 10, 2015

+1

@marklar21

This comment has been minimized.

Show comment
Hide comment

+1

@john-bower

This comment has been minimized.

Show comment
Hide comment

+1

@jack-guy

This comment has been minimized.

Show comment
Hide comment

jack-guy commented Aug 9, 2015

+1

@asafam

This comment has been minimized.

Show comment
Hide comment

asafam commented Aug 23, 2015

+1

@whyhankee

This comment has been minimized.

Show comment
Hide comment
@whyhankee

whyhankee Aug 25, 2015

Contributor

+1

Contributor

whyhankee commented Aug 25, 2015

+1

@ArcoMul

This comment has been minimized.

Show comment
Hide comment

ArcoMul commented Aug 25, 2015

+1

@cannonpalms

This comment has been minimized.

Show comment
Hide comment
@cannonpalms

cannonpalms Sep 3, 2015

Big agree. I'd love the ability to define custom population schemes. It'd be great for any schema with one or more fields that are file paths, allowing populate to read from storage.

Big agree. I'd love the ability to define custom population schemes. It'd be great for any schema with one or more fields that are file paths, allowing populate to read from storage.

@raphaelokon

This comment has been minimized.

Show comment
Hide comment

+1

@naoxink

This comment has been minimized.

Show comment
Hide comment

naoxink commented Sep 14, 2015

+1

@enthusiast94

This comment has been minimized.

Show comment
Hide comment

+1

@Poplava

This comment has been minimized.

Show comment
Hide comment

Poplava commented Oct 9, 2015

+1

@lfk

This comment has been minimized.

Show comment
Hide comment

lfk commented Oct 20, 2015

+1

@RithwikAnand

This comment has been minimized.

Show comment
Hide comment

+1

@justvisiting

This comment has been minimized.

Show comment
Hide comment

+1

@kumasento

This comment has been minimized.

Show comment
Hide comment
@kumasento

kumasento Nov 13, 2015

Contributor

+1

Contributor

kumasento commented Nov 13, 2015

+1

@Neophy7e

This comment has been minimized.

Show comment
Hide comment

+1

@loris

This comment has been minimized.

Show comment
Hide comment
@loris

loris Nov 17, 2015

Contributor

+1

Contributor

loris commented Nov 17, 2015

+1

@tomerb15

This comment has been minimized.

Show comment
Hide comment

+1

@keithl8041

This comment has been minimized.

Show comment
Hide comment

+1

@danielflippance

This comment has been minimized.

Show comment
Hide comment
@prabhupadas

This comment has been minimized.

Show comment
Hide comment

+1

@vkarpov15 vkarpov15 added this to the 4.4 milestone Nov 24, 2015

@leonardokl

This comment has been minimized.

Show comment
Hide comment

+1

@robjens

This comment has been minimized.

Show comment
Hide comment

robjens commented Dec 18, 2015

+1

@beinnova

This comment has been minimized.

Show comment
Hide comment

+1

@MangeshPimpalkar

This comment has been minimized.

Show comment
Hide comment
@alexmingoia

This comment has been minimized.

Show comment
Hide comment
@alexmingoia

alexmingoia Feb 19, 2016

Contributor

Ping me on IRC (freenode, amingoia) or gitter if you have any questions about the code.

Contributor

alexmingoia commented Feb 19, 2016

Ping me on IRC (freenode, amingoia) or gitter if you have any questions about the code.

@sibelius

This comment has been minimized.

Show comment
Hide comment
@sibelius

sibelius Feb 28, 2016

https://github.com/whitecolor/mongoose-fill also does a great job to populate using data from other models

https://github.com/whitecolor/mongoose-fill also does a great job to populate using data from other models

@BenSower

This comment has been minimized.

Show comment
Hide comment

BenSower commented Mar 2, 2016

+1

@juanpujol

This comment has been minimized.

Show comment
Hide comment

+1

@obengwilliam

This comment has been minimized.

Show comment
Hide comment

+1

@gitawego

This comment has been minimized.

Show comment
Hide comment
@gitawego

gitawego Mar 9, 2016

this is a must :)

gitawego commented Mar 9, 2016

this is a must :)

@amit3vr

This comment has been minimized.

Show comment
Hide comment

amit3vr commented Mar 11, 2016

+1

@teja-paketts

This comment has been minimized.

Show comment
Hide comment

+1

@stewwan

This comment has been minimized.

Show comment
Hide comment

stewwan commented Mar 22, 2016

+1

@nguyentrongbang

This comment has been minimized.

Show comment
Hide comment
@ingoncalves

This comment has been minimized.

Show comment
Hide comment

+1

@markotom

This comment has been minimized.

Show comment
Hide comment

markotom commented Apr 7, 2016

+1

@forksofpower

This comment has been minimized.

Show comment
Hide comment

+1

@ashe540

This comment has been minimized.

Show comment
Hide comment

ashe540 commented Apr 16, 2016

+1

@BenBestmann

This comment has been minimized.

Show comment
Hide comment

+1

@realbisoye

This comment has been minimized.

Show comment
Hide comment

+1

@rychkog

This comment has been minimized.

Show comment
Hide comment

rychkog commented May 13, 2016

+1

@atamas101

This comment has been minimized.

Show comment
Hide comment

+1

@mogita

This comment has been minimized.

Show comment
Hide comment

mogita commented May 19, 2016

+1

@samidarko

This comment has been minimized.

Show comment
Hide comment

+1

@zing777

This comment has been minimized.

Show comment
Hide comment

zing777 commented Jun 7, 2016

+1

vkarpov15 added a commit that referenced this issue Jun 10, 2016

@vkarpov15 vkarpov15 added the fixed? label Jun 10, 2016

vkarpov15 added a commit that referenced this issue Jun 13, 2016

Merge pull request #4219 from Automattic/gh-2562
Expose mongoose-populate-virtuals API in core

@vkarpov15 vkarpov15 closed this Jun 13, 2016

@oguzhanaslan

This comment has been minimized.

Show comment
Hide comment

+1

@thatisuday

This comment has been minimized.

Show comment
Hide comment
@atakangktepe

This comment has been minimized.

Show comment
Hide comment

+1

@anaibol

This comment has been minimized.

Show comment
Hide comment

anaibol commented Oct 26, 2016

+1

@teemuniiranen

This comment has been minimized.

Show comment
Hide comment
@coderafei

This comment has been minimized.

Show comment
Hide comment

+1

@fbegue

This comment has been minimized.

Show comment
Hide comment

fbegue commented Mar 26, 2017

+1

@anaibol

This comment has been minimized.

Show comment
Hide comment

anaibol commented Mar 30, 2017

+3

@omarps

This comment has been minimized.

Show comment
Hide comment

omarps commented Jun 16, 2017

+1

@Automattic Automattic locked and limited conversation to collaborators Jun 16, 2017

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.