bi-directional virtuals population issue

[grreeenn]

Versions

• NodeJS: 10.22.0
• Typegoose(NPM): 7.3.4
• Typegoose(GIT): commithash
• mongoose: 5.10.1
• mongodb: 4.0.20 Community

Code Example

Thank you for your work - Typegoose makes life with Mongo and Typescript a breeze :)

I have a strange issue (probably, because I'm trying to do a strange thing).
In my use case, Users can follow each other, and these relations contain additional properties - so I moved them to the separate model Followship.

Followship has followerId and followeeId (both IDs of Users) to define the relation. What I want is an ability to pull the Followships separately and populate Users in them, and also an ability to pull Users with their Followships populated.

So here is how my models look:

User:

@modelOptions({
  schemaOptions: {
    toJSON: { virtuals: true },
    toObject: { virtuals: true }
  }
})
export class User {

  @prop({ required: true })
  public name!: string;

  @prop({unique: true})
  public email!: string;

  @prop()
  public password!: string;

  // 
  @prop({
    ref: Followship,
    type: Followship,
    localField: '_id',
    foreignField: 'followeeId',
    justOne: false
  })
  public followers?: Ref<Followship>[];

  @prop({
    ref: Followship,
    type: Followship,
    localField: '_id',
    foreignField: 'followerId',
    justOne: false
  })
  public followees?: Ref<Followship>[];

  public static getById(this: ReturnModelType<typeof User>,
                        id: string
  ): Promise<DocumentType<User> | null> {
    return this.findById(id)
      .select('name email address')
      .populate('followees')
      .populate('followers')
      .exec();
  }
}

Followship:

@modelOptions({
  schemaOptions: {
    toJSON: { virtuals: true },
    toObject: { virtuals: true }
  }
})
export class Followship {
  @prop({ required: true })
  followerId!: ObjectID;

  @prop({ required: true })
  followeeId!: ObjectID;

  @prop({
    ref: User,
    localField: "followerId",
    foreignField: "_id",
    justOne: true
  })
  public follower?: Ref<User>;

  @prop({
    ref: User,
    localField: "followeeId",
    foreignField: "_id",
    justOne: true
  })
  public followee?: Ref<User>;

  @prop()
  seeFirst!: boolean;

  public static getFollowersOfUser(this: ReturnModelType<typeof Followship>,
                                   userId: string
  ): Promise<DocumentType<Followship>[]> {
    return this.find({ followeeId: userId })
      .populate({ path: "follower" })
      .exec();
  }
}

The issue is the following:
When I call User.getById(), I get the User object correctly populated with followers and followees (is there a better term for it in English btw?). But when I call Followship.getFollowersOfUser(), I get the array of Followships - with follower: null.
Debug mode shows, that Followship.getFollowersOfUser() tries to find the User ids in the Followship collection, and obviously fails:

Mongoose: followships.find({ followeeId: ObjectId("5f5836e2de810118a26bddba") }, { projection: {} })
Mongoose: followships.find({ _id: { '$in': [ ObjectId("5f56808196a0b238cf32fcff") ] } }, { skip: undefined, limit: undefined, perDocumentLimit: undefined, projection: {} })

The thing is, that if I remove the virtual fields from User model, Followship.getFollowersOfUser() returns just fine:

Mongoose: followships.find({ followeeId: ObjectId("5f5836e2de810118a26bddba") }, { projection: {} })
Mongoose: users.find({ _id: { '$in': [ ObjectId("5f56808196a0b238cf32fcff") ] } }, { skip: undefined, limit: undefined, perDocumentLimit: undefined, projection: {} })

Am I doing something wrong? Is there any workaround for this? Except using only the User's virtuals

I've seen the szokodiakos/typegoose#295, but I'm not sure I understand the examples there.

Do you know why it happens?

no

[grreeenn]

btw, if I populate the Followshops from users, like this

  public static getUserWithFollowers(this: ReturnModelType<typeof User>,
                                     id: string
  ): Promise<DocumentType<User> | null> {
    return this.findById(id)
      .populate({path: 'followers', select: 'followerId', populate: 'follower'})
      .exec();
  }

the follower field from the populated Followship returns as null:

Mongoose: users.findOne({ _id: ObjectId("5f5836e2de810118a26bddba") }, { projection: { name: 1, email: 1, address: 1, profilePicUrl: 1, isEmailValidated: 1, followers: 1 } })
Mongoose: followships.find({ followeeId: { '$in': [ ObjectId("5f5836e2de810118a26bddba") ] } }, { skip: undefined, limit: undefined, perDocumentLimit: undefined, projection: { followerId: 1, followeeId: 1, follower: 1 } })
Mongoose: followships.find({ _id: { '$in': [ ObjectId("5f56808196a0b238cf32fcff") ] } }, { skip: undefined, limit: undefined, perDocumentLimit: undefined, projection: {} })

as you can see, again it tries to lookup the follower User ID in the Followship collection.

[hasezoey]

@grreeenn firstly, when you set ref, then you should only set type with an refType (default ObjectId) (this is should be the type of the referenced model's _id type)

secondly, i dont know if this is just the example, but in Followship.getFollowersOfUser you try to populate path follower, which i dont see in User

is there a better term for it in English btw?

this isnt much different, but i know them in terms like following & followers

[grreeenn]

Thanks for your reply.

firstly, when you set ref, then you should only set type with an refType (default ObjectId) (this is should be the type of the referenced model's _id type)

Ok, so I removed the types from User:

  @prop({
    ref: Followship,
    localField: "_id",
    foreignField: "followeeId",
    justOne: false
  })
  public followers?: Ref<Followship>[];

  @prop({
    ref: Followship,
    localField: "_id",
    foreignField: "followerId",
    justOne: false
  })
  public followees?: Ref<Followship>[];

which didn't influence the behavior.

secondly, i dont know if this is just the example, but in Followship.getFollowersOfUser you try to populate path follower, which i dont see in User

That's something new to me. Should I populate the path of the foreign model? Path follower exists in class Followship, and that's what I'm trying to populate (and it even works correctly if I remove the virtuals from User).

[grreeenn]

Just to clarify:
When the User class looks like this

@modelOptions({
  schemaOptions: {
    toJSON: { virtuals: true },
    toObject: { virtuals: true }
  }
})
export class User {

  @prop({ required: true })
  public name!: string;

  @prop({unique: true})
  public email!: string;

  @prop()
  public password!: string;

  // @prop({
  //   ref: Followship,
  //   localField: "_id",
  //   foreignField: "followeeId",
  //   justOne: false
  // })
  // public followers?: Ref<Followship>[];
  //
  // @prop({
  //   ref: Followship,
  //   localField: "_id",
  //   foreignField: "followerId",
  //   justOne: false
  // })
  // public followees?: Ref<Followship>[];

  public static getById(this: ReturnModelType<typeof User>,
                        id: string
  ): Promise<DocumentType<User> | null> {
    return this.findById(id)
      .select('name email address')
      .populate('followees')
      .populate('followers')
      .exec();
  }
}

the Followship.getFollowersOfUser returns the 100% correct results, and Mongoose debug saying that it's searching in the correct collection

Mongoose: followships.find({ followeeId: ObjectId("5f5836e2de810118a26bddba") }, { projection: {} })
Mongoose: users.find({ _id: { '$in': [ ObjectId("5f56808196a0b238cf32fcff") ] } }, { skip: undefined, limit: undefined, perDocumentLimit: undefined, projection: {} })

[hasezoey]

That's something new to me. Should I populate the path of the foreign model? Path follower exists in class Followship, and that's what I'm trying to populate (and it even works correctly if I remove the virtuals from User).

sorry my bad, i read it wrong

---

sorry that i read your issue wrong, if you use virtuals with justOne: true, i in 99% of cases would recommend to not use virtuals and use ref directly (with that you would write less code)

[grreeenn]

the issue I'm solving with that is typing - when using virtuals, I get related object IDs in an array of IDs, and if I need objects themselves - I get them as an array of objects in a different key. This way, for example, I always know that the followerId is an ObjectID, but follower is a User object. It may be less convenient in the models, but it makes all the rest of the logic wayyyy cleaner.

[github-actions]

Marking Issue as stale, will be closed in 7 days if no more activity is seen

[github-actions]

Closing Issue because it is marked as stale