Populate not working as expected

[ashmore11]

Bug report

Describe the bug

Populate is not respecting given fields in params and instead populating all fields during find query. During update query relation fields are not returned at all and there is no means to populate them.

Steps to reproduce the behavior

1. Change any model relation to autoPopulate: false
2. Execute find query and set populate fields to only return a specific set of fields. IE: ["users", "users.id"], have also tried simply: ["users.id"]
3. Observe that all fields within the relation are returned and not only the id field
4. Update a relation on the same model
5. Observe relation is not returned when logging the updated model and the update does not actually update anything.

Expected behavior

When finding, only the provided fields in the populate params should be provided, not the entire object.
When updating, I should either be able to populate the relation I'm updating or it should return the relation I've just updated automatically.

Code snippets

Problem with find query:

const article = await strapi.query("article").findOne({ id }, ["users", "users.id"]);
console.log(article) returns:
{
  users: [{
    id: 1,
    name: John,
    email: john@test.com,
    ...
  }]
}

Problem with update query:

const updatedArticle = await strapi.query("article").update({ id }, { users: updatedUsers });
console.log(updatedArticle.users) // returns null and also update did not actually happen

Something to note: If I change autoPopulate back to true in the models settings then updates work fine again. Not sure if I'm doing something wrong here or if this is actually a bug.

System

• Node.js version: 12.18.3
• NPM version: 6.14.6
• Strapi version: 3.4.4 & 3.5.4
• Database: mySQL
• Operating system: macOS Big Sur 11.2.2

[tention]

const { data } = await strapi.services.YOUR_MODEL_NAME.find({}, ['FIELD_NAME1', 'FIELD_NAME2'])

Getting the same problem for query, it doesn't limit the results to the specified fields. Somehow it even throw me the following promise rejection warning.

  result = await strapi.services.articles.findOne({ uid: uid }, ['field1', 'field2']);

This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). The promise rejected with the reason:
Error: Undefined binding(s) detected when compiling SELECT. Undefined column(s): [id] query: select "articles".* from "articles" where "id" 
= ? limit ?

[tention]

I did some further check with afterFind lifecycle.

    afterFind: async (results, params, populate) => {
      console.log(results);
      console.log(populate);
    },

Populate was properly parsed as below, but the results are not being filtered. Full list of fields are being returned, tested with both console.log and Graphql. Seems like populate is not working at all for find() and findOne().

[ 'sender.name', 'type.name', 'type.description' ]

The only way I can fully control what fields to be returned is via the following codes, but there are a lot of codes to be added.

    const entity = await strapi.query('articles').model.fetchAll({
      columns: ['id','sender','type'],
      withRelated: [
        {
          sender: qb => {qb.columns('id','name')},
          type: qb => {qb.columns('id','name','description')},
        },
      ],
    });

@derrickmehaffy Would you be able to advise is this a bug or I have done it wrongly?

[derrickmehaffy]

@derrickmehaffy Would you be able to advise is this a bug or I have done it wrongly?

The populate array only accepts relational fields (and sub-relational), selecting actual fields isn't supported yet and it's something we are planning in the database refresh in Q3 when we rework the entire filtering and population system.

What you are doing is fine as it's skipping past the Strapi logic and is directly passing through the query to the ORM in your connector (Mongoose/Bookshelf)

[ashmore11]

@derrickmehaffy Thanks for responding!

If selecting actual fields isn't supported yet, I'm curious why in the documentation it states to use specific fields?

populate (array): you have to mention data you want populate ["author", "author.name", "comment", "comment.content"]

Thanks : )

[ashmore11]

@derrickmehaffy I can understand you are probably very busy but any thoughts on my question above? Would also be good to understand why update doesn't work with autoPopulate set to false.

[derrickmehaffy]

@derrickmehaffy Thanks for responding!

If selecting actual fields isn't supported yet, I'm curious why in the documentation it states to use specific fields?

populate (array): you have to mention data you want populate ["author", "author.name", "comment", "comment.content"]

Thanks : )

Short answer was a documentation issue (legacy docs) that we are working to rewrite most of the old documentation. That populate array only support relational fields and the structure follows the following structure:

["relationalField", "relationalField.nestedRelationField", ...]

[baermathias]

In my case the media field and other relational fields disappear after mentioning one relational field for populate. Also nesting is not considered.

In the following example I would want to see only user.id of the user relation inside the post object:

let params = { 'user.id': [1, 2, 3, 4] }
let populate = ['user.id']
entities = await strapi.services.post.find(params, populate);

But:

1. You still see the whole user object inside the post object, except the media field, which disappears from the user object.
2. Other relational fields (in my case comments, likes) disappear from the post object.

[derrickmehaffy]

In my case the media field and other relational fields disappear after mentioning one relational field for populate. Also nesting is not considered.

Yes that's intended, if you pass nothing (leave it as undefined) then it will populate all relations one level deep, an empty array [] populates nothing, and if you add relational fields then only that field is populated.

In the following example I would want to see only user.id of the user relation inside the post object:

This is not currently possible (will be in v4), you can only pass relational fields and nested relational fields. Standard fields do not count.

[derrickmehaffy]

This issue has been mentioned on Strapi Community Forum. There might be relevant details there:

https://forum.strapi.io/t/use-graphql-queries-from-rest-services/1101/3

[derrickmehaffy]

Fixed in v4, marking as closed