Auto-populate option for Components & Dynamic Zones

[Mcastres]

Feature request

Please describe your feature request

Having the option to auto-populate or not Components and Dynamic Zones in API responses from the Content-Types Builder.

• I have created my request on the Product Board before I submitted this issue
• I have looked at all the other requests on the Product Board before I submitted this issue

Summary

Components and Dynamic Zones (and media also) are not populated by default on v4. Why not have the option to auto-populate them?

Why is it needed?

You might always need these fields to be returned in your API response for some Content-Types while not having to populate them manually using populate every time you create some new ones.

ex: Page Content-Type. I will always need to have my components and DZ components (Auto-populate: true)
ex: Restaurant Content-Types. I will not always need my dynamic zones components (Auto-populate: false)

This can also prevent users to create big queries containing dozens of populate params if your Dynamic Zone contains dozens of components.

Suggested solution(s)

Implement a checkbox for auto-populating or not their Components, Dynamic Zones (Media fields?)

[derrickmehaffy]

Instead of a boolean on the DZ, I would suggest a file to specify the default full population if a population param isn't sent in the REST or GraphQL request. This would give a lot more control over the population in a standardized way.

Something like ./src/api/*/content-types/*/populate.js with a simple structure like:

module.exports = {
  populate: {
    metaDZ: {
      populate: {
        metaImage: {
          populate: {
            image: {
              fields: ['name', 'url']
            }
          }
        },
        relatedArticles: {
          fields: ['title', 'slug'],
        }
      }
    }
  }
}

[tomnovotny7]

Something like this will be great. We are using this solution, for now, #11836 (comment) You can specify the depth, but not choose what to populate. This helper function can be modified to use autoPopulate parameter.

[gu-stav]

3 thoughts on this, since it seems to cause a lot of confusion. To me idea 2 seems to be the most interesting one, but I haven't used the REST API from v4 yet, so it's pure theory :D

1. Reverse the behavoir

Since it's mainly a performance optimization, but you want the REST API to be as easy as possible, why not reverse the behavoir?

• no populate query param: means populate=* in the sense that it populates every field
• in a populate query param is set, then only include the things requested

This way advanced users, who need to take performance into consideration, have the ability, but at the same time the things you need to know as a starter about the api is minimal.

2. @derrickmehaffy to build on top of your idea

I think it would sense to make this a programmatic API, which can respond to the query. So maybe it's something like:

module.exports = {
  populate({ ctx }) {
     return { /* your proposal */ }
  }
}

This would even allow to implement my proposal 1) on the user side of things. As far as I understand this is specific to the REST API, right? So this should somehow reflect that in the file structure, to make it clear that GraphQL users don't have to do this. Also the query engine wouldn't take this into account by default I guess, since this needs to be as flexible as possible?

Edit: different thought, to extend what currently already exists: should populate be part of a content-type controller?

3. Make this a flag on the schema JSON

A field/ zone whatever could define whether it should populate by default (basically the idea @Mcastres had). This lacks flexibility from approach 2, but could be an easier fix, I guess ...

[tomnovotny7]

1. I do not think this is the way, populate=* is not sufficient, it does only the first level. We need something like v3 behavior.
2. This would be nice if such a file would be used by core services. But for dynamic zones or nested relations, this could be hard to maintain. I would really appreciate having a generator for this. We have dozens of components in a dynamic zone and recursive relations. Handling changes in such a structure could be tedious when the population object for dynamic zones is the merge of population objects of components (some attributes may overlap).
3. This is suggesting Strapi v3 like behavior, I think this approach could be made a default population with some depth to avoid recursion.

[derrickmehaffy]

Just going to clarify immediately since it was confirmed by the engineering team, we will not revert to v3 behavior as that would be a breaking change

We can only add additional features that aren't breaking changes. I believe we already plan to add more wildcards regardless of other features being discussed here (eg ** and *.*)

[WardenCommander]

Hello all is there possibly to populate relation/ component/dynamic zone i lifecycles hook?

Example:
before Update (data)
"data" only shows data in collection type and not in relation/ component/dynamic zone.

[derrickmehaffy]

Hello all is there possibly to populate relation/ component/dynamic zone i lifecycles hook?

Example: before Update (data) "data" only shows data in collection type and not in relation/ component/dynamic zone.

No, it is not possible to populate the data passed into the lifecycle. Specifically that beforeUpdate data payload is what should be coming from the user who sent it.

[WardenCommander]

Hello all is there possibly to populate relation/ component/dynamic zone i lifecycles hook?
Example: before Update (data) "data" only shows data in collection type and not in relation/ component/dynamic zone.

No, it is not possible to populate the data passed into the lifecycle. Specifically that beforeUpdate data payload is what should be coming from the user who sent it.

Hello @derrickmehaffy, thank you for reply. How would I be able to run function on collection type and its components every time collection type is updated?

If i would use lifecycle and call a function that in return updates that same collection type it would trigger Lifecycle again and create infinite loop.

[derrickmehaffy]

Hello all is there possibly to populate relation/ component/dynamic zone i lifecycles hook?
Example: before Update (data) "data" only shows data in collection type and not in relation/ component/dynamic zone.

No, it is not possible to populate the data passed into the lifecycle. Specifically that beforeUpdate data payload is what should be coming from the user who sent it.

Hello @derrickmehaffy, thank you for reply. How would I be able to run function on collection type and its components every time collection type is updated?

If i would use lifecycle and call a function that in return updates that same collection type it would trigger Lifecycle again and create infinite loop.

This is not the place to talk about this type of question, please move the discussion on this to either our Discord or Forum.

[projectnoontide]

I've tried many variations to achieve the following in a simple construct, to no avail:
?populate=&populate[DZ]=

If I try to open up a dynamic zone (i.e., populate[content][populate]=*), then all other auto-populate objects must be looped through and added individually to the query.

Is there currently any way to accomplish this simply? Developers really need to have full parameter control to open up the depths of their data models in both a granular way, as well as setting defaults for all data types. Let the developers be responsible for caching responses to complex queries and greater optimization if they choose to open things up.

I'm tempted to restructure my data models so that everything is contained within a single dynamic zone, then just use populate[dynamiczone][populate]=*. This would change the way objects are organized for data entry, though. But I shouldn't need to do so in order to simplify my query logic.

[derrickmehaffy]

I've tried many variations to achieve the following in a simple construct, to no avail: ?populate=&populate[DZ]=

If I try to open up a dynamic zone (i.e., populate[content][populate]=*), then all other auto-populate objects must be looped through and added individually to the query.

Is there currently any way to accomplish this simply? Developers really need to have full parameter control to open up the depths of their data models in both a granular way, as well as setting defaults for all data types. Let the developers be responsible for caching responses to complex queries and greater optimization if they choose to open things up.

I'm tempted to restructure my data models so that everything is contained within a single dynamic zone, then just use populate[dynamiczone][populate]=*. This would change the way objects are organized for data entry, though. But I shouldn't need to do so in order to simplify my query logic.

Not currently but we are planning to add additional glob wildcards

[Erns]

Something like this will be great. We are using this solution, for now, #11836 (comment) You can specify the depth, but not choose what to populate. This helper function can be modified to use autoPopulate parameter.

This solution by @tomnovotny7 I found to be a great solution for my needs as it addresses the need for an easier nested populate as well as Strapi v4's use of the json:api format with the data.attributes way of things. I think an "either or" sort of approach available by the Strapi API would be really beneficial.

Edit to add: To elaborate further, with Strapi v4's default API behavior, the Next.js example is simply not possible due to the dynamic nature of being able to generate static pages with whatever is setup in Strapi. Getting the populate setup for the page level dynamic zone is one thing, but then within those sections tends to be their own components and images and the like (the data types that aren't returned by the API by default) that need to be populated and the v4 API doesn't have a means to just make everything populate within the default API. Fortunately we can account for this within the controller files themselves on the Strapi side (thanks again to Tom), but this will likely be another pain point for others migrating from v3 to v4.