Amazing populate hook! several questions:

[beeplin]

1. I like the following powerful population syntax very much:

include: { // Which child items to join to the parent item
        post: { // This name is only used for some defaults
          service: 'posts', // The service to populate from
          parentField: 'postId', // The matching field in the parent. Supports dot notation a.b.c
          childField: 'id', // The matching field in the child. Supports dot notation a.b.c
          include: {
            author: {
              service: 'users',
              parentField: 'author',
              childField: 'id' // Would convert a.b.c to find({'a.b.c':value}). Use .query or .select for something else.
            },
            comment: {
              service: 'comments',
              parentField: 'id',
              childField: 'postId',
              select: (hook, parent) => ({ something: { $exists: false }}), // add to query using runtime data
              nameAs: 'comments', // Parent prop name where to place the populated items
              asArray: true, // store as an array if result has just 1 element
              query: { // Normal feathers query syntax. Get selected fields from the last 5 comments
                $limit: 5,
                $select: ['title', 'content', 'postId'],
                $sort: { createdAt: -1 }
              },
            },
            readers: {
              service: 'users',
              parentField: 'readers', // This is an array, so id: { $in: { readers } } will be used.
              childField: 'id'
            }
          }
        }
      }

But just one question: when nameAs is absent, where to place the populated items in the parent? Say:

include: {
            author: {
              service: 'users',
              parentField: 'author',
              childField: 'id' // Would convert a.b.c to find({'a.b.c':value}). Use .query or .select for something else.
            },

Here we get from service users an item whose id is the parent's author, and in which field do we put this item in the parent? The doc didn't seem to make this clear.

2. What's the relationship between select and query in the following part?

comment: {
              service: 'comments',
              parentField: 'id',
              childField: 'postId',
              select: (hook, parent) => ({ something: { $exists: false }}), // add to query using runtime data
              nameAs: 'comments', // Parent prop name where to place the populated items
              asArray: true, // store as an array if result has just 1 element
              query: { // Normal feathers query syntax. Get selected fields from the last 5 comments
                $limit: 5,
                $select: ['title', 'content', 'postId'],
                $sort: { createdAt: -1 }
              },
            },

I understand the select function returns an object that would be added into the query object, so basically the two are one thing, only that select is a function that accepting hook and parent as parameters so it is dynamic, but query is static. Is this right?

If so, I don't think it's necessary to make it into two different settings. We could only have one setting, say, query, and allow query to be either a static object or a function that dynamically returns an object using runtime data.

3. BTW, what does parent mean in select: (hook, parent) => ({ something: { $exists: false }})? the parent service in the poplulation schema?

4. I am not familiar with feathers-permissions so I didn't get how the permissions work. Any more examples?

5. I am not very sure what serialize means here, either, since this term is never officially introduced in feathers docs. I found serializers are excluding things from the result but populaters are adding things into the result. Is that right?

6. It's wonderful that the client side could determine which popluation schema should be used in a find opeartion. But the folloing syntax seems not very pretty:

  favorites.find({
    query: {
      _clientParams: { // how client passes params to server
        populate: 'standard', // Client wants favorites.standard populate. Supports dot notation a.b.c
        serialize: 'standard', // Client wants favorites.standard serialize. Supports dot notation a.b.c
      }
    }
  })

The _clientParams doesn't look nice with an underscore ahead. Maybe better to use something like $params. I understand currently only query is sent back to server, so it is a pity that we are not able to make it look nice like this:

  favorites.find({
    query: { ...},
    populate: 'standard', // Client wants favorites.standard populate. Supports dot notation a.b.c
    serialize: 'standard', // Client wants favorites.standard serialize. Supports dot notation a.b.c
  })

But even embedded in query we still be able to make it look nicer like this:

  favorites.find({
    query: {
      $populate: 'standard', // Client wants favorites.standard populate. Supports dot notation a.b.c
      $serialize: 'standard', // Client wants favorites.standard serialize. Supports dot notation a.b.c
    }
  })

[beeplin]

1. It would be nice to have another option to determine it will populate to hook.data or hook.result or hook.params, as we discussed before.

[eddyystop]

Thanks very much for looking at the design. To answer some of your questions:

(1) But just one question: when nameAs is absent, where to place the populated items in the parent?

You are correct that its not clear. The name of the object containing nameAs is the default for nameAs. So in your example the children would be saved under author.

(2) What's the relationship between select and query in the following part?

The construct resulting from childField: parentField, the result of the select, and query are combined to form the query part of find(query). Its logically like Object.assign({}, select(), { childField: parentField }, query).

that select is a function that accepting hook and parent as parameters so it is dynamic, but query is static. Is this right?

Correct.

If so, I don't think it's necessary to make it into two different settings.

True. I expect query will be used more often than select in practise and I wanted this 'happy path' to be a simple object. I will think on your reasonable suggestion.

(3) what does parent mean in select: (hook, parent)

Its the parent item to which we are wanting to join child items.

(4) I am not familiar with feathers-permissions so I didn't get how the permissions work.

feathers-permissions arrives with the new feathers-authentication which should drop this week or early next. Populate's permissions design is based on but not the same as feathers-permissions. We may try to integrate the two.

I added a section describing populate's permissions to the README. Please let me know if anything unclear.

(5) I am not very sure what serialize means here

I'm used to: to serialize an object means to convert its state to a byte stream so that the byte stream can be reverted back into a copy of the object.

However the Feathers team is comfortable also using it to mean extracting desired data from an object, though in our case it means removing unwanted data.

I found serializers are excluding things from the result but populaters are adding things into the result. Is that right?

Correct. Its not uncommon to combine the two needs in cases like this. However we felt there were conceptual and control benefits in separting them.

(6) But the following syntax seems not very pretty:

Agreed. Placing values in params.query on the client and later extracting them on the server is not uncommon, and similar boilerplate hooks are written each time. I wanted to standarize this with hook.getClientParams().

What do you think of using $clientParams' instead of _clientParams?

(A1) It would be nice to have another option to determine it will populate to hook.data or hook.result or hook.params, as we discussed before.

+1. That's already covered as the signatures are populate(defn, where = 'result', name) and serializeWith(defn, where = 'result', name).

Items outside a hook can be populated by calling the functions with a temporay hook object.

[beeplin]

Thank you for the explanations.

I still don't get how feathers-permissions works, but I can wait for the new feathers-authentication and feathers-permissions published.

I think $clientParams is much better, but the ideal way is to persuade the feathers core team to support transporting more params fields other than query back to the server.

Currently only query in params can be sent back to server, with params.token if avaiable. And then in verifyToken and populateUser hooks, more fields like params.user are added.

It would be wonderful if besides params.query, another special field, say, params.options, could also be sent back to server. If so, we could put populate: standard, serialize: standard into params.options, and no need to call hook.getClientParams() over and over again.

[eddyystop]

I agree with you. I suggest you create an issue for it in feathers-client.

[beeplin]

done. feathersjs-ecosystem/client#115