Nested denormalization does not work as expected

[Floriferous]

When you link multiple collections, and try to make use of recursive denormalization, grapher queries start to fail.

Here's an example:

const Users = new Meteor.Collection('users');
const Organisations = new Meteor.Collection('organisations');
const Posts = new Meteor.Collection('posts');

Users.addLinks({
  organisations: {
    collection: Organisations,
    inversedBy: 'users',
    denormalize: {
      field: 'organisationsCache',
      body: { name: 1 },
    },
  },
});

Organisations.addLinks({
  users: {
    collection: Users,
    field: 'userLinks',
    type: 'many',
    metadata: true,
  },
});

Posts.addLinks({
  users: {
    field: 'userLinks',
    collection: Users,
    type: 'many',
    metadata: true,
    denormalize: {
      field: 'organisationsCache',
      body: { firstName: 1, lastName: 1, organisations: { name: 1 } },
    },
  },
});

// Does not return the expected result
Posts.createQuery({
  title: 1,
  users: { firstName: 1, lastName: 1, organisations: { name: 1 } },
}).fetch();

Now when you try to fetch a Post who has a user with an organisation, the cache does not include the organisation, but only the user. This kind of "recursive" caching should work according to this, but it fails.

It is also hard to verify that grapher denormalization actually reduces the amount of queries. How do you test if such a query only does a single fetch on the Posts collection?

[theodorDiaconu]

@Floriferous caching is not links aware, meaning it would simply treat "organisations.name" as a field. There is no such feature inside Grapher, nor in denormalize. If you would like to hold the organisation name of users inside Posts, the trick would be to cache organisationsCache: 1 from users.

Denormalisation is meant to work 1 level only. If you need multi-level, you have to go custom, create a reducer at Posts level, and cache organisationsCache: 1 from Users.

I guess you are hitting the limits of NoSQL here. If your purposes are for search, then think of using something like elasticsearch or algolia, or a separate SQL cache. If your purpose is just for data rendering, then you really don't have to worry as Grapher is incredibly fast.

The main purpose of introducing denormalise was to allow easy search, not speed. For example I want to search all users from a given organisation by organisation name. There is also the possibility of doing this in $filter() at server level, but this was a neat way to skip a step.

[Floriferous]

Well, denormalize does in fact support what it calls "recursive caching": https://github.com/Herteby/denormalize#recursive-caching

Which is exactly what I'm looking for. In the end, it does not matter if grapher properly fetches both caches and only runs a single query, even though that'd be cool. The most important part is that the data should always be what I ask for, and in this case it is broken I believe.