enhance: Maintain referential equality globally

[ntucker]

Motivation

SSR

To make SSR easy, the cache itself should be easily serializable/deserializable. Currently the cache stores instances of Entities, which means any deserialization construct would have to deal with knowing how to load the Entity classes. While this may be achievable, this amount of complexity while maintaining codesplitting seems undesirable.

Entities nested in entities

Since the flat entities are stored as results of fromJS() (normalized form) - they simply contain ids to their nested counterparts. This makes the typing somewhat odd, but more importantly means that denormalizing them on the fly violates referential equality expectations of any entity. In the past this hasn’t been a huge deal because we mostly don’t use nested entities in retail.

Update Performance

Referential guarantees are currently provided by a useDenormalized() hook. Using useMemo() ensures concurrent mode compatibility - but it also means all caches are distinct. This has several downsides.

• requiring recomputation even for repeated denormalizations
• no referential equality guarantees cross-component (though this is unlikely to cause problems, and hasn’t yet, it would be safer to provide global guarantees.)
• This ties the pub/sub system to React itself, limiting our ability to use things like recoil or context selector systems that can improve performance needed for react native, and potentially web with faster updates

Solution

Denormalize function will take in an extra optional argument that is a denormalization cache. This will be used to short-circuit computations if the pieces exist. This also means useDenormalized() will be simplified - possibly removed later.

The denormalized cache will be provided via context, but maintain referential equality due to its mutable nature.

Design constraints

Since concurrent mode compatibility is necessary, extra care is needed when creating a cache that is global. Unlike component-level caches which only need the last value, global caches need to be able to provide answers to questions from disjoint state trees.

Hence, we need a mechanism to look up based on the current state tree, which cache result might exist. Furthermore, once React has discarded a tree, and thus the state representation - these cache answers should be released to be collected from the GC.

This makes WeakMap an ideal supplementary data structure. However, there is an additional challenge to provide a list of keys that will be outlined below.

Cache data structure

const denormalizedCacheInstance = {
  entities: {
    [key]: {
      [pk]: WeakListMap([og, ...nestedEntities]
               -> { author: { id: 1, name: 'bob' }, content: 'hi' } (fromJS() result)
            )
    }
  }
  results: {
    [key]: WeakListMap([og, ...entities] -> EntireDenormalizedResponse)
  }
}

This structure mirrors the state cache itself, with the values of entities, and results being supplemented with a WeakMap.

WeakListMap

Mapping a list of referential objects to a value is not actually provided by the WeakMap interface. WeakMap can only map a singular object. To create the necessary data structure, we pull from lisps’ cons data construct. We can recursively apply a WeakMap to each entry in the list to achieve the appropriate mapping. This also resembles the Trie data structure.

This structure

WM([og, entitya, entityb] -> denorm(og, entitya, entityb))

Is represented by

WM(og -> WM(entitya -> WM(entityb -> denorm(og, entitya, entityb)))

Open questions

Future work

• Store pojos, not Entity instances

[github-actions]

Size Change: +9.56 kB (9%) 🔍

Total Size: 105 kB

Filename	Size	Change
packages/core/dist/index.cjs.js	12.3 kB	-56 B (0%)
packages/core/dist/index.umd.min.js	5.62 kB	-2 B (0%)
packages/normalizr/dist/normalizr.amd.js	9.64 kB	+1.57 kB (16%)	⚠️
packages/normalizr/dist/normalizr.amd.min.js	5.24 kB	+832 B (15%)	⚠️
packages/normalizr/dist/normalizr.browser.js	9.66 kB	+1.58 kB (16%)	⚠️
packages/normalizr/dist/normalizr.browser.min.js	5.24 kB	+831 B (15%)	⚠️
packages/normalizr/dist/normalizr.js	9.54 kB	+1.56 kB (16%)	⚠️
packages/normalizr/dist/normalizr.min.js	5.24 kB	+833 B (15%)	⚠️
packages/normalizr/dist/normalizr.umd.js	9.75 kB	+1.58 kB (16%)	⚠️
packages/normalizr/dist/normalizr.umd.min.js	5.32 kB	+830 B (15%)	⚠️

ℹ️ View Unchanged

Filename	Size	Change
packages/endpoint/dist/index.cjs.js	1.66 kB	0 B
packages/endpoint/dist/index.umd.min.js	1.23 kB	0 B
packages/hooks/dist/index.cjs.js	714 B	0 B
packages/hooks/dist/index.umd.min.js	429 B	0 B
packages/legacy/dist/index.cjs.js	565 B	0 B
packages/legacy/dist/index.umd.min.js	509 B	0 B
packages/rest-hooks/dist/index.cjs.js	7.47 kB	0 B
packages/rest-hooks/dist/index.umd.min.js	3.93 kB	0 B
packages/rest/dist/index.cjs.js	3.88 kB	0 B
packages/rest/dist/index.umd.min.js	2.15 kB	0 B
packages/test/dist/index.cjs.js	2.75 kB	0 B
packages/use-enhanced-reducer/dist/index.cjs.js	1.08 kB	0 B
packages/use-enhanced-reducer/dist/index.umd.min.js	607 B	0 B

_{compressed-size-action}

[ljharb]

LGTM but would be good to get more eyes on it if possible