Allow passing in memoize functions directly to createSelector.

[aryaemami59]

This PR:

• Solves Add separate options for memoize and argMemoize, and merge together #608 by allowing us to pass in the memoizer function directly to createSelector.
• No breaking changes.
• New createSelectorCreator signature overload.
• Adds memoize, memoizeOptions, argsMemoize and argsMemoizeOptions to createSelector.

Runtime changes

• Add createSelectorCreator function overload that allows passing in an options object to allow for more customization. The memoize property is mandatory, the rest are optional.
• Add memoize and argsMemoize to output selector fields.
• Allow passing memoize, argsMemoize and argsMemoizeOptions to options argument inside createSelector. This solves the issue of having to call createSelectorCreator anytime we want to use an alternate memoize function.
• So now instead of doing this:

const createSelectorWeakMap = createSelectorCreator(weakMapMemoize)

const selectTodoIds = createSelectorWeakMap(
  [(state: State) => state.todos],
  todos => todos.map(todo => todo.id)
)

• We can do this:

const selectTodoIds = createSelector(
  [(state: State) => state.todos],
  todos => todos.map(todo => todo.id),
  { memoize: weakMapMemoize }
)

Why add memoize and argsMemoize to output selector fields?

It's simple, it allows us to swap out memoize functions, compose selectors from other selectors easier and it helps with debugging:

export const findFastestSelector = <S extends OutputSelector>(
  selector: S,
  ...selectorArgs: Parameters<S>
) => {
  const memoizeFuncs = [defaultMemoize, weakMapMemoize, autotrackMemoize]
  const results = memoizeFuncs.map(memoizeFunc => {
    const alternateSelector = createSelector(
      selector.dependencies as [...SelectorArray],
      selector.resultFunc,
      { memoize: memoizeFunc }
    )
    const start = performance.now()
    // @ts-ignore
    alternateSelector(...selectorArgs)
    const time = performance.now() - start
    return { name: memoizeFunc.name, time, selector: alternateSelector }
  })
  const fastest = results.reduce((minResult, currentResult) =>
    currentResult.time < minResult.time ? currentResult : minResult
  )
  const ratios = results
    .filter(({ time }) => time !== fastest.time)
    .map(
      ({ time, name }) =>
        `\x1b[33m \x1b[1m${
          time / fastest.time
        }\x1b[0m times faster than \x1b[34m${name}`
    )
  if (fastest.selector.memoize.name !== selector.memoize.name) {
    console.warn(
      `The memoization method for \x1B[1;41m${
        selector.name
      }\x1B[0m is \x1B[31m${
        selector.memoize.name
      }\x1B[0m!\nChange it to \x1b[32m\x1b[1m${
        fastest.selector.memoize.name
      }\x1b[0m to be more efficient.\nYou should use \x1b[32m\x1b[1m${
        fastest.name
      }\x1b[0m because it is${ratios.join('\nand\n')}`
    )
  }
  return { results, fastest } as const
}

So now we can find out which memoize function would be fastest:

const selectorAutotrack = createSelector(
  [(state: State, id: number) => id, (state: State) => state.todos],
  (id, todos) => todos.filter(todo => todo.id === id),
  { memoize: autotrackMemoize }
)
Object.defineProperty(selectorAutotrack, 'name', {
  value: 'selectorAutotrack'
})
// @ts-ignore
findFastestSelector(selectorAutotrack, state, 1)

We get the following message in the console:

The memoization method for selectorAutotrack is autotrackMemoize!
Change it to defaultMemoize to be more efficient.
You should use defaultMemoize because it is 1.4709747668511033 times
faster than weakMapMemoize and 9.652792674178125 times faster than autotrackMemoize

Or we can also do this:

export function createSelectorCreatorCurried(
  ...args: Parameters<typeof createSelectorCreator>
) {
  const createSelector = createSelectorCreator(...args)
  return (...args: Parameters<ReturnType<typeof createSelectorCreator>>) => {
    const selector = createSelector(...args)
    const curriedSelector = selector.argsMemoize((...params) =>
      selector.argsMemoize((state: unknown) => selector(state, ...params))
    )
    Object.assign(curriedSelector, selector)
    return curriedSelector
  }
}

const createCurriedSelector = createSelectorCreatorCurried(defaultMemoize)
const curriedSelector = createCurriedSelector(
  [(state: State, id: number) => id, (state: State) => state.todos],
  (id, todos) => todos.filter(todo => todo.id === id)
)
curriedSelector(1)(state) // Returns [ { id: 1, completed: false } ]

---

Types

New types added

Add UnknownMemoizer type alias

• UnknownMemoizer is exactly what it sounds like. It's a function that takes another function as its first argument and returns it. It can also take an options rest parameter.

Add Combiner type alias

• This was mostly because the pattern of (...args: SelectorResultArray<Selectors>) => Result is seen too often in the codebase, so it made sense to shorten it by aliasing its type.

Add OmitIndexSignature utility type

• This is mainly used to remove explicit anys from the return type of some memoizers. e.g: microMemoize

Add IfNever utility type

• It helps conditionally resolve the type of memoizeOptions based on whether memoize is provided or not. Same applies to argsMemoize and argsMemoizeOptions.

Add ExtractMemoizerFields helper type

• A simple type alias that helps extract the extra properties that are attached to the return value of a memoizer. e.g.: clearCache

Add MemoizeOptionsFromParameters helper type

• A simple type alias that helps extract the type of memoizeOptions from the parameters of memoize. Same applies to argsMemoize and argsMemoizeOptions.

Add OverrideMemoizeOptions helper type

• It helps derive the type of memoizeOptions based on whether memoize itself was overridden inside the options argument of createSelector. Same applies to argsMemoize and argsMemoizeOptions.

Add FallbackIfNever utility type

• It is used to help us detect whether a memoize function has been overridden inside the options argument of createSelector. If no memoize is passed in, it will fallback to whatever memoize was originally passed into createSelectorCreator. Same applies to argsMemoize but since argsMemoize is optional inside createSelectorCreator, it will ultimately fallback to defaultMemoize.

Types changed

Rewrite CreateSelectorFunction

• This might be the biggest type change so far, yet it still pretty much functions like before. It does not break anything which means it is backwards-compatible. Here is all that has changed:

  • It used to take 4 generic type parameters, now it takes 2. One of which was added due to the new createSelectorCreator signature overload.
  • First generic parameter is MemoizeFunction which is almost the same as the old MemoizeFunction. The only difference is it now extends UnknownMemoizer and the type of the function it memoizes can be inferred.
  • Second generic parameter is ArgsMemoizeFunction which has been added due to the new options argument in createSelector and the new function overload for createSelectorCreator. It is the type of argsMemoize which has a default of typeof defaultMemoize.
  • The input selectors parameter and the combiner parameter are still pretty much the same as before. Within the signature overloads provided by CreateSelectorFunction, the 2 parts that have changed are the options parameter and the return type specifically the output selector fields that are attached to the memoized selector e.g.: clearCache
  • The options parameter can now take memoize, argsMemoize and argsMemoizeOptions in addition to the previously provided inputStabilityCheck and memoizeOptions.
  • If we pass in a value for memoize, the type of memoizeOptions will dynamically change to the type of the second parameter of memoize. If memoize is not provided, the type of memoizeOptions falls back to the second parameter of whatever memoize was initially passed in to createSelectorCreator whether it was passed in as an object with the new signature overload or traditionally as a plain function.

• Here is an example:

  const createSelectorWeakMap = createSelectorCreator(weakMapMemoize)
  // This would result in a TS error.
  const selector1 = createSelectorWeakMap(
    [(state: State) => state.todos],
    todos => todos.map(todo => todo.id),
    { memoizeOptions: { maxSize: 2 } } // This causes the TS error.
  )
  // This on the other hand is fine.
  const selector2 = createSelectorWeakMap(
    [(state: State) => state.todos],
    todos => todos.map(todo => todo.id),
    { memoize: defaultMemoize, memoizeOptions: { maxSize: 2 } } // When `memoize` is changed to `defaultMemoize`, `memoizeOptions` can now be the same type as the options args in `defaultMemoize`.
  )

• The same relationship dynamic applies to argsMemoize and argsMemoizeOptions. The only difference is since argsMemoize is optional when passing in an object to createSelectorCreator, the type of argsMemoize defaults to typeof defaultMemoize and the type of argsMemoizeOptions is derived from the second parameter of defaultMemoize.

// This is fine
const createSelector1 = createSelectorCreator({
  memoize: defaultMemoize,
  argsMemoizeOptions: { maxSize: 2 }
})
// This would result in a TS error.
const createSelector2 = createSelectorCreator({
  memoize: defaultMemoize,
  argsMemoize: weakMapMemoize,
  argsMemoizeOptions: { maxSize: 2 } // This causes the TS error.
})

• Same rules apply inside the options argument of createSelector:

// This would result in a TS error.
const selector1 = createSelector(
  [(state: State) => state.todos],
  todos => todos.map(todo => todo.id),
  {
    memoize: weakMapMemoize,
    memoizeOptions: { maxSize: 2 } // This causes the TS error.
  }
)
// This on the other hand is fine.
const selector2 = createSelector(
  [(state: State) => state.todos],
  todos => todos.map(todo => todo.id),
  { memoize: defaultMemoize, memoizeOptions: { maxSize: 2 } } // When `memoize` is set to `defaultMemoize`, `memoizeOptions` can now be the same type as the options args in `defaultMemoize`.
)
// This would result in a TS error.
const selector3 = createSelector(
  [(state: State) => state.todos],
  todos => todos.map(todo => todo.id),
  {
    argsMemoize: weakMapMemoize,
    argsMemoizeOptions: { maxSize: 2 } // This causes the TS error.
  }
)
// This on the other hand is fine.
const selector4 = createSelector(
  [(state: State) => state.todos],
  todos => todos.map(todo => todo.id),
  { argsMemoize: defaultMemoize, argsMemoizeOptions: { maxSize: 2 } } // When `argsMemoize` is set to `defaultMemoize`, `argsMemoizeOptions` can now be the same type as the options args in `defaultMemoize`.
)

createSelectorCreator type changes

New createSelectorCreator overload signature

• createSelectorCreator can now accept an options object as its argument.
• Here is what the new signature overload looks like:

export function createSelectorCreator<
  MemoizeFunction extends UnknownMemoizer,
  ArgsMemoizeFunction extends UnknownMemoizer = typeof defaultMemoize
>(options: {
  /** Overrides the global input stability check for the selector. */
  inputStabilityCheck?: StabilityCheck
  /** A function that accepts another function and returns it. This function is used to memoize `resultFunc`. */
  memoize: MemoizeFunction
  /** The memoizer function used to memoize the arguments of the selector. */
  argsMemoize?: FallbackIfNever<ArgsMemoizeFunction, typeof defaultMemoize>
  /** Options object passed to `memoize` as the second argument. */
  memoizeOptions?: MemoizeOptionsFromParameters<MemoizeFunction>
  /** Options object passed to `argsMemoize` as the second argument. */
  argsMemoizeOptions?: OverrideMemoizeOptions<
    typeof defaultMemoize,
    ArgsMemoizeFunction
  >
}): CreateSelectorFunction<MemoizeFunction, ArgsMemoizeFunction>

Rewrite old createSelectorCreator function signature

• The initial createSelectorCreator function signature works just like before except now it only takes one generic parameter of MemoizeFunction which is the type of the memoize function passed in. The type of the memoizeOptionsFromArgs rest parameter is derived from MemoizeFunction.
• Here is what it looks like now:

export function createSelectorCreator<MemoizeFunction extends UnknownMemoizer>(
  memoize: MemoizeFunction,
  ...memoizeOptionsFromArgs: DropFirstParameter<MemoizeFunction>
): CreateSelectorFunction<MemoizeFunction, typeof defaultMemoize>

Change DropFirst to DropFirstParameter

• Since this type originally was a duplicate of Tail, it made sense to change it from a type that drops the first item in an array to a type that drops the first parameter of a function.

Rewrite OutputSelectorFields

• It now has 2 additional properties memoize and argsMemoize. Instead of taking a Combiner and a Keys generic type parameter, it now takes 4 generic parameters:

  • Selectors: the input selectors. This enables us to get a strongly typed dependencies property as well.
  • Result: the return value of resultFunc.
  • MemoizeFunction: type of memoize. It is used to infer the correct type for memoizedResultFunc and the new memoize output selector field.
  • ArgsMemoizeFunction: the memoize function used for the new argsMemoize output selector field.

Rewrite OutputSelector

• It now takes the exact same generic type parameters as OutputSelectorFields which in turn simplifies the return type of CreateSelectorFunction and should make it easier to maintain.

Rewrite OutputParametricSelector

• Remove the Combiner and Keys generic type parameters as they are no longer needed. This is one of those types that I don't think is insanely popular or useful, so technically we could just remove it without it becoming too much of a problem. But I still rewrote it to match the new type definitions just in case.

Rename UnknownFunction to AnyFunction

• Rename UnknownFunction which used to be of type (...args: any[]) => any to AnyFunction. This was mostly due to the implied correlation between the word Any and parameters of type any[]. To avoid potential further confusion, I also added UnknownFunction which is an alias for (...args: unknown[]) => unknown. This can be helpful for situations where you want to avoid using any.

---

Other minor type related tweaks

• Remove union with never in the Params generic type parameter of Selector since a union of any[] and never is just any[].
• Simplify Selector by localizing the conditional to Params.
• Add more explicit type assertions inside createSelectorCreator function body to increase readability.

Explicit type imports/exports

• Separate type imports/exports from plain js imports/exports by using the import type/export type syntax.

Deduplicate types

• A duplicate of EqualityFn was defined inside autotracking.ts which has now been removed and replaced by an import of EqualityFn from types.ts.
• A duplicate of AnyFunction originally defined as UnknownFunction was defined inside ts47-mergeParameters.ts which after build would emit UnknownFunction$1. It has now been removed and replaced by AnyFunction imported from types.ts.

Import and reuse AnyFunction wherever possible

• Import and reuse AnyFunction anywhere in the code where there is (...args: any[]) => any

---

Miscellaneous changes

• Add JSDocs to createSelectorCreator
• Remove "indent" rule from .eslintrc as it conflicts with prettier
• Add "@internal/" path to tsconfig for type tests

File structure changes / relocation

• Move StabilityCheck and CreateSelectorOptions into types.ts.
• Move createSelectorCreator into its own file.
• Move createStructuredSelector into its own file.
• Change index.ts to be a barrel file exporting everything.

Add utils.ts to src folder

It contains the following:

• assertIsFunction: Contains a small part of code inside createSelectorCreator extracted to make the function body smaller and easier to read, and to add a small layer of type safety and reusability at the same time. Since we were doing a runtime check for a type of something and throwing an error on upon failure, the pattern looked very much like an assertion function which can help us kill 2 birds with one stone.
• ensureIsArray: Another small part of code inside createSelectorCreator. But since it is a pattern that we repeat, it does not hurt to turn it into a function that we can reuse.
• getDependencies: Same function from before, the only difference is now upon failure, in order to be more explicit, it throws a TypeError instead of an Error.
• collectInputSelectorResults: Another piece of logic extracted from the body of createSelectorCreator. It only focuses on creating an array from the return values of input selectors.
• runStabilityCheck: The extracted logic of running stability checks inside createSelectorCreator made into its own standalone function.

---

Just to make sure everything is now working as expected, I tested both the types and runtime behavior in my personal projects using yalc. I also wrote units tests and type tests for everything new that was added or changed so merging should not be a problem. Hope this helps, I would love to contribute more so please let me know if anything else is needed.

Currently working on:

• Removing variadic function signature CreateSelectorFunction which will introduce a breaking change.
• Code mod to convert variadic args to array args. The existing code mod does not detect the options argument.
• Creating TypedCreateSelectorFunction, the current implementation only works with array args.
• Writing more unit tests.
• Writing more type tests.
• Writing some benchmarks using vitest.

[codesandbox-ci]

This pull request is automatically built and testable in CodeSandbox.

To see build info of the built libraries, click here or the icon next to each commit SHA.

Latest deployment of this branch, based on commit 9d7290f:

Sandbox	Source
Vanilla	Configuration
Vanilla Typescript	Configuration

[markerikson]

Thanks, I'll try to look at this over the weekend!

[markerikson]

fyi this is still on my list to review, but was at a conf last week and am also trying to sort out a couple other outstanding RTK PRs before I switch over to look at this.

[aryaemami59]

fyi this is still on my list to review, but was at a conf last week and am also trying to sort out a couple other outstanding RTK PRs before I switch over to look at this.

Not a problem at all, I know you're busy. I'm just trying to make everything easier for you to review so hopefully you can merge without any issues. Is there anything specific you would want me to do before you get a chance to review it? I'd be happy to help.

[markerikson]

Nothing I can think of immediately, other than "fix the TS types so they compile" :)

[aryaemami59]

Nothing I can think of immediately, other than "fix the TS types so they compile" :)

Oh shoot. My apologies, I will fix that right away.

[markerikson]

I've updated master to run the RTK CI example app tests against the latest RTK betas on rtk/v2.0-integration, and dropped support for TS <4.7. Rebased this PR against master to pick those up.

Now trying to actually get a sense of what all's changed in this PR :)

[markerikson]

✋ We need the process.env.NODE_ENV check to be wrapped around any calls to shouldRunInputStabilityCheck() and runStabilityCheck(). With the current code in the PR, those functions and the error messages are left in the production build artifact, which increases bundle size. We want the bundler to strip them out, and to get that flow control analysis to work it has to be wrapped around those calls in the parent function (ie createSelectorCreator).

A bit uglier, and I totally understand the intent of "let's wrap this in a named util!" :) But gotta move that check out to make the prod build strip this logic out. (The inputStabilityCheck part can stay here.)

[markerikson]

Initial thoughts:

• I like refactoring the codebase to move all the definitions out of index.ts, and leaving that as just the package exports
• The JS logic seems reasonable
• I really appreciate all the added JSDoc comments on all the types!
• I haven't fully grokked or followed through on all the TS types changes, but things seem to make sense overall
• I love the additional type, unit, and perf tests! Even without reading them carefully, that gives me a lot of confidence that your changes are working as intended.

In fact, I think I've really only got two bits of feedback:

First, per review comment, the process.env.NODE_ENV check needs to be at the parent function level to make sure this code gets eliminated during the prod build step

The other one's trickier, and has to do with user DX for TS/VS Code hover previews.

As an example: if you look at typescript_test/test.ts around line 1268, we've got this call:

  const selector = createSelector(
    (state: { foo: string }) => 1,
    (state: { bar: string }) => 2,
    (...args) => 0
  )

Currently, the TS hover preview looks like this:

It's a bit annoying (especially with the double {clearCache()} in there), but it shows you it's a function that takes an object as an argument.

With this PR, the preview is:

which is a lot more verbose and doesn't even look like a function at all.

Ideally, I'd want to see something akin to (made up):

const selector: (state: {foo: string, bar: string}) => number
  & {clearCache: () => void, recomputations: () => void, .......}

In other words, a more human-readable representation that "this is a function with these args, and also has these fields attached".

I have some long-left-over TS util types in types.ts that are supposed to convince TS to expand types: Expand<T>, ExpandRecursive<T>, ComputeDeep<T>, and Mapped<T>. I spent a few minutes trying to play with them and insert them in various spots around OutputSelector<T>. I was able to get the "it's just a function" part working, but that lost all knowledge of the attached selector fields.

I'm fine with merging this PR basically as-is (and I'll try to push the process.env.NODE_ENV changes myself shortly), but could you also take a stab at trying to get a more readable preview for selectors?

[markerikson]

Okay, this looks amazing! Thank you!

I'm going to go ahead and merge this, then do a couple other bits of cleanup and put out 5.0.0-beta.0.

Let's do any hover preview changes and docs work in follow-up PRs.

[aryaemami59]

Thanks for all the feedback. I'm glad you mentioned the issue with the hover preview, because for the past couple of days, I've been looking for a way to simplify that as well. I will go ahead and submit PRs for some more JSDocs I was adding and the hover preview issue.

[markerikson]

Yep! Also FYI, we're now in a bizarre cyclical dep issue because I renamed autotrackMemoize to unstable_autotrackMemoize and published that as Reselect 5.0.0-beta.0, but RTK beta tries to re-export autotrackMemoize, and so the CI examples are now failing since that export doesn't exist :)

I'm going to publish a new RTK beta shortly with fixes for this.

[aryaemami59]

I figured out how to solve the hover preview issue, but while doing so, I also (coincidentally) resolved the infinite type instantiation issue. The problem is it looks like resolving the hover preview issue, causes the inifnite type instantiation problem (to some extent). So I'm going to figure out if there is a reasonable middleground to reach. Also fixed some type issues related to createStructuredSelector.

[aryaemami59]

I tried really hard to get VSCode to show as much info as possible in the hover preview without sacrificing too much TS performance / type safety. Do you think this suffices? I'm gonna have another go at it in the next few days either way, I just wanted to get your feedback on it if possible.

[markerikson]

@aryaemami59 that's certainly a step in the right direction!

It'd be nice if we could get OutputSelectorFields simpler in some way, but that at least has the most critical info visible.

[aryaemami59]

@aryaemami59 that's certainly a step in the right direction!

It'd be nice if we could get OutputSelectorFields simpler in some way, but that at least has the most critical info visible.

I will try again some more tomorrow, but I did have one question, is there a reason why we have an overload for createStructuredSelector?

[markerikson]

No idea, actually!

[aryaemami59]

No idea, actually!

Yeah me neither, it doesn't make any sense. I'll see if I can get rid of it since the function itself has only one single behavior.