Option to limit concurrent invocations?

[mmkal]

Hi! Is it possible to use p-memoize to optimistically memoize promises when several are kicked off before any results have come back? Here's the scenario we're hitting:

const getData = async params => {
  const data = await getDataFromSomeAPI(params) // quite slow
  return data.foo
}

export default pMemoize(getData, { cacheKey: JSON.stringify })

This function is on our server side and ends up being hit by next.js as part of static-site generation (and during ISR). But what this means is that it gets hit hundreds of times simultaneously, once per page that's being generated. The API we're using has a pretty aggressive rate limit, so we're aiming to use pMemoize to get around that, since all of the pages hit the backend with the same parameters.

The workaround we're trying now:

import pMemoize from 'p-memoize'
import pLimit from 'p-limit'

const memoize = (fn, options) => {
  const getLimiter = pMemoize(async (...args) => pLimit(1), { cacheKey: JSON.stringify })
  const memoized = pMemoize(fn, {cacheKey: JSON.stringify, ...options})
  return (async (...args) => {
    const limiter = await getLimiter(...args)
    return limiter(async () => memoized(...args))
  })
}

Effectively, this means that memoizable calls will wait until the "first" attempt has settled, and use the memoized result if it succeeded. If the first attempt failed they'll run the full funciton.

Which seems to work, but is pretty awkward to write and involves adding the p-limit dependency manually. Is there a better/recommended way of doing this? If not, maybe there could be an extra option for it?

[sindresorhus]

I believe this is how it worked before #32.

Also see: #34

[Richienb]

Implementation:

Use a queue to keep track of deferred promises returned from each invocation. Every time an item is added, check if the amount of currently running tasks is low enough to start it right away. Otherwise, this will be discovered as soon as a task finishes executing. Since the original intention of the refactor was to allow communication with a database, should the global amount of running tasks be stored there?

Do executions that hit the cache count towards the operation count? Does checking the cache in the first place count?

[Richienb]

Your use case appears to be default behaviour now as of v7.

[mmkal]

Ok, great. Not quite ready to upgrade but can reopen or make a new issue when I try it out