Serving stale response from cache middleware

[alexgleason]

I'm building a memory cache middleware, but I'm stuck on making it serve a stale response and then build the new response in the background. Is this possible?

import { type MiddlewareHandler } from '@/deps.ts';

interface CacheOpts {
  expires: number;
}

/** In-memory cache middleware. */
export const cache = (opts: CacheOpts): MiddlewareHandler => {
  let response: Response | undefined;
  let expires = Date.now() + opts.expires;

  return async (c, next) => {
    const now = Date.now();
    const expired = now > expires;

    async function updateCache() {
      await next();
      const res = c.res.clone();
      if (res.ok) {
        response = res;
      }
      return res;
    }

    if (response && !expired) {
      return response.clone();
    } else {
      expires = Date.now() + opts.expires;
      if (response && expired) {
        const stale = response.clone();
        updateCache();
        return stale;
      }
      return await updateCache();
    }
  };
};

When updateCache() is called, it throws an error:

error: Uncaught (in promise) TypeError: Body is unusable.
      const res = c.res.clone();
                        ^
    at Response.clone (ext:deno_fetch/23_response.js:398:13)
    at updateCache (file:///home/alex/Projects/ditto/src/middleware/cache.ts:20:25)
    at eventLoopTick (ext:core/01_core.js:169:7)
Watcher Process failed. Restarting on file change...

Is seems I'm unable to call the next controller after I've already returned a response.

Is there any way around this?

[alexgleason]

This seems to fix it, but it's pretty dirty:

        updateCache();
+       await new Promise((resolve) => setTimeout(resolve, 0));
        return stale;

[yusukebe]

Hi @alexgleason

Do you run it on Deno? I've tried the same code on Deno, but the error was not thrown. Or how about await there:

if (response && expired) {
  const stale = response.clone()
  await updateCache()
  return stale
}

[alexgleason]

Yes, on Deno. To throw the error you have to set a low expires, like 3000, so it will trigger updateCache. Using await there defeats the purpose because it will call the slow endpoint. I should probably solve this problem inside my controller.

[yusukebe]

Using await there defeats the purpose because it will call the slow endpoint.

Indeed.

I could reproduce if I set the setTimeout() in the handler:

app.get('/', async (c) => {
  await new Promise((resolve) => setTimeout(resolve, 300))
  return c.text('Hello')
})

Hmm. But I think it is not possible if it does not await updateCache() because updateCache() should handle returned stale response, which is already used.