Support soft ttl

[argaen]

Right now there is only the option for ttl which removes the key when it expires in any backend.

There is an middle solution which may return expire values but avoids recalculating the value when the user request occurs.

Current behavior:

await cache.set("key", "value", ttl=1)  # Key is removed after 1 second
await asyncio.sleep(1)
await cache.get("key")  # This will be None now user would normally call the function to recalculate the value and store it in cache 

Now the new behavior:

await cache.set("key", "value", soft_ttl=20, ttl=120) # Key is stale after 20 seconds and removed after 2 minutes, arg name can be soft_ttl or stale_after or ...
await asyncio.sleep(20)
await cache.get("key")  # Stale value will be returned instead of None

In above code there is missing the way of triggering automatically the recalculation of the new value for the key so it never expires and refreshes values. Need to check what's the best and cleanest way of doing it.

[argaen]

Closing this. Will open when there is a real use case

[krrg]

I know this is an old thread, but I was actually trying to solve a problem that would have benefited greatly from this. My use case is in a web context; a user needs to make a low-latency request, and it is okay if the data returned is a little bit stale.

Since refreshing the cache can take upwards of 5-10 seconds (it requires computing results across a big cluster of machines), I would prefer to just return the stale/cached data to the user and start the refresh process in the background rather than making the user wait for the very slow refresh.

[argaen]

The way I thought on implementing this is by storing "complex" data with the timestamp of when the data would be stale and then on every get, checking that. However, it makes things complex and for now I would like to keep them simple unless there is more people interested. If you have implementation ideas, I'm happy (or even better, feel brave and would like to actually implement it :D)

[cancan101]

I can provide another use case for the stale TTL: when the cached operation has a transient failure (for example when fetching data from a 3rd party). In this case I may want to return the prior value if it is within a more generous ttl.

As an example:

cached_value, age = read_from_cache(key)

if cached_value and age < ttl_soft:
   return cached_value

try:
   new_value = expensive_operation(key)
   save_cache(new_value, now)
   return new_value
except ex:
   if age < ttl:
      return cached_value
   raise ex