add decorated_fn.refresh(*args) feature

[sqlalchemy-bot]

Migrated issue, originally created by lxyu (lxyu)

do dogpile.cache have this feature yet:

one thread refresh the cached value while others directly return the old?

so I can update the cache in the background without slowing down the response time.

[sqlalchemy-bot]

lxyu (lxyu) wrote:

it would be great if I can call a refresh with this, some_decorated_func.refresh(*args).

[sqlalchemy-bot]

Michael Bayer (zzzeek) wrote:

I assume the "one thread" here is some background thread that's not directly returning to a user. you can build this approach using async_creation_runner here: http://dogpilecache.readthedocs.org/en/latest/api.html#module-dogpile.cache.region .

As far as dogpile.cache having the background thread system in place, that's out of scope. Users need to roll themselves how the background system should work (thread, message queue, etc.).

[sqlalchemy-bot]

Michael Bayer (zzzeek) wrote:

already implemented as async_creation_runner

[sqlalchemy-bot]

Changes by Michael Bayer (zzzeek):

• changed status to closed

[sqlalchemy-bot]

lxyu (lxyu) wrote:

didn't notice it already exists!

It works great except one thing: it seems I can't manually expire a value.

Maybe we can change the invalidate behavior to expire rather than delete? Or may be add a expire func?

[sqlalchemy-bot]

lxyu (lxyu) wrote:

The problem I'm solving is this:

I'm working on a server-client model based on thrift, sometimes I need to force ignore the cache and get a real-time value when some events happen, and if I got a real-time value, why not just refresh the cache in succession.

So how about a more specific feature, to get a non-cached value from a decorated function and refresh the cache in succession?

maybe new_value = some_decorated_func(*args, refresh=True) or just new_value = some_decorated_func.refresh(*args)?

[sqlalchemy-bot]

Changes by lxyu (lxyu):

• changed status to reopened

[sqlalchemy-bot]

Changes by Michael Bayer (zzzeek):

• removed labels: feature
• added labels: feature

[sqlalchemy-bot]

Changes by Michael Bayer (zzzeek):

• changed title from "About "refresh" feature" to "add decorated_fn.refresh(*args) feature"

[sqlalchemy-bot]

Michael Bayer (zzzeek) wrote:

pullreq ?

[sqlalchemy-bot]

Michael Bayer (zzzeek) wrote:

767dd6a

[sqlalchemy-bot]

Changes by Michael Bayer (zzzeek):

• changed status to closed

[sqlalchemy-bot]

lxyu (lxyu) wrote:

wow you're sooo fast!

I've prepared to give a pull request today, and you're faster than me again. ;)

And, glad to see the repo migrate to git!

[sqlalchemy-bot]

lxyu (lxyu) wrote:

Hi, I've monitored that in some rare cases, the refresh may not work as expected under distributed env.

Take the following example, suppose we have 2 thread calling refresh.

thread-a comes at 0s and somehow returns the value at 3s.

thread-b comes at 1s and returns value at 2s.

Then what if the value changed at 0.5s? The final cache is flushed to an expired one.

So maybe it require a lock in refresh too?

[sqlalchemy-bot]

Changes by lxyu (lxyu):

• changed status to reopened