handling memcache failures

[tzachyrm]

Some memcached backeds return False from the add function due to sporadic failures. In this case the following call to incr will raise a ValueError, and the call to get will return None.

[willkg]

Nit: Comments require two space before them when they're inline like this.

[willkg]

When you say "some memcached backends", which ones are you talking about specifically? Are there links to documentation about the issue?

[adamchainz]

This affected us @YPlan too 😄

The cause of this is not 'sporadic', but a race condition with evictions; add() returns False since the key is there when it checks, but by the time incr() or get() runs, the key has been evicted due to other parallel traffic, or expired. This is only something you'd see with enough traffic.

The ValueError-on-failed-incr() behaviour is for all Django built-in memcached backends - the behaviour is normalized in BaseMemcachedCache: https://github.com/django/django/blob/master/django/core/cache/backends/memcached.py#L105

We are currently using a similar patch like this one but with some tests 😄 I've submitted it as a second PR here: #81

[tzachyrm]

@willkg In my case it's the Google App Engine memcache. The docs state that:

Memcache operations can fail for various reasons. Applications should be designed to catch failed operations without exposing these errors to end users. This applies especially to Set operations.

[jsocol]

Closing in favor of #81 (which itself may be fixed)

[jsocol]

@adamchainz et al, can you check to see if #88 fixed this for you?

[adamchainz]

Yes it did./