redis acts like a db, not a cache #5

6a68 opened this Issue Jul 9, 2013 · 1 comment


None yet

1 participant

6a68 commented Jul 9, 2013

The default redis install process seen in sets redis to never evict stuff from memory. This led to an outage today because redis had gradually sucked up over a gig of memory, starving the VM.

The fix is pretty simple, we just set maxmemory to a limit in the redis conf file, and specify an eviction algorithm using maxmemory-policy.

@jedp is preparing a PR that'll set TTLs on keys. Once that's done, redis will reap expired keys by scanning 1000/second and killing expired ones. To guarantee we don't run out of memory, though, we can additionally set maxmemory-policy volatile-lru to evict least-recently-used expired keys when maxmemory is reached.

Annoyingly, installing redis via its install_server shell script doesn't give us any time between creating the config file and starting the server--and I've had trouble, for some reason, with /etc/init.d/redis_6379 stop not actually doing anything on the VM.

I assume we're building redis because the amzn repo doesn't have a redis package. I kinda think the best solution is to add an extra repo like EPEL, get redis from it, and then append the maxmemory and maxmemory-policy lines to it, assuming the init.d script will work properly with the upstream package:

echo "maxmemory 524288000 >> /etc/redis/6379.conf"
echo "maxmemory-policy volatile-lru >> etc/redis/6379.conf"
/etc/init.d/redis_6379 stop
/etc/init.d/redis_6379 start
6a68 commented Jul 12, 2013

@jedp After looking into enforcing memory limits programmatically, I think it'd be way simpler and adequate to just use TTLs to enforce a memory limit. Closing as fixed by your TTL changes--reopen if you disagree.

@6a68 6a68 closed this Jul 12, 2013
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment