Real LRU eviction algorithm #1459

Open
wants to merge 2 commits into
from

Conversation

Projects
None yet
5 participants
@fadimko

fadimko commented Dec 11, 2013

Patch that implements allkeys-real-lru maxmemory policy.

@antirez

This comment has been minimized.

Show comment Hide comment
@antirez

antirez Dec 12, 2013

Owner

This is very interesting, thanks! In the past this was never attempted for the fear of the additional memory usage, however there is to evaluate the cost with care and check if there are fields we could remove from the object structure to make room... checking what allocation class of jemalloc to target, and so forth.

Also I would love to write a simulation to check the real advantage of this feature over the imprecise sampling we do. The simulation should just write keys on a Redis instance configured with a given maxmemory setting. We could write the application so that it remembers what are the recently used objects, and verify what is inside the database. Your implementation should score 100% (100% of objects inside the database are the ones with the smallest access time), while the current implementation should not score so well.

I think it is an important step to actually measure what we gain exactly. Great contribution btw! Thanks again.

Owner

antirez commented Dec 12, 2013

This is very interesting, thanks! In the past this was never attempted for the fear of the additional memory usage, however there is to evaluate the cost with care and check if there are fields we could remove from the object structure to make room... checking what allocation class of jemalloc to target, and so forth.

Also I would love to write a simulation to check the real advantage of this feature over the imprecise sampling we do. The simulation should just write keys on a Redis instance configured with a given maxmemory setting. We could write the application so that it remembers what are the recently used objects, and verify what is inside the database. Your implementation should score 100% (100% of objects inside the database are the ones with the smallest access time), while the current implementation should not score so well.

I think it is an important step to actually measure what we gain exactly. Great contribution btw! Thanks again.

@1602

This comment has been minimized.

Show comment Hide comment
@1602

1602 Dec 12, 2013

I think this is must have feature! Thanks for implementing this, @fadimko.

1602 commented Dec 12, 2013

I think this is must have feature! Thanks for implementing this, @fadimko.

+void dlTouch (dictList *dl, struct dictEntry *de);
+
+#endif /* __DICTLIST_H */
+/* !LRU */

This comment has been minimized.

Show comment Hide comment
@leoromanovsky

leoromanovsky Dec 15, 2013

newline :-)

@dustindoiron

This comment has been minimized.

Show comment Hide comment
@dustindoiron

dustindoiron Jan 2, 2014

Would be fantastic to get this added, 👍.

Would be fantastic to get this added, 👍.

@antirez

This comment has been minimized.

Show comment Hide comment
@antirez

antirez Mar 19, 2014

Owner

Hello, I'm starting a code review about this feature with the intention to merge it and/or work in the direction of fixing the issues if any in order to eventually merge it. Thanks.

Owner

antirez commented Mar 19, 2014

Hello, I'm starting a code review about this feature with the intention to merge it and/or work in the direction of fixing the issues if any in order to eventually merge it. Thanks.

@antirez antirez removed this from the Redis 3.0 milestone May 15, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment