Skip to content

Configurable PUBLISH events on Key expiry 2.2 #99

wants to merge 164 commits into from

7 participants

maxpert commented Sep 22, 2011

I've been working with redis lately [ blog post describes the whole problem and reasons], so while working with some of my problems I've added this new feature in Redis that's configurable (you can enable disable it) and makes Redis do a PUBLISH on key expiration. I would love this feature to be part of upcoming redis versions, so I am starting off with code directly.

pietern and others added some commits Dec 7, 2010
@pietern pietern Move logic concerned with zset ranges
This also optimizes ZREVRANGEBYSCORE for pathological cases where a
sorted set contains many elements with the same score. Previously,
it would traverse the list from back to front in such a case.
@pietern pietern Undo rename of function names where something went wrong 4769dc7
@antirez fixed typo in release notes 076ff11
@antirez overflow detection in INCR family functions 1443a81
@antirez removed some dead code, added evicted keys counter separated from exp…
…ired keys.
@antirez Merge branch '2.2' of into 2.2 54e9f28
@antirez TODO file just pointing to the master branch TODO file b902c14
@antirez Improved release notes. Added RC2 changes 9ac5be2
@antirez version bumped to 2.1.10 0540df2
@antirez Merge branch '2.2' of into 2.2 c4e93ca
@pietern pietern Update hiredis to 0.9.2 71b6f64
@pietern pietern Solaris doesn't support -rdynamic 736d447
@pietern pietern Solaris 10 doesn't know AF_LOCAL 78055bd
@pietern pietern limits.h is already included from redis.h 1408d14
@pietern pietern Fix compiler warnings on Solaris 454f4bf
@antirez Merge remote branch 'pietern/bench-fix' into 2.2 ae7db3f
@pietern pietern Remove glueoutputbuf option and broken code 90419b5
@pietern pietern Remove glueoutputbuf from redis.conf 01a0b09
@pietern pietern Remove newlines for networking errors 4b2499c
@antirez allocation stats in INFO f487bbb
@pietern pietern Compiler should decide on inlining 1b2b8cb
@pietern pietern Merge branch 'zset-range' into 2.2 7ac1b36
@antirez master-slave replication fixed, it was not listing any key using KEYS…
… command in the slave.
@antirez useless expression removed in zmalloc.c c13c408
@antirez implemented two new INFO fields showing the size of clients max input…
… and output buffers.
@antirez lame typo causing bad hashes memory characteristic f43502f
@antirez default hash encoding settings also fixed in redis.conf 99677e8
@pietern pietern Merge branch '2.2' of git:// into 2.2 8ff1353
@pietern pietern Remove client from list of unblocked clients when it is free'd e18b59a
@antirez version bumped to 2.1.11 2ccf6e8
@pietern pietern Reverse commits changing sorted set code for 2.2 3becef9
@antirez release notes updated for 2.2.0 RC3 825d290
@antirez typo fixed in release notes for 2.2.0 RC3 da67b5f
@antirez Fixed compilation on FreeBSD 9738e58
@antirez Merge branch '2.2' of into 2.2 d526d09
@antirez Fixed issue #435 and at the same time introduced explicit ping in the…
… master-slave channel that will detect a blocked master or a broken even if apparently connected TCP link.
@antirez portability fix in redis benchmark, using zu format specified with si…
…ze_t instead of lu
@antirez Merge branch '2.2' of into 2.2 ef8bc13
@antirez release noted updated, version bumped to 2.1.12 (2.2.0 RC4) 65e79a8
@antirez README updated 3ce30a6
@antirez other minor README changes 5ef8f95
@antirez add tarball script added 2f91d7a
@antirez propagate key eviction to slaves and AOF, like for expires, so that r…
…eplicas and AOFs are always consistent even in maxmemory mode.
@pietern pietern Reply with single null bulk for unsuccesful BRPOPLPUSH 669f302
@pietern pietern Clarify comment bbaf76e
@antirez If port zero is specified, Redis will not listen for TCP connections a36d524
@antirez Fix for the previous port 0 patch, thanks to Pieter for noticing the …
@antirez Merge remote branch 'pietern/2.2-brpoplpush' into 2.2 2446fbb
@antirez CONFIG GET/SET dir implemented ca9d961
@pietern pietern Morph SPOP to SREM for correct behavior on log replay/replication 97aeda9
@antirez CONFIG SET/GET for all the special encoding parameters of sets, lists…
…, hashes
@antirez version bumped to 2.2.0 and release notes added 1a6281f
@antirez check return value of getcwd() eb5aeaf
@antirez suppress a Linux warning, for 2.2 sake eece2d5
@antirez seed the random number generator at startup 7377aac
@giddie giddie Allow port to be 0 in configuration, in order to allow disabling of TCP. 1afef16
@antirez fixed color support for test suite. Now colors are shown as long as t…
…here is xterm somewhere inside the TERM env var
@antirez linenoise updated 58418d7
@antirez fixed client-libraries/README to reflect the new site path instead of…
… google code
@pietern pietern Restore argc/argv in EXEC after command is executed 61e2547
@antirez Redis 2.1.1, release notes updated 4e4b3b9

hmm. "Redis 2.1.1" typpo in commit message?

and others added some commits Mar 4, 2011
@antirez AOF file descriptor leak fixed 7a55d72
@antirez Fixed return value of GETRANGE / SUBSTR 74f6ee8
@antirez version is now 2.2.2 3eb3cc9
@antirez RELEASE NOTES updated 048c0f0
@pietern pietern Save RDB on SIGTERM (see issue #471) 2d2c17b
@pietern pietern Minor memory leak in redis-cli (issue #464) 45332cc
@pietern pietern Store SELECTed database for reconnect (issue #468) 699465c
@pietern pietern Show database number in cli prompt when non-zero 69bfdcf
@antirez Merge remote branch 'pietern/2.2-sigs' into 2.2 de5d4be
@antirez Fixed sdssplitargs() handling of hex-style escapes. abdbfc1
@antirez Fixed issue #503. MONITOR + QUIT could crash the server, there are ac…
…tually other interactions that could have the same effect (for instance Pub/Sub).
@antirez Merge branch '2.2' of into 2.2 207ca3c
@antirez fixed memory leak introduced with the previous commit. Many thanks to…
… Pieter Noordhuis for spotting it in no time
@antirez OBJECT command implemented cf6c3f4
@antirez version is now 2.2.3 920c45b
@antirez Merge branch '2.2' of into 2.2 0b01578
@antirez CHANGELOG updated 27fc619
@antirez make sure that OBJECT ENCODING returns skiplist for sorted sets, and …
…not raw, so that once we will merge specially encoded sorted sets everything will make sense.
@antirez Redis 2.2.4 2b88627
@antirez tests for variadic list push 1b90527
@antirez variadic LPUSH/RPUSH backported to 2.2 c138dc7
@antirez Merge branch '2.2' of into 2.2 6a9764d
@antirez Revert "variadic LPUSH/RPUSH backported to 2.2"
This reverts commit c138dc7.
Moving all the new stuff in 2.4 branch.
@antirez Revert "tests for variadic list push"
This reverts commit 1b90527.
Moving all the new stuff in 2.4 branch.
@pietern pietern Reformat AOF tests da06854
@pietern pietern Use correct argc/argv for cleanup when loading AOF bf36927
@antirez Redis 2.2.5 19b55f8
@antirez Fixed a bug with replication where SLAVEOF NO ONE caused a slave to c…
…lose the connection with its slaves
@pietern pietern Fix ziplist regression and update stresser 50e50d6
@antirez Redis version 2.2.6 97ddfbb
@antirez when Redis fails accepting a new connection reports the error at WARN…
…ING and not VERBOSE error level. Thanks to offby1 for proposing this in the Redis mailing list. #backport-candidate
@antirez Merge branch '2.2' of into 2.2 3554f09
@antirez Introduced a safe iterator interface that can be used to iterate whil…
…e accessing the dictionary at the same time. Now the default interface is consireded unsafe and should be used only with dictNext()
@antirez use safe Dict iteration in DEBUG DIGEST c86a4f9
@antirez removed assert causing an illegal memory access. This was responsible…
… of crashes during BLPOP and other list blocking operations.
@antirez Redis 2.2.7 a7fa2ba
@pietern pietern Convert intset encoded set before Z*STORE 1ffa5d7
@antirez regression for zinterstore bug c717adb
@antirez replication with expire test modified to produce no or less false fai…
antirez Fix for DEBUG DIGEST, key may expire on lookup cc8a0f8
@antirez Fixed bug introduced with the copy-on-write friendly iteartor 430719c
@antirez Fixed SINTER[STORE] problem related to the new copy on write safe ite…
@antirez Print version info before running the test 748a2da
@antirez ZUNIONSTORE new iterator API fix backported into 2.2 f447a7e
@antirez Comment typo fixed 480a2e7
@antirez Redis 2.2.8 ec27920
@antirez Fix for Pub/Sub system, introduced in Redis 2.2.6 with the new copy-o…
…n-write safe iterator semantics. In the hope this is the last bug I introduced this way.
@antirez Merge branch '2.2' of into 2.2 10e987c
@antirez Fix for issue 516, rewriting the command vector to correctly repliate…
… BRPOPLPUSH. Still to test everything, especially edge cases
@antirez Removed a leak in the BRPOPLPUSH unrelated to issue 561 d7061f8
@antirez modified the BRPOPLPUSH target field cleanup strategy to fix it the p…
…roper way.
@antirez use the new rewriteClientCommandVector() function for SPOP -> SREM re…
…plication translation as well.
@antirez test for the BRPOPLPUSH issue 561 related issues dfc7405
@antirez AOF log rewriting in slave performed after a successful sych with the…
… master was misplaced. Now it happens at the right time.
@antirez Merge remote branch 'origin/2.2' into 2.2 be87e76
@antirez Redis 2.2.9 7d8c021
@antirez Changelog typo fixed 25a585a
@pietern pietern Work with make v3.80 b13216a
@antirez Better init script 0b4eb5f
@antirez Fixed init script bug, thanks to Henrik Westphal 0df2e5b
@pietern pietern SETNX regression tests. 5afce9b
@antirez Fix for issue 566, fixed in a more long term way into the 2.4 branch 5d91926
@antirez redis-cli updated to the same version as the 2.4 branch 046b4f6
@antirez Redis 2.2.10 4ec8b73
@antirez Use a safe iterator while saving the DB, since the getExpire() functi…
…on will access the iterating dictionary.
@antirez Redis 2.2.11 a2d68e6
@antirez Partial fix for a problem with AOF and BRPOP. We have at least a sing…
…le evidence of an AOF that ended with a BRPOP call against an empty list, causing later issues at run time. Now the problem is detected while loading the AOF at startup, even if it is still to understand how the corrupted AOF was generated.
@antirez Slowlog backported to 2.2 ad6347b
@antirez ustime() backported from unstable, used by slow log 33d2761
@antirez no more allocation stats info in INFO, useless now that we have jemal…
@antirez colorized make for 2.2 as well 5a7bfb8
@antirez childrens -> children in INFO output field names. This is a backward …
…incompatible change, but really a minimal one.
@antirez Fixed typo in redis.conf related to slow log parameters 2e1adff
@antirez Fixed a problem with AOF when it is stopped via CONFIG SET appendonly…
… no. We used to kill the wrong child...
@hampus hampus Don't expire keys while loading AOF.
They will be expired (and a DEL will be logged) after the loading is done
@antirez use the new rewriteClientCommandVector() function for SPOP -> SREM re…
…plication translation as well.
@antirez don't process EXPIRE with negative TTL or EXPIREAT with time in the p…
…ast if we are a slave too (see Also propagate it as DEL.
@antirez Take a pointer to the relevant entry of the command table in the clie…
…nt structure. This is generally a more sounding design, simplifies a few functions prototype, and as a side effect fixes a bug related to the conversion of EXPIRE -1 to DEL: before of this fix Redis tried to convert it into an EXPIREAT in the AOF code, regardless of our rewrite of the command.
@antirez Manual merge after cherry-pick 891f919
@antirez removed a second copy of rewriteClientCommandVector put inside the so…
…urce code for a merge error
@antirez Fixed another merge typo 9702c92
@antirez Warn the user in redis.conf that VM is deprecated and discouraged eed4ec4
@antirez Warn the user that will try to enable VM that VM sucks. But still all…
…ows him to enable VM with a special option.
@antirez force VM usage when doing the VM test in 2.2, adding the really-use-v…
…m config option.
@hampus hampus Add test for incorrect expiration when loading AOF. aa96ebe
@antirez prepareForShutdown() fixed for correctness regarding what to do with …
…AOF and RDB persistence on exit.
@antirez Close the listening sockets. Apparently this allows faster restarts. ec7bbad
@antirez Redis 2.2.12 5960ac9
@pietern pietern Fix adding bulk reply when getcwd fails 62e6f6c
@antirez Merge remote-tracking branch 'origin/2.2' into 2.2 c1b2701
@pietern pietern Fix link order for ld --as-needed (thanks to Nigel Babu) e1f01c9
@hampus hampus Fix crash when chaining brpoplpush with other blocking commands. 1d05b53
@antirez fixed typos in the comments of rpoplpushHandlePush() 4793a2c
@antirez Fix for 2.2 TCMALLOC build, thanks to Pieter Noordhuis for the patch. 24a1580
@antirez Remove the write handler only if there are no longer objects in the o…
…utput queue AND if the static buffer is empty. This bug was the cause of a possible server-stop-responding-to-client bug under some specific work load. Thanks to Pieter Noordhuis for spotting and fixing it.
@antirez Version bumped to 2.2.13, changelog updated. a8ed663
@antirez Don't replicate/AOF SAVE 8ac3c86
@antirez Optimize LRANGE to scan the list starting from the head or the tail i…
…n order to traverse the minimal number of elements. Thanks to Didier Spezia for noticing the problem and providing a patch.
@pietern pietern Fix AOF race that may duplicate commands 5d46759
Zohaib Sibt-e-Hassan First hardcoded publish on expire publish event on channel 07dd061
Zohaib Sibt-e-Hassan Updating display message 80c1454
Zohaib Sibt-e-Hassan Fixing README 68e7809
Zohaib Sibt-e-Hassan Making publishing configurable c84ab40
antirez commented Sep 22, 2011

Hello, closing this pull request as it is a duplicated of #83

Please comment there if possible linking your proposed implementation and design.


@antirez antirez closed this Sep 22, 2011

omg, a hidden bug for a long time..

@JackieXie168 JackieXie168 pushed a commit that referenced this pull request Sep 16, 2014
@wmrowan wmrowan closes #99 41a8058
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.