New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Flush propagation list in active-expire of writable replicas to fix an assertion #11615
Conversation
Otherwise the DELs won't be propagated. Note that this is an edge-case that only happens in case volatile keys were created directly on a writable replica
@guybe7 unless i'm missing something, the top comment is misleading. see: void replicationFeedSlaves(list *slaves, int dictid, robj **argv, int argc) {
/* If the instance is not a top level master, return ASAP: we'll just proxy
* the stream of data we receive from our master instead, in order to
* propagate *identical* replication stream. In this way this slave can
* advertise the same replication ID as the master (since it shares the
* master replication history and has the same backlog and offsets). */
if (server.masterhost != NULL) return; so this fix is needed for the sake of modules maybe, but not for replication. |
@oranagra so when a writable replica actively expires a key it doesn't send that DEL? but it does emit a KSN? This could be problematic for modules, the sub-replica will not receive what the module propagated from within the PEJ (post execution-unit job) maybe we want to keep the code as-is and just document somewhere that PEJ doesn't work in case of writable-replica active-expire? or better to leave the code and just document that in this case nothing is propagated? |
i think we rather keep honoring the module API (current fix in the PR), since i don't know what the module will do. writable replicas, and specifically ones with local expiry are a mess anyway, i hope the use cases for that don't involve modules, and if they do, the module should probably be explicitly designed for that. |
@oranagra updated the top comment |
marking this to be backported for 7.0 with a twist, see #11778 (comment) |
…redis#11789) This test case is to cover a edge scenario: when a writable replica enabled AOF at the same time, active expiry keys which was created in writable replicas should propagate to the AOF file, and some versions might crash (fixed by redis#11615). For details, please refer to redis#11778
redis#11615) We need to honor the post-execution-unit API and call it after each KSN Note that this is an edge case that only happens in case volatile keys were created directly on a writable replica, and that anyway nothing is propagated to sub-replicas Co-authored-by: Oran Agra <oran@redislabs.com> (cherry picked from commit df327b8)
…redis#11789) This test case is to cover a edge scenario: when a writable replica enabled AOF at the same time, active expiry keys which was created in writable replicas should propagate to the AOF file, and some versions might crash (fixed by redis#11615). For details, please refer to redis#11778 (cherry picked from commit 40659c3)
#11615) We need to honor the post-execution-unit API and call it after each KSN Note that this is an edge case that only happens in case volatile keys were created directly on a writable replica, and that anyway nothing is propagated to sub-replicas Co-authored-by: Oran Agra <oran@redislabs.com> (cherry picked from commit df327b8)
…#11789) This test case is to cover a edge scenario: when a writable replica enabled AOF at the same time, active expiry keys which was created in writable replicas should propagate to the AOF file, and some versions might crash (fixed by #11615). For details, please refer to #11778 (cherry picked from commit 40659c3)
redis#11615) We need to honor the post-execution-unit API and call it after each KSN Note that this is an edge case that only happens in case volatile keys were created directly on a writable replica, and that anyway nothing is propagated to sub-replicas Co-authored-by: Oran Agra <oran@redislabs.com>
…redis#11789) This test case is to cover a edge scenario: when a writable replica enabled AOF at the same time, active expiry keys which was created in writable replicas should propagate to the AOF file, and some versions might crash (fixed by redis#11615). For details, please refer to redis#11778
redis#11615) We need to honor the post-execution-unit API and call it after each KSN Note that this is an edge case that only happens in case volatile keys were created directly on a writable replica, and that anyway nothing is propagated to sub-replicas Co-authored-by: Oran Agra <oran@redislabs.com>
…redis#11789) This test case is to cover a edge scenario: when a writable replica enabled AOF at the same time, active expiry keys which was created in writable replicas should propagate to the AOF file, and some versions might crash (fixed by redis#11615). For details, please refer to redis#11778
Make sure to flush also_propagate after expiring from expireSlaveKeys to avoid an assertion in beforeSleep.
Note that this is an edge case that only happens in case volatile keys were created directly on a writable replica, and that anyway nothing is propagated to sub-replicas.
For Redis 7.2, We need to honor the post-execution-unit API and call it after each KSN.