Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Flush propagation list in active-expire of writable replicas to fix an assertion #11615

Merged
merged 2 commits into from Dec 15, 2022

Conversation

guybe7
Copy link
Collaborator

@guybe7 guybe7 commented Dec 13, 2022

Make sure to flush also_propagate after expiring from expireSlaveKeys to avoid an assertion in beforeSleep.

Note that this is an edge case that only happens in case volatile keys were created directly on a writable replica, and that anyway nothing is propagated to sub-replicas.

For Redis 7.2, We need to honor the post-execution-unit API and call it after each KSN.

Otherwise the DELs won't be propagated.

Note that this is an edge-case that only happens in case volatile keys were created directly on a writable replica
@oranagra
Copy link
Member

@guybe7 unless i'm missing something, the top comment is misleading.
sub-replicas don't really propagate anything to their replicas (it'll mess up the replication offset).

see:

void replicationFeedSlaves(list *slaves, int dictid, robj **argv, int argc) {

    /* If the instance is not a top level master, return ASAP: we'll just proxy
     * the stream of data we receive from our master instead, in order to
     * propagate *identical* replication stream. In this way this slave can
     * advertise the same replication ID as the master (since it shares the
     * master replication history and has the same backlog and offsets). */
    if (server.masterhost != NULL) return;

so this fix is needed for the sake of modules maybe, but not for replication.
please ack.

@guybe7
Copy link
Collaborator Author

guybe7 commented Dec 15, 2022

@oranagra so when a writable replica actively expires a key it doesn't send that DEL? but it does emit a KSN? This could be problematic for modules, the sub-replica will not receive what the module propagated from within the PEJ (post execution-unit job)

maybe we want to keep the code as-is and just document somewhere that PEJ doesn't work in case of writable-replica active-expire? or better to leave the code and just document that in this case nothing is propagated?

@MeirShpilraien ^

@oranagra
Copy link
Member

i think we rather keep honoring the module API (current fix in the PR), since i don't know what the module will do.

writable replicas, and specifically ones with local expiry are a mess anyway, i hope the use cases for that don't involve modules, and if they do, the module should probably be explicitly designed for that.

@guybe7
Copy link
Collaborator Author

guybe7 commented Dec 15, 2022

@oranagra updated the top comment

@oranagra
Copy link
Member

oranagra commented Feb 8, 2023

marking this to be backported for 7.0 with a twist, see #11778 (comment)

oranagra pushed a commit that referenced this pull request Feb 20, 2023
…#11789)

This test case is to cover a edge scenario: when a writable replica enabled AOF
at the same time, active expiry keys which was created in writable replicas should
propagate to the AOF file, and some versions might crash (fixed by #11615).
For details, please refer to #11778
odaiwa pushed a commit to odaiwa/redis that referenced this pull request Feb 20, 2023
…redis#11789)

This test case is to cover a edge scenario: when a writable replica enabled AOF
at the same time, active expiry keys which was created in writable replicas should
propagate to the AOF file, and some versions might crash (fixed by redis#11615).
For details, please refer to redis#11778
@oranagra oranagra changed the title Call postExecutionUnitOperations in active-expire of writable replicas Flush propagation list in active-expire of writable replicas to fix an assertion Feb 26, 2023
@oranagra oranagra added the release-notes indication that this issue needs to be mentioned in the release notes label Feb 26, 2023
oranagra pushed a commit to oranagra/redis that referenced this pull request Feb 27, 2023
redis#11615)

We need to honor the post-execution-unit API and call it after each KSN

Note that this is an edge case that only happens in case volatile keys were
created directly on a writable replica, and that anyway nothing is propagated to sub-replicas

Co-authored-by: Oran Agra <oran@redislabs.com>
(cherry picked from commit df327b8)
oranagra pushed a commit to oranagra/redis that referenced this pull request Feb 27, 2023
…redis#11789)

This test case is to cover a edge scenario: when a writable replica enabled AOF
at the same time, active expiry keys which was created in writable replicas should
propagate to the AOF file, and some versions might crash (fixed by redis#11615).
For details, please refer to redis#11778

(cherry picked from commit 40659c3)
@oranagra oranagra mentioned this pull request Feb 27, 2023
oranagra pushed a commit that referenced this pull request Feb 28, 2023
#11615)

We need to honor the post-execution-unit API and call it after each KSN

Note that this is an edge case that only happens in case volatile keys were
created directly on a writable replica, and that anyway nothing is propagated to sub-replicas

Co-authored-by: Oran Agra <oran@redislabs.com>
(cherry picked from commit df327b8)
oranagra pushed a commit that referenced this pull request Feb 28, 2023
…#11789)

This test case is to cover a edge scenario: when a writable replica enabled AOF
at the same time, active expiry keys which was created in writable replicas should
propagate to the AOF file, and some versions might crash (fixed by #11615).
For details, please refer to #11778

(cherry picked from commit 40659c3)
Mixficsol pushed a commit to Mixficsol/redis that referenced this pull request Apr 12, 2023
redis#11615)

We need to honor the post-execution-unit API and call it after each KSN

Note that this is an edge case that only happens in case volatile keys were
created directly on a writable replica, and that anyway nothing is propagated to sub-replicas

Co-authored-by: Oran Agra <oran@redislabs.com>
Mixficsol pushed a commit to Mixficsol/redis that referenced this pull request Apr 12, 2023
…redis#11789)

This test case is to cover a edge scenario: when a writable replica enabled AOF
at the same time, active expiry keys which was created in writable replicas should
propagate to the AOF file, and some versions might crash (fixed by redis#11615).
For details, please refer to redis#11778
enjoy-binbin pushed a commit to enjoy-binbin/redis that referenced this pull request Jul 31, 2023
redis#11615)

We need to honor the post-execution-unit API and call it after each KSN

Note that this is an edge case that only happens in case volatile keys were
created directly on a writable replica, and that anyway nothing is propagated to sub-replicas

Co-authored-by: Oran Agra <oran@redislabs.com>
enjoy-binbin pushed a commit to enjoy-binbin/redis that referenced this pull request Jul 31, 2023
…redis#11789)

This test case is to cover a edge scenario: when a writable replica enabled AOF
at the same time, active expiry keys which was created in writable replicas should
propagate to the AOF file, and some versions might crash (fixed by redis#11615).
For details, please refer to redis#11778
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
release-notes indication that this issue needs to be mentioned in the release notes
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

None yet

2 participants