Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Integrate closed replicated indices with closed follower indices #42442

Open
dnhatn opened this issue May 23, 2019 · 3 comments
Open

Integrate closed replicated indices with closed follower indices #42442

dnhatn opened this issue May 23, 2019 · 3 comments
Assignees
Labels
:Distributed/CCR Issues around the Cross Cluster State Replication features Team:Distributed Meta label for distributed team

Comments

@dnhatn
Copy link
Member

dnhatn commented May 23, 2019

A spin-off from #33888.

The follower indices do not always have the global checkpoint equals to max_seq_no when closing. Therefore, closing follower indices can make the cluster red. We need to make an alternative sanity check for follower closed indices.

Relates #33888
Relates #38767
Relates #41317

@dnhatn dnhatn added the :Distributed/Distributed A catch all label for anything in the Distributed Area. If you aren't sure, use this one. label May 23, 2019
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-distributed

@tlrx tlrx mentioned this issue May 23, 2019
50 tasks
@rjernst rjernst added the Team:Distributed Meta label for distributed team label May 4, 2020
@dnhatn dnhatn removed their assignment Feb 10, 2021
@pxsalehi pxsalehi added :Distributed/CCR Issues around the Cross Cluster State Replication features and removed :Distributed/Distributed A catch all label for anything in the Distributed Area. If you aren't sure, use this one. labels Jul 29, 2022
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-distributed (Team:Distributed)

@tlrx
Copy link
Member

tlrx commented Jun 29, 2023

Quoting Yannick in #38767 (comment):

We need to think how to integrate closed replicated indices with closed follower indices. In the current form, closing follower indices will result in them not being allocated, making the cluster go red. Perhaps instead of requiring max sequence number to be equal to global checkpoint for these shards, which is not guaranteed by the CCR following logic, it would be sufficient to check that max seq number and global checkpoint agree with the primary and require a sync id marker (i.e. perform a synced flush during the VerifyShardBeforeCloseAction) so that peer recovery will be instantaneous.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Distributed/CCR Issues around the Cross Cluster State Replication features Team:Distributed Meta label for distributed team
Projects
None yet
Development

No branches or pull requests

6 participants