Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade ZK from 3.5.9 to 3.5.10 to avoid data inconsistency risk #13715

Merged
merged 2 commits into from Mar 15, 2023

Conversation

asdf2014
Copy link
Member

Description

Upgrade ZK from 3.5.9 to 3.5.10 to avoid data inconsistency risk:

If the leader crashes after sending a SNAP sync to a learner, but before sending the NEWLEADER message, the learner will not save the snapshot to disk. But it will advance its lastProcessedZxid to that from the snapshot (call it Zxid X)

A new leader will get elected, and it will resync our learner again immediately. But this time, it will use the incremental DIFF method, starting from Zxid X. A DIFF-based resync does not trigger snapshots, so the learner is still holding the original snapshot purely in memory. If the learner restarts after that, it will silently lose all the data up to Zxid X.

More details please refer to https://issues.apache.org/jira/browse/ZOOKEEPER-3642 and https://zookeeper.apache.org/doc/r3.5.10/releasenotes.html


This PR has:

  • been self-reviewed.
  • added documentation for new or modified features or behaviors.
  • a release note entry in the PR description.
  • added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links.
  • added or updated version, license, or notice information in licenses.yaml
  • added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
  • added unit tests or modified existing tests to cover new code paths, ensuring the threshold for code coverage is met.
  • added integration tests.
  • been tested in a test Druid cluster.

@abhishekagarwal87
Copy link
Contributor

@asdf2014 - can you resolve the travis failures?

@asdf2014
Copy link
Member Author

asdf2014 commented Mar 8, 2023

Hi @abhishekagarwal87, the failure looks irrelevant, it is gone after merging the latest main branch 😂

@abhishekagarwal87 abhishekagarwal87 merged commit cee2dfd into apache:master Mar 15, 2023
@asdf2014 asdf2014 deleted the upgrade_zk branch March 16, 2023 03:12
@clintropolis clintropolis added this to the 26.0 milestone Apr 10, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants