Skip to content
Mirror of Apache Hadoop ZooKeeper
Java C++ C JavaScript Python Shell Other
Branch: master
Clone or download

Latest commit

symat and nkalmar ZOOKEEPER-3829: fix rolling restart when dynamic reconfig is disabled
In four different Jira tickets (ZOOKEEPER-3829, ZOOKEEPER-3830, ZOOKEEPER-3814, ZOOKEEPER-3842) we saw different errors when dynamic reconfig was disabled and we used rolling restart to change the quorum membership configuration. These rolling restart sequences was working fine on 3.4 but caused errors in 3.5 or 3.6.

In worst case the rolling restart leads to the scenario that we had an elected leader which was up but unable to commit any changes. This happens, when the quorum is extended with a new member in the following sequence:
* start server.1, server.2, server.3 (with config: 1,2,3)
* start server.4 (with config 1,2,3,4)
* stop server.1, then restart it with config 1,2,3,4
* stop server.2, then restart it with config 1,2,3,4
* stop server.3, then restart it with config 1,2,3,4
* at this point leader is server.4, but it can not commit any transaction

An other error we saw was when we changed a host name of an existing member (removing server.5 and add a new host as server.6). In this case we found in the logs of the new server (server.6) that it is still tried to connect to the old invalid server (server.5), although it was missing from it's config. The same problem remained even after making a full rolling-restart on all the nodes.

In this patch I try to fix these issues without breaking anything. The patch contains the following changes:
* We are making sure that neither the committed, nor the last seen config gets updated if dynamic reconfig is disabled.
* It is not possible now to start the leader without the ability of committing transaction, when dynamic reconfig is disabled (this is only needed to avoid a reconfig edge-case).
* I added a testcase simulating the enablement of dynamic reconfig using rolling restart
* I added a few more unit tests to cover rolling restart scenarios. (the tests are failing without the patch but succeeding after applying it).
* The enablement / disablement of reconfig is getting initialized now in the QuorumPeer and gets propagated to the other classes. This was needed for the rolling restart tests to be able to enable/disable reconfig only for the newly created servers without affecting the servers running already in the same JVM.

I also tested the changes with docker, using: https://github.com/symat/zookeeper-docker-test

target branches: 3.5, 3.6, master

Author: Mate Szalay-Beko <symat@apache.org>

Reviewers: Enrico Olivelli <eolivelli@apache.org>, Norbert Kalmar <nkalmar@apache.org>

Closes #1356 from symat/ZOOKEEPER-3829
Latest commit e91455c May 29, 2020

Files

Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
bin ZOOKEEPER-3818: client SSL support for zkServer.sh status command May 12, 2020
conf ZOOKEEPER-1260: Audit logging in ZooKeeper servers. Nov 11, 2019
dev/docker ZOOKEEPER-1112: Add (Cyrus) SASL authentication support to C client l… Jan 22, 2020
tools/cmake/Modules ZOOKEEPER-1112: Add (Cyrus) SASL authentication support to C client l… Jan 22, 2020
zookeeper-assembly ZOOKEEPER-3791: Miscellaneous Maven fixes May 4, 2020
zookeeper-client ZOOKEEPER-1998: Allow C client to throttle host name resolutions May 17, 2020
zookeeper-contrib ZOOKEEPER-3791: Miscellaneous Maven fixes May 4, 2020
zookeeper-docs ZOOKEEPER-3792: fix apidocs path May 21, 2020
zookeeper-it ZOOKEEPER-3791: Miscellaneous Maven fixes May 4, 2020
zookeeper-jute ZOOKEEPER-3791: Miscellaneous Maven fixes May 4, 2020
zookeeper-metrics-providers ZOOKEEPER-3791: Miscellaneous Maven fixes May 4, 2020
zookeeper-recipes ZOOKEEPER-3834: Do Not Set Explicit Test Includes in POM May 20, 2020
zookeeper-server ZOOKEEPER-3829: fix rolling restart when dynamic reconfig is disabled May 29, 2020
.gitattributes ZOOKEEPER-1966. VS and line breaks (Orion Hodson via fpj) Jul 16, 2014
.gitignore ZOOKEEPER-3738: Use mavanagaiata for git commit id Mar 2, 2020
.travis.yml ZOOKEEPER-3791: Miscellaneous Maven fixes May 4, 2020
LICENSE.txt Created notice and renamed the existing license/readme files. See ZOO… Jun 24, 2008
NOTICE.txt ZOOKEEPER-3745: Update copyright notices from 2019 to 2020 Mar 14, 2020
README.md ZOOKEEPER-3668: Clean up release package for 3.6.0 Jan 6, 2020
README_packaging.md ZOOKEEPER-3567: add SSL support for zkpython Jan 27, 2020
checkstyle-simple.xml ZOOKEEPER-3517: Turn on BookKeeper checkstyle configuration at projec… Aug 27, 2019
checkstyle-strict.xml ZOOKEEPER-3468: Enable BookKeeper checkstyle configuration on zookeep… Jul 29, 2019
checkstyleSuppressions.xml ZOOKEEPER-3832: ZKHostnameVerifier rejects valid certificates with su… May 21, 2020
excludeFindBugsFilter.xml ZOOKEEPER-3620: Allow to override calls to System.exit in server side… Nov 28, 2019
owaspSuppressions.xml ZOOKEEPER-3817: suppress log4j SmtpAppender related CVE-2020-9488 May 4, 2020
pom.xml ZOOKEEPER-3761: Upgrade JLine version to 2.14.6 May 12, 2020
zk-merge-pr.py ZOOKEEPER-3782: Replace filter() with list comprehension for returnin… Apr 8, 2020

README.md

Apache ZooKeeper Build Status Maven Central License

alt text

For the latest information about Apache ZooKeeper, please visit our website at:

http://zookeeper.apache.org/

and our wiki, at:

https://cwiki.apache.org/confluence/display/ZOOKEEPER


Packaging/release artifacts

Either downloaded from https://zookeeper.apache.org/releases.html or found in zookeeper-assembly/target directory after building the project with maven.

apache-zookeeper-[version].tar.gz

    Contains all the source files which can be built by running:
    mvn clean install

    To generate an aggregated apidocs for zookeeper-server and zookeeper-jute:
    mvn javadoc:aggregate
    (generated files will be at target/site/apidocs)

apache-zookeeper-[version]-bin.tar.gz

    Contains all the jar files required to run ZooKeeper
    Full documentation can also be found in the docs folder

As of version 3.5.5, the parent, zookeeper and zookeeper-jute artifacts are deployed to the central repository after the release is voted on and approved by the Apache ZooKeeper PMC:

https://repo1.maven.org/maven2/org/apache/zookeeper/zookeeper/

Java 8

If you are going to compile with Java 1.8, you should use a recent release at u211 or above.

Contributing

We always welcome new contributors to the project! See How to Contribute for details on how to submit patch through pull request and our contribution workflow.

You can’t perform that action at this time.