New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Deprecate support for Apache ZooKeeper 3.4.x #10780
Comments
What would the migration path look like? Would people need to update Druid and their ZK quorum in a specific order or would various orders works? Any downtime needed? The Curator link says:
But I'm not sure if this refers to a 3.4.x client or a 3.4.x server (or both). |
If 3.5.x ZK server is backward compatible with 3.4.x client then we can update ZK server version in Druid integration test builds to ensure that it works as expected. |
I am assuming since newer client libs do not support 3.4.x that we would announce deprecation while continuing to use the existing client libs, to give time for operators to migrate clusters to 3.5.x. So, (if I understand correctly) the path initially would only involve upgrading ZK, which I would think could be done out of band with any Druid updates. Assuming this is correct, I think this should be relatively low risk to encourage cluster operators to upgrade their ZK cluster to 3.5.x (other than the risk of doing the upgrade of course); my anecdotal evidence for this being that I have been using it locally for all of my testing since shortly after 3.5.x became a 'stable' branch and haven't ran into any issues using the current set of client libraries.
Based on this comment in our pom.xml about zk version, I assume client libraries? |
In ZK 3.6.0 release notes (which I can't directly link to a specific one unfortunately), it states
I didn't see anything explicitly stating this for the 3.5.5 release notes (when 3.5.x became "stable"), but I assume it holds true. Updating the integration tests to use ZK 3.5.x (or 3.6.2? the downloads page seems to indicate this is the latest stable version) seems like a good first step to me. |
Based on what y'all are saying, it sounds like the migration plan would be:
|
@gianm yes, the migration path is to first upgrade ZK 3.5 before upgrade Druid to whichever version will no longer support ZK 3.4.x. We have been running Druid 0.19 successfully with both 3.4.x and 3.5.x ZK clusters, so any recent Druid version should run fine on either ZK version. |
since there is a clear and tested migration path, I agree with everything in #10780 (comment) |
In that case, I suggest add the tests for ZK 3.5 now, and we announce in the very next release that people should upgrade to ZK 3.5 when they can, because a future release will ship with the ZK 3.5 client, and will no longer support ZK 3.4 servers. |
It makes sense to me to deprecate 3.4 in 0.21.0. |
Should we go to 3.5.x or 3.6.x? Given that both are supported, I'm leaning towards 3.5.x, because it's more likely to be battle-tested / stable. That's mostly what we want from ZooKeeper anyway. It should also maximize server compatibility, since it seems that older ZK clients typically work well with newer servers. |
https://downloads.apache.org/zookeeper/stable/ shows 3.5.9 as stable version, while 3.6.2 is 'current' https://downloads.apache.org/zookeeper/current/, so I agree with going 3.5.x for now |
With this change, Druid will only support ZooKeeper 3.5.x and later. In order to support Java 15 we need to switch to ZK 3.5.x client libraries and drop support for ZK 3.4.x (see #10780 for the detailed reasons) * remove ZooKeeper 3.4.x compatibility * exclude additional ZK 3.5.x netty dependencies to ensure we use our version * keep ZooKeeper version used for integration tests in sync with client library version * remove the need to specify ZK version at runtime for docker * add support to run integration tests with JDK 15 * build and run unit tests with Java 15 in travis
Closed by #11073 |
ZooKeeper 3.4.x has been EOL for some time, and its client libraries are no longer supported by the community.
In order to support newer JDK versions (i.e. JDK 14, 15 and above) Druid will need to upgrade to 3.5.x client libraries (see https://issues.apache.org/jira/browse/ZOOKEEPER-3779)
Unfortunately Apache Curator does not support ZooKeeper 3.4.x with newer clients libraries (see https://curator.apache.org/zk-compatibility-34.html) and as such we will need to drop 3.4.x support in order to upgrade.
I propose we deprecate support for ZooKeeper 3.4.x in the next release, and remove support in a subsequent one, once we validate that 3.5.x client libraries perform as expected.
The text was updated successfully, but these errors were encountered: