fix-npe-ZkBookieRackAffinityMapping #11947

gaozhangmin · 2021-09-07T10:49:44Z

Motivation

NPE when ZkBookieRackAffinityMapping

17:24:03.724 [main-SendThread(bigdata-pulsar-bookie004.ys:2181)] INFO  org.apache.zookeeper.ClientCnxn - Session establishment complete on server bigdata-pulsar-bookie004.ys/10.89.146.53:2181, session id = 0x57ba9d6a8d800a8, negotiated timeout = 30000
17:24:03.725 [main-EventThread] INFO  org.apache.bookkeeper.zookeeper.ZooKeeperWatcherBase - ZooKeeper client is connected now.
17:24:03.792 [main] ERROR org.apache.pulsar.zookeeper.ZkBookieRackAffinityMapping - Error creating zookeeper client
java.lang.IllegalArgumentException: Path must not end with / character
        at org.apache.zookeeper.common.PathUtils.validatePath(PathUtils.java:55) ~[org.apache.zookeeper-zookeeper-3.6.3.jar:3.6.3]
        at org.apache.zookeeper.client.ConnectStringParser.<init>(ConnectStringParser.java:61) ~[org.apache.zookeeper-zookeeper-3.6.3.jar:3.6.3]
        at org.apache.zookeeper.ZooKeeper.createDefaultHostProvider(ZooKeeper.java:1515) ~[org.apache.zookeeper-zookeeper-3.6.3.jar:3.6.3]
        at org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:1108) ~[org.apache.zookeeper-zookeeper-3.6.3.jar:3.6.3]
        at org.apache.bookkeeper.zookeeper.ZooKeeperClient.<init>(ZooKeeperClient.java:286) ~[org.apache.bookkeeper-bookkeeper-server-4.15.0-SNAPSHOT.jar:4.15.0-SNAPSHOT]
        at org.apache.bookkeeper.zookeeper.ZooKeeperClient$Builder.build(ZooKeeperClient.java:247) ~[org.apache.bookkeeper-bookkeeper-server-4.15.0-SNAPSHOT.jar:4.15.0-SNAPSHOT]
        at org.apache.pulsar.zookeeper.ZkBookieRackAffinityMapping.getAndSetZkCache(ZkBookieRackAffinityMapping.java:138) [org.apache.pulsar-pulsar-zookeeper-utils-2.8.0.jar:2.8.0]
        at org.apache.pulsar.zookeeper.ZkBookieRackAffinityMapping.setConf(ZkBookieRackAffinityMapping.java:75) [org.apache.pulsar-pulsar-zookeeper-utils-2.8.0.jar:2.8.0]
        at org.apache.bookkeeper.client.RackawareEnsemblePlacementPolicyImpl.initialize(RackawareEnsemblePlacementPolicyImpl.java:264) [org.apache.bookkeeper-bookkeeper-server-4.15.0-SNAPSHOT.jar:4.15.0-SNAPSHOT]
        at org.apache.bookkeeper.client.RackawareEnsemblePlacementPolicyImpl.initialize(RackawareEnsemblePlacementPolicyImpl.java:80) [org.apache.bookkeeper-bookkeeper-server-4.15.0-SNAPSHOT.jar:4.15.0-SNAPSHOT]
        at org.apache.bookkeeper.client.BookKeeper.initializeEnsemblePlacementPolicy(BookKeeper.java:582) [org.apache.bookkeeper-bookkeeper-server-4.15.0-SNAPSHOT.jar:4.15.0-SNAPSHOT]
        at org.apache.bookkeeper.client.BookKeeper.<init>(BookKeeper.java:506) [org.apache.bookkeeper-bookkeeper-server-4.15.0-SNAPSHOT.jar:4.15.0-SNAPSHOT]
        at org.apache.bookkeeper.client.BookKeeper$Builder.build(BookKeeper.java:307) [org.apache.bookkeeper-bookkeeper-server-4.15.0-SNAPSHOT.jar:4.15.0-SNAPSHOT]
        at org.apache.bookkeeper.replication.Auditor.createBookKeeperClient(Auditor.java:270) [org.apache.bookkeeper-bookkeeper-server-4.15.0-SNAPSHOT.jar:4.15.0-SNAPSHOT]
        at org.apache.bookkeeper.replication.AutoRecoveryMain.<init>(AutoRecoveryMain.java:92) [org.apache.bookkeeper-bookkeeper-server-4.15.0-SNAPSHOT.jar:4.15.0-SNAPSHOT]
        at org.apache.bookkeeper.server.service.AutoRecoveryService.<init>(AutoRecoveryService.java:41) [org.apache.bookkeeper-bookkeeper-server-4.15.0-SNAPSHOT.jar:4.15.0-SNAPSHOT]
        at org.apache.bookkeeper.server.Main.buildBookieServer(Main.java:320) [org.apache.bookkeeper-bookkeeper-server-4.15.0-SNAPSHOT.jar:4.15.0-SNAPSHOT]
        at org.apache.bookkeeper.server.Main.doMain(Main.java:226) [org.apache.bookkeeper-bookkeeper-server-4.15.0-SNAPSHOT.jar:4.15.0-SNAPSHOT]
        at org.apache.bookkeeper.server.Main.main(Main.java:208) [org.apache.bookkeeper-bookkeeper-server-4.15.0-SNAPSHOT.jar:4.15.0-SNAPSHOT]
17:24:04.806 [main] ERROR org.apache.bookkeeper.client.RackawareEnsemblePlacementPolicyImpl - Failed to initialize DNS Resolver org.apache.pulsar.zookeeper.ZkBookieRackAffinityMapping, used default subnet resolver : java.lang.NullPointerException null

Modifications

1、change some deprecated method
2、add a judgment.

Does this pull request potentially affect one of the following parts:

If yes was chosen, please highlight the changes

Dependencies (does it add or upgrade a dependency): (yes / no) no
The public API: (yes / no) no
The schema: (yes / no / don't know) no
The default values of configurations: (yes / no) no
The wire protocol: (yes / no)no
The rest endpoints: (yes / no)no
The admin cli options: (yes / no)no
Anything that affects deployment: (yes / no / don't know)no

Documentation

no need doc.

gaozhangmin · 2021-09-07T10:52:58Z

@hanbo1990 @BewareMyPower PTAL

nicoloboschi

I left a minor comment and a question

...r-zookeeper-utils/src/main/java/org/apache/pulsar/zookeeper/ZkBookieRackAffinityMapping.java

nicoloboschi

+1

* up/master: (37 commits) re-enabling integration tests for Sinks (apache#12307) [PIP 95][Issue 12040][web] Topic lookup with listener header (apache#12072) Fix the master CI broken with update dispatch rate block issue (apache#12360) Fix message being ignored when the non-persistent topic reader reconnect. (apache#12348) Fix log format. (apache#12346) [website][upgrade]feat: docs migration - version-2.7.2 Concepts and Architecture (apache#12354) [website][upgrade] feat: full docs migration for version 2.8.0 (apache#12359) [website][upgrade]feat: dynamic replace version info before build (apache#12337) Fix flaky tests: ElasticSearchClientTests (apache#12347) Use asyncCloseCursorLedger to replace cursorLedger.asyncClose method in the ManagedCursorImpl.VoidCallback#operationComplete (apache#12113) fix-npe-ZkBookieRackAffinityMapping (apache#11947) [pulsar-admin] Allow setting --forward-source-message-property to false when updating a pulsar function (apache#12128) [website][upgrade]feat: docs migration - Development (apache#12320) Update delete inactive topic configuration documentation (apache#12350) [PIP 95][Issue 12040][broker] Multiple bind addresses for Pulsar protocol (apache#12056) Added Debezium Source for MS SQL Server (apache#12256) Fix: flaky oracle tests (apache#12306) [C++] Use URL encoded content type for OAuth 2.0 authentication (apache#12341) [C++] Handle OAuth 2.0 exceptional cases gracefully (apache#12335) feat(cli): add restart command to pulsar-daemon (apache#12279) ... # Conflicts: # site2/website-next/docusaurus.config.js # site2/website-next/versioned_sidebars/version-2.7.2-sidebars.json # site2/website-next/versions.json

### Motivation NPE when `ZkBookieRackAffinityMapping` ``` 17:24:03.724 [main-SendThread(bigdata-pulsar-bookie004.ys:2181)] INFO org.apache.zookeeper.ClientCnxn - Session establishment complete on server bigdata-pulsar-bookie004.ys/10.89.146.53:2181, session id = 0x57ba9d6a8d800a8, negotiated timeout = 30000 17:24:03.725 [main-EventThread] INFO org.apache.bookkeeper.zookeeper.ZooKeeperWatcherBase - ZooKeeper client is connected now. 17:24:03.792 [main] ERROR org.apache.pulsar.zookeeper.ZkBookieRackAffinityMapping - Error creating zookeeper client java.lang.IllegalArgumentException: Path must not end with / character at org.apache.zookeeper.common.PathUtils.validatePath(PathUtils.java:55) ~[org.apache.zookeeper-zookeeper-3.6.3.jar:3.6.3] at org.apache.zookeeper.client.ConnectStringParser.<init>(ConnectStringParser.java:61) ~[org.apache.zookeeper-zookeeper-3.6.3.jar:3.6.3] at org.apache.zookeeper.ZooKeeper.createDefaultHostProvider(ZooKeeper.java:1515) ~[org.apache.zookeeper-zookeeper-3.6.3.jar:3.6.3] at org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:1108) ~[org.apache.zookeeper-zookeeper-3.6.3.jar:3.6.3] at org.apache.bookkeeper.zookeeper.ZooKeeperClient.<init>(ZooKeeperClient.java:286) ~[org.apache.bookkeeper-bookkeeper-server-4.15.0-SNAPSHOT.jar:4.15.0-SNAPSHOT] at org.apache.bookkeeper.zookeeper.ZooKeeperClient$Builder.build(ZooKeeperClient.java:247) ~[org.apache.bookkeeper-bookkeeper-server-4.15.0-SNAPSHOT.jar:4.15.0-SNAPSHOT] at org.apache.pulsar.zookeeper.ZkBookieRackAffinityMapping.getAndSetZkCache(ZkBookieRackAffinityMapping.java:138) [org.apache.pulsar-pulsar-zookeeper-utils-2.8.0.jar:2.8.0] at org.apache.pulsar.zookeeper.ZkBookieRackAffinityMapping.setConf(ZkBookieRackAffinityMapping.java:75) [org.apache.pulsar-pulsar-zookeeper-utils-2.8.0.jar:2.8.0] at org.apache.bookkeeper.client.RackawareEnsemblePlacementPolicyImpl.initialize(RackawareEnsemblePlacementPolicyImpl.java:264) [org.apache.bookkeeper-bookkeeper-server-4.15.0-SNAPSHOT.jar:4.15.0-SNAPSHOT] at org.apache.bookkeeper.client.RackawareEnsemblePlacementPolicyImpl.initialize(RackawareEnsemblePlacementPolicyImpl.java:80) [org.apache.bookkeeper-bookkeeper-server-4.15.0-SNAPSHOT.jar:4.15.0-SNAPSHOT] at org.apache.bookkeeper.client.BookKeeper.initializeEnsemblePlacementPolicy(BookKeeper.java:582) [org.apache.bookkeeper-bookkeeper-server-4.15.0-SNAPSHOT.jar:4.15.0-SNAPSHOT] at org.apache.bookkeeper.client.BookKeeper.<init>(BookKeeper.java:506) [org.apache.bookkeeper-bookkeeper-server-4.15.0-SNAPSHOT.jar:4.15.0-SNAPSHOT] at org.apache.bookkeeper.client.BookKeeper$Builder.build(BookKeeper.java:307) [org.apache.bookkeeper-bookkeeper-server-4.15.0-SNAPSHOT.jar:4.15.0-SNAPSHOT] at org.apache.bookkeeper.replication.Auditor.createBookKeeperClient(Auditor.java:270) [org.apache.bookkeeper-bookkeeper-server-4.15.0-SNAPSHOT.jar:4.15.0-SNAPSHOT] at org.apache.bookkeeper.replication.AutoRecoveryMain.<init>(AutoRecoveryMain.java:92) [org.apache.bookkeeper-bookkeeper-server-4.15.0-SNAPSHOT.jar:4.15.0-SNAPSHOT] at org.apache.bookkeeper.server.service.AutoRecoveryService.<init>(AutoRecoveryService.java:41) [org.apache.bookkeeper-bookkeeper-server-4.15.0-SNAPSHOT.jar:4.15.0-SNAPSHOT] at org.apache.bookkeeper.server.Main.buildBookieServer(Main.java:320) [org.apache.bookkeeper-bookkeeper-server-4.15.0-SNAPSHOT.jar:4.15.0-SNAPSHOT] at org.apache.bookkeeper.server.Main.doMain(Main.java:226) [org.apache.bookkeeper-bookkeeper-server-4.15.0-SNAPSHOT.jar:4.15.0-SNAPSHOT] at org.apache.bookkeeper.server.Main.main(Main.java:208) [org.apache.bookkeeper-bookkeeper-server-4.15.0-SNAPSHOT.jar:4.15.0-SNAPSHOT] 17:24:04.806 [main] ERROR org.apache.bookkeeper.client.RackawareEnsemblePlacementPolicyImpl - Failed to initialize DNS Resolver org.apache.pulsar.zookeeper.ZkBookieRackAffinityMapping, used default subnet resolver : java.lang.NullPointerException null ``` ### Modifications 1、change some deprecated method 2、add a judgment. Co-authored-by: gavingaozhangmin <gavingaozhangmin@didiglobal.com> (cherry picked from commit c3d540c)

### Motivation NPE when `ZkBookieRackAffinityMapping` ``` 17:24:03.724 [main-SendThread(bigdata-pulsar-bookie004.ys:2181)] INFO org.apache.zookeeper.ClientCnxn - Session establishment complete on server bigdata-pulsar-bookie004.ys/10.89.146.53:2181, session id = 0x57ba9d6a8d800a8, negotiated timeout = 30000 17:24:03.725 [main-EventThread] INFO org.apache.bookkeeper.zookeeper.ZooKeeperWatcherBase - ZooKeeper client is connected now. 17:24:03.792 [main] ERROR org.apache.pulsar.zookeeper.ZkBookieRackAffinityMapping - Error creating zookeeper client java.lang.IllegalArgumentException: Path must not end with / character at org.apache.zookeeper.common.PathUtils.validatePath(PathUtils.java:55) ~[org.apache.zookeeper-zookeeper-3.6.3.jar:3.6.3] at org.apache.zookeeper.client.ConnectStringParser.<init>(ConnectStringParser.java:61) ~[org.apache.zookeeper-zookeeper-3.6.3.jar:3.6.3] at org.apache.zookeeper.ZooKeeper.createDefaultHostProvider(ZooKeeper.java:1515) ~[org.apache.zookeeper-zookeeper-3.6.3.jar:3.6.3] at org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:1108) ~[org.apache.zookeeper-zookeeper-3.6.3.jar:3.6.3] at org.apache.bookkeeper.zookeeper.ZooKeeperClient.<init>(ZooKeeperClient.java:286) ~[org.apache.bookkeeper-bookkeeper-server-4.15.0-SNAPSHOT.jar:4.15.0-SNAPSHOT] at org.apache.bookkeeper.zookeeper.ZooKeeperClient$Builder.build(ZooKeeperClient.java:247) ~[org.apache.bookkeeper-bookkeeper-server-4.15.0-SNAPSHOT.jar:4.15.0-SNAPSHOT] at org.apache.pulsar.zookeeper.ZkBookieRackAffinityMapping.getAndSetZkCache(ZkBookieRackAffinityMapping.java:138) [org.apache.pulsar-pulsar-zookeeper-utils-2.8.0.jar:2.8.0] at org.apache.pulsar.zookeeper.ZkBookieRackAffinityMapping.setConf(ZkBookieRackAffinityMapping.java:75) [org.apache.pulsar-pulsar-zookeeper-utils-2.8.0.jar:2.8.0] at org.apache.bookkeeper.client.RackawareEnsemblePlacementPolicyImpl.initialize(RackawareEnsemblePlacementPolicyImpl.java:264) [org.apache.bookkeeper-bookkeeper-server-4.15.0-SNAPSHOT.jar:4.15.0-SNAPSHOT] at org.apache.bookkeeper.client.RackawareEnsemblePlacementPolicyImpl.initialize(RackawareEnsemblePlacementPolicyImpl.java:80) [org.apache.bookkeeper-bookkeeper-server-4.15.0-SNAPSHOT.jar:4.15.0-SNAPSHOT] at org.apache.bookkeeper.client.BookKeeper.initializeEnsemblePlacementPolicy(BookKeeper.java:582) [org.apache.bookkeeper-bookkeeper-server-4.15.0-SNAPSHOT.jar:4.15.0-SNAPSHOT] at org.apache.bookkeeper.client.BookKeeper.<init>(BookKeeper.java:506) [org.apache.bookkeeper-bookkeeper-server-4.15.0-SNAPSHOT.jar:4.15.0-SNAPSHOT] at org.apache.bookkeeper.client.BookKeeper$Builder.build(BookKeeper.java:307) [org.apache.bookkeeper-bookkeeper-server-4.15.0-SNAPSHOT.jar:4.15.0-SNAPSHOT] at org.apache.bookkeeper.replication.Auditor.createBookKeeperClient(Auditor.java:270) [org.apache.bookkeeper-bookkeeper-server-4.15.0-SNAPSHOT.jar:4.15.0-SNAPSHOT] at org.apache.bookkeeper.replication.AutoRecoveryMain.<init>(AutoRecoveryMain.java:92) [org.apache.bookkeeper-bookkeeper-server-4.15.0-SNAPSHOT.jar:4.15.0-SNAPSHOT] at org.apache.bookkeeper.server.service.AutoRecoveryService.<init>(AutoRecoveryService.java:41) [org.apache.bookkeeper-bookkeeper-server-4.15.0-SNAPSHOT.jar:4.15.0-SNAPSHOT] at org.apache.bookkeeper.server.Main.buildBookieServer(Main.java:320) [org.apache.bookkeeper-bookkeeper-server-4.15.0-SNAPSHOT.jar:4.15.0-SNAPSHOT] at org.apache.bookkeeper.server.Main.doMain(Main.java:226) [org.apache.bookkeeper-bookkeeper-server-4.15.0-SNAPSHOT.jar:4.15.0-SNAPSHOT] at org.apache.bookkeeper.server.Main.main(Main.java:208) [org.apache.bookkeeper-bookkeeper-server-4.15.0-SNAPSHOT.jar:4.15.0-SNAPSHOT] 17:24:04.806 [main] ERROR org.apache.bookkeeper.client.RackawareEnsemblePlacementPolicyImpl - Failed to initialize DNS Resolver org.apache.pulsar.zookeeper.ZkBookieRackAffinityMapping, used default subnet resolver : java.lang.NullPointerException null ``` ### Modifications 1、change some deprecated method 2、add a judgment. Co-authored-by: gavingaozhangmin <gavingaozhangmin@didiglobal.com>

Anonymitaet added the doc-not-needed Your PR changes do not impact docs label Sep 7, 2021

gaozhangmin mentioned this pull request Sep 8, 2021

fix-npe-when-pulsar-ZkBookieRackAffinityMapping-getBookieAddressResolver apache/bookkeeper#2788

Merged

nicoloboschi requested changes Sep 8, 2021

View reviewed changes

...r-zookeeper-utils/src/main/java/org/apache/pulsar/zookeeper/ZkBookieRackAffinityMapping.java Outdated Show resolved Hide resolved

...r-zookeeper-utils/src/main/java/org/apache/pulsar/zookeeper/ZkBookieRackAffinityMapping.java Show resolved Hide resolved

hanbo1990 approved these changes Sep 8, 2021

View reviewed changes

nicoloboschi approved these changes Sep 8, 2021

View reviewed changes

gavingaozhangmin added 3 commits September 24, 2021 10:18

fix-npe-ZkBookieRackAffinityMapping

a0e79f0

delete unused code

d1aaac1

change to private

8ca4dd2

gaozhangmin force-pushed the fix-npe-ZkBookieRackAffinityMapping branch from 78bea84 to 8ca4dd2 Compare September 24, 2021 02:19

eolivelli approved these changes Sep 24, 2021

View reviewed changes

eolivelli requested review from jiazhai and merlimat September 24, 2021 07:27

BewareMyPower assigned gaozhangmin Oct 14, 2021

BewareMyPower added the release/2.8.2 label Oct 14, 2021

BewareMyPower approved these changes Oct 14, 2021

View reviewed changes

BewareMyPower added this to the 2.10.0 milestone Oct 14, 2021

BewareMyPower merged commit c3d540c into apache:master Oct 14, 2021

hangc0276 added the cherry-picked/branch-2.8 Archived: 2.8 is end of life label Oct 17, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix-npe-ZkBookieRackAffinityMapping #11947

fix-npe-ZkBookieRackAffinityMapping #11947

gaozhangmin commented Sep 7, 2021 •

edited

Loading

gaozhangmin commented Sep 7, 2021

nicoloboschi left a comment

nicoloboschi left a comment

fix-npe-ZkBookieRackAffinityMapping #11947

fix-npe-ZkBookieRackAffinityMapping #11947

Conversation

gaozhangmin commented Sep 7, 2021 • edited Loading

Motivation

Modifications

Does this pull request potentially affect one of the following parts:

Documentation

gaozhangmin commented Sep 7, 2021

nicoloboschi left a comment

Choose a reason for hiding this comment

nicoloboschi left a comment

Choose a reason for hiding this comment

gaozhangmin commented Sep 7, 2021 •

edited

Loading