Apache Pinot facing issues with Helix version 1.4.0

[abhioncbr]

Describe the bug

Apache Pinot is using Helix for various cluster management tasks. I recently tried to upgrade the Helix dependency from 1.3.1 to 1.4.0; however, we are seeing a lot of test failures and startup failures with the newer version of Helix.

Based on the investigation, I pin down the issues because of the change in this PR.

I am trying to understand the change introduced in the PR mentioned above and workaround for the issue. For reference(Apache Pinot Helix upgrade PR)

cc: @jackjlli / @Jackie-Jiang

[junkaixue]

@abhioncbr did Pinot use WAGED rebalance? This WAGED related changes.

Original behavior is when new resource added without partition can cause NPE and break the Helix pipeline. This PR fix this behavior.

adding the author : @himanshukandwal