Skip to content

Add an option for using dedicated ZkConnection in ZkBaseDataAccessor#655

Merged
narendly merged 3 commits intoapache:masterfrom
narendly:baseAccessor
Dec 13, 2019
Merged

Add an option for using dedicated ZkConnection in ZkBaseDataAccessor#655
narendly merged 3 commits intoapache:masterfrom
narendly:baseAccessor

Conversation

@narendly
Copy link
Contributor

@narendly narendly commented Dec 12, 2019

Issues

  • My PR addresses the following Helix issues and references them in the PR description:

Resolves #654

Description

  • Here are some details about my PR, including screenshots of any UI changes:

There are usage patterns where a user needs to create ephemeral nodes using a ZkBaseDataAccessor. We need to support this use case since we want users to avoid using ZkClient directly, so we do it in ZkBaseDataAccessor, which is Helix's wrapper API around the raw ZkClient.

The default behavior would be to use a shared ZK connection resource.

Tests

  • The following tests are written for this issue:

Tests for Dedicated and Shared ZkClients already exist in the repository.

  • The following is the result of the "mvn test" command on the appropriate module:

[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR] TestCardDealingAdjustmentAlgorithmV2.testAlgorithmConstructor:127 expected:<9> but was:<6>
[ERROR] org.apache.helix.controller.strategy.crushMapping.TestCardDealingAdjustmentAlgorithmV2.testComputeMappingForDifferentReplicas(org.apache.helix.controller.strategy.crushMapping.TestCardDealingAdjustmentAlgorithmV2)
[ERROR] Run 1: TestCardDealingAdjustmentAlgorithmV2.testComputeMappingForDifferentReplicas:273 Total movements: 4 != expected 8, replica: 1
[ERROR] Run 2: TestCardDealingAdjustmentAlgorithmV2.testComputeMappingForDifferentReplicas:273 Total movements: 4 != expected 8, replica: 2
[ERROR] Run 3: TestCardDealingAdjustmentAlgorithmV2.testComputeMappingForDifferentReplicas:273 Total movements: 14 != expected 21, replica: 3
[INFO] Run 4: PASS
[INFO] Run 5: PASS
[INFO]
[ERROR] TestRebalanceRunningTask.testFixedTargetTaskAndEnabledRebalanceAndNodeAdded:298 expected: but was:
[INFO]
[ERROR] Tests run: 883, Failures: 3, Errors: 0, Skipped: 0
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE


[INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 19.178 s - in org.apache.helix.integration.task.TestRebalanceRunningTask
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 23.770 s
[INFO] Finished at: 2019-12-12T08:56:34-08:00
[INFO] ------------------------------------------------------------------------

[INFO] Tests run: 27, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.469 s - in org.apache.helix.controller.strategy.crushMapping.TestCardDealingAdjustmentAlgorithmV2
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 27, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 5.009 s
[INFO] Finished at: 2019-12-12T08:58:18-08:00
[INFO] ------------------------------------------------------------------------

Commits

  • My commits all reference appropriate Apache Helix GitHub issues in their subject lines, and I have squashed multiple commits if they address the same issue. In addition, my commits follow the guidelines from "How to write a good git commit message":
    1. Subject is separated from body by a blank line
    2. Subject is limited to 50 characters (not including Jira issue reference)
    3. Subject does not end with a period
    4. Subject uses the imperative mood ("add", not "adding")
    5. Body wraps at 72 characters
    6. Body explains "what" and "why", not "how"

Code Quality

  • My diff has been formatted using helix-style.xml

@narendly narendly changed the title Add a flag for using dedicated ZkConnection in ZkBaseDataAccessor Add an option for using dedicated ZkConnection in ZkBaseDataAccessor Dec 12, 2019
@jiajunwang
Copy link
Contributor

I revisited the design doc about our ongoing ZK scalability project, it seems that we will migrate this accessor layer to be "zk realm aware". In that case, the raw ZkAddress will mostly be upgraded to a realm address, right? By then, I'm not sure if we still need the type to differentiate or the realm information would be rich enough for us to determine?

@narendly
Copy link
Contributor Author

This PR is ready to be merged, approved by @jiajunwang

@narendly
Copy link
Contributor Author

There are usage patterns where a user needs to create ephemeral nodes using a ZkBaseDataAccessor. We need to support this use case since we want users to avoid using ZkClient directly, so we do it in ZkBaseDataAccessor, which is Helix's wrapper API around the raw ZkClient.

The default behavior would be to use a shared ZK connection resource.

@narendly narendly merged commit 2416922 into apache:master Dec 13, 2019
narendly added a commit to narendly/helix that referenced this pull request Dec 24, 2019
…pache#655)

There are usage patterns where a user needs to create ephemeral nodes using a ZkBaseDataAccessor. We need to support this use case since we want users to avoid using ZkClient directly, so we do it in ZkBaseDataAccessor, which is Helix's wrapper API around the raw ZkClient.

The default behavior would be to use a shared ZK connection resource.
zhangmeng916 pushed a commit to zhangmeng916/helix that referenced this pull request Feb 7, 2020
…pache#655)

There are usage patterns where a user needs to create ephemeral nodes using a ZkBaseDataAccessor. We need to support this use case since we want users to avoid using ZkClient directly, so we do it in ZkBaseDataAccessor, which is Helix's wrapper API around the raw ZkClient.

The default behavior would be to use a shared ZK connection resource.
mgao0 pushed a commit to mgao0/helix that referenced this pull request Mar 6, 2020
…pache#655)

There are usage patterns where a user needs to create ephemeral nodes using a ZkBaseDataAccessor. We need to support this use case since we want users to avoid using ZkClient directly, so we do it in ZkBaseDataAccessor, which is Helix's wrapper API around the raw ZkClient.

The default behavior would be to use a shared ZK connection resource.
huizhilu pushed a commit to huizhilu/helix that referenced this pull request Aug 16, 2020
…pache#655)

There are usage patterns where a user needs to create ephemeral nodes using a ZkBaseDataAccessor. We need to support this use case since we want users to avoid using ZkClient directly, so we do it in ZkBaseDataAccessor, which is Helix's wrapper API around the raw ZkClient.

The default behavior would be to use a shared ZK connection resource.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add a flag for using dedicated ZkConnection in ZkBaseDataAccessor

4 participants