Skip to content

Conversation

@xyuanlu
Copy link
Contributor

@xyuanlu xyuanlu commented Nov 2, 2020

Issues

Description

  • Here are some details about my PR, including screenshots of any UI changes:

During code inspection, We found there are multiple subscribeForChanges being called in CallBackHandler.handelChildChange. This leads to longer time spend when process callbacks in zkClient, witch eventually leads to increased PendingCallback queue size.
This PR removes duplicate subscribeForChanges in CallBackHandler to improve performance.

Tests

  • The following tests are written for this issue:

NA

  • The following is the result of the "mvn test" command on the appropriate module:
[INFO] Tests run: 1244, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5,043.226 s - in TestSuite
[INFO] 
[INFO] Results:
[INFO] 
[INFO] Tests run: 1244, Failures: 0, Errors: 0, Skipped: 0
[INFO] 
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  01:24 h
[INFO] Finished at: 2020-11-04T14:47:44-08:00
[INFO] ------------------------------------------------------------------------

Documentation (Optional)

  • In case of new functionality, my PR adds documentation in the following wiki page:

(Link the GitHub wiki you added)

Commits

  • My commits all reference appropriate Apache Helix GitHub issues in their subject lines. In addition, my commits follow the guidelines from "How to write a good git commit message":
    1. Subject is separated from body by a blank line
    2. Subject is limited to 50 characters (not including Jira issue reference)
    3. Subject does not end with a period
    4. Subject uses the imperative mood ("add", not "adding")
    5. Body wraps at 72 characters
    6. Body explains "what" and "why", not "how"

Code Quality

  • My diff has been formatted using helix-style.xml
    (helix-style-intellij.xml if IntelliJ IDE is used)

Copy link
Contributor

@junkaixue junkaixue left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add an unit test? For example, we invoke the with different changes, we can make sure all changes we subscribe back?

subscribeForChanges(changeContext.getType(), _path, _watchChild);
} else {
// put SubscribeForChange run in async thread to reduce the latency of zk callback handling.
subscribeForChangesAsyn(changeContext.getType(), _path, _watchChild);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did we try to get rid of async subscribe in this PR and do all subscription synchronously? I thought we only remove the extra one.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TFTR. The duplicated subscribeForChanges in HandleChildChange and do resubscribe for child change in line 361. For data changes, this async subscribe is also duplicated since the path is already resubscribed in zkClient.

@xyuanlu
Copy link
Contributor Author

xyuanlu commented Nov 2, 2020

Can we add an unit test? For example, we invoke the with different changes, we can make sure all changes we subscribe back?

TFTR. Let me try.

@xyuanlu xyuanlu marked this pull request as draft November 4, 2020 01:01
@xyuanlu xyuanlu marked this pull request as ready for review November 4, 2020 05:04
@xyuanlu xyuanlu force-pushed the callBackHandler branch 2 times, most recently from e3185ac to c840da8 Compare November 4, 2020 21:22
@xyuanlu xyuanlu marked this pull request as draft November 4, 2020 23:24
@xyuanlu xyuanlu force-pushed the callBackHandler branch 2 times, most recently from e420e61 to c2f5eba Compare November 4, 2020 23:33
@xyuanlu xyuanlu marked this pull request as ready for review November 5, 2020 01:02
}

// verify new watcher is installed on the new node
Thread.sleep(5000);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we try to get rid of this kind of thread sleep, and use verifier. But as this is legacy test, I think it's fine. You can let @kaisun2000 know.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure if we have watcher or listener installation verifier. If not, may be larger change is needed.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for offline discuss. Updated.

Copy link
Contributor

@junkaixue junkaixue left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I dont have any concerns. But just make sure resolved other comments.

@xyuanlu
Copy link
Contributor Author

xyuanlu commented Nov 9, 2020

This PR is ready to be merged. Approved by @dasahcc

Commit message:
Remove duplicate subscribe in CallBackHandler.handleChildChange()

Duplicate subscribes lead to longer time spend when process callbacks in zkClient, witch eventually leads to increased PendingCallback queue size. This PR removes duplicate subscribeForChanges in CallBackHandler to improve performance.

@jiajunwang jiajunwang merged commit 11fa63a into apache:master Nov 9, 2020
@xyuanlu xyuanlu deleted the callBackHandler branch November 9, 2020 22:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants