Use updaters to update read messages to ZK

[mgao0]

Issues

• My PR addresses the following Helix issues and references them in the PR description:

#1001

Description

• Here are some details about my PR, including screenshots of any UI changes:

This PR uses an updater to replace set, so it checks whether there is a message in the path beforehand, and only write the "READ" state message to ZK if there is a message, to avoid write back a message which is already removed.

Tests

• The following tests are written for this issue:

TestHelixTaskExecutor -> testNoWriteReadStateForRemovedMessage

• The following is the result of the "mvn test" command on the appropriate module:

[INFO]
[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR] TestWorkflowTermination.testWorkflowRunningTimeout:131->verifyWorkflowCleanup:257 expected: but was:
[INFO]
[ERROR] Tests run: 1147, Failures: 1, Errors: 0, Skipped: 0
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 01:15 h
[INFO] Finished at: 2020-05-12T18:59:45-07:00

Commits

• My commits all reference appropriate Apache Helix GitHub issues in their subject lines. In addition, my commits follow the guidelines from "How to write a good git commit message":
  1. Subject is separated from body by a blank line
  2. Subject is limited to 50 characters (not including Jira issue reference)
  3. Subject does not end with a period
  4. Subject uses the imperative mood ("add", not "adding")
  5. Body wraps at 72 characters
  6. Body explains "what" and "why", not "how"

Documentation (Optional)

• In case of new functionality, my PR adds documentation in the following wiki page:

(Link the GitHub wiki you added)

Code Quality

• My diff has been formatted using helix-style.xml
  (helix-style-intellij.xml if IntelliJ IDE is used)

[jiajunwang]

I think the design is a little bit awkward. Please check out my comments and see if that would be an easier design.

[jiajunwang]

One minor thing, if the unexpected override happens, shall we log a warning?

After all, this PR is to fix the symptom not fix the root cause. Let's ensure that we still have the capability to debug the root cause.

[mgao0]

I think the design is a little bit awkward. Please check out my comments and see if that would be an easier design.

I moved all your comments on periodic message refresh to #1000. I decided to separate the PR since it's trying to resolve two different issues. Sorry for the confusion, please refer to the link above for the replies on your comments.

[mgao0]

@jiajunwang I have added the logging. Can you please check if the message is good?

[mgao0]

This PR is ready to be merged, approved by @dasahcc @jiajunwang .
Final commit message:
Use updaters to update messages with READ state to ZK

[lei-xia]

One optimization here maybe to create just one updater instead of one per each message. You can define the update logic in the updater, i.e, read the message and mark the read field as "read".