[REEF-726] Race condition with completed Containers

[afchung]

This addressed the issue by

• Not releasing Evaluators directly in YarnContainerManager and instead delegate the job to EvaluatorManager on container complete.

JIRA:
REEF-726

[afchung]

Tested on HDInsight.

[markusweimer]

@afchung Did you run the whole Java test suite as well?

[afchung]

@markusweimer Yes. Please have a look. Thanks!

[markusweimer]

@afchung Awesome! I'll do a pass.

[asfbot]

REEF-pull-request-windows3 #406 SUCCESS
This pull request looks good

[markusweimer]

This assumes that the release message from YARN is the last message, correct? What if we get the YARN message first and then the final heartbeat? I believe that is what EvaluatorManager.isResourceReleased was introduced for.

[afchung]

@markusweimer No, this works in either case because both end up calling close(), and whoever wins the race will be the one to release the Evaluator.

[afchung]

In other words, if we get the final heartbeat later, we will just ignore the message because this.stateManager.isDoneOrFailedOrKilled() would have been set to true. onResourceStatusMessage() and onEvaluatorHeartbeatMessage() are both synchronized on this.evaluatorDescriptor.

[asfbot]

Reef-pull-request-ubuntu #505 SUCCESS
This pull request looks good

[afchung]

@markusweimer Are there any concerns that I should address here?

[markusweimer]

@afchung Not really, I will do another pass now.