[SPARK-5219][Core] Add locks to avoid scheduling race conditions

[zsxwing]

No description provided.

[SparkQA]

Test build #25471 has started for PR 4019 at commit 36a8b4e.

• This patch merges cleanly.

[SparkQA]

Test build #25471 has finished for PR 4019 at commit 36a8b4e.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[AmplabJenkins]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/25471/
Test PASSed.

[andrewor14]

Hey @zsxwing can you update the PR title to be more descriptive? Maybe just adding "avoid scheduling race conditions" is sufficient. Also, is there a particular behavior that you observed that is caused by this race condition? Do you have a reproduction of it?

[zsxwing]

I found it when I dig into codes for another irrelevant failure. Just analyzed the codes and didn't reproduce it.

[andrewor14]

You mentioned there are shared variables that these methods use. Which ones in particular are you referring to?

[zsxwing]

These methods are called in threads of TaskResultGetter.getTaskResultExecutor. And they access variables such as isZombie, taskInfos in TaskSetManager, which also are used in TaskSchedulerImpl.

[rxin]

Thanks. I'm merging this into master.