[SPARK-34125][CORE][2.4] Make EventLoggingListener.codecMap thread-safe

[cxzl25]

What changes were proposed in this pull request?

EventLoggingListener.codecMap change mutable.HashMap to ConcurrentHashMap

Why are the changes needed?

2.x version of history server
EventLoggingListener.codecMap is of type mutable.HashMap, which is not thread safe.
This will cause the history server to suddenly get stuck and not work.
The 3.x version was changed to EventLogFileReader.codecMap to ConcurrentHashMap type, so there is no such problem.(SPARK-28869)

Multiple threads call openEventLog, codecMap is updated by multiple threads, mutable.HashMap may fall into an infinite loop during resize, resulting in history server not working.
scala/bug#10436

PID 117049 0x1c939

Does this PR introduce any user-facing change?

No

How was this patch tested?

exist ut

[HeartSaVioR]

ok to test

[HeartSaVioR]

Could we please elaborate more on the impact of the issue in Why are the changes needed? via linking Scala issue as we did in JIRA issue?

[SparkQA]

Test build #134104 has finished for PR 31194 at commit 076e28e.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #134117 has started for PR 31194 at commit 2b94425.

[SparkQA]

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38700/

[SparkQA]

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38700/

[srowen]

Does this not affect master / 3.x?

[HeartSaVioR]

Yes, we changed it already in Spark 3.0, but as a part of commit and we haven't realized it should be fixed in 2.4.x as well.

[dongjoon-hyun]

+1, LGTM. Thank you, @cxzl25 and all.

[HeartSaVioR]

retest this, please

[HeartSaVioR]

Just triggered Jenkins/GA. Once one of them passes I'll merge this.

[SparkQA]

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38715/

[SparkQA]

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38715/

[SparkQA]

Test build #134132 has finished for PR 31194 at commit 2b94425.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[dongjoon-hyun]

FYI, the first commit already passed with Jenkins here. And, the second commit is an empty commit to retriever Jenkins.

• [SPARK-34125][CORE][2.4] Make EventLoggingListener.codecMap thread-safe #31194 (comment)

[SparkQA]

Test build #134140 has finished for PR 31194 at commit 5ed0ee9.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[HeartSaVioR]

Test passed. Merging.

[HeartSaVioR]

Thanks @cxzl25 for the contribution! Merged to branch-2.4.

[dongjoon-hyun]

Thank you all!