[SPARK-9886][CORE] Fix to use ShutdownHookManager in

[naveenminchu]

ExternalBlockStore.scala

[srowen]

LGTM, but not sure how we missed this one. It makes me wonder if there's a particular reason for it. But I don't see a reason the current implementation would guarantee some behavior that's required.

[JoshRosen]

Hey, can we add a linter rule to ban uses of "Runtime.getRuntime.addShutdownHook", similar to our existing rule for "Class.forName", which recommends using our own utility function instead?

[naveenminchu]

Hi @JoshRosen, Are you suggesting to go ahead look over "Runtime.getRuntime.addShutdownHook" use and get rid of same

[JoshRosen]

@naveenminchu, yeah, I'm suggesting that we add a rule to scalastyle-config.xml (like

spark/scalastyle-config.xml

Line 160 in bc1ff9f

<check customId="classforname" level="error" class="org.scalastyle.file.RegexChecker" enabled="true">

), add excludes to ignore the legitimate uses, then fix any violations that we find (see #7350 for an example of this)

I suppose the Scalastyle lint doesn't catch similar problems in Java code, but I say "let's not let perfect be the enemy of the good" here; it's better to have some checks vs. no checks.

[srowen]

The comment from @vanzin on the JIRA indicated he wasn't sure whether the other 2 usages were better left alone as they're used in particular stand-alone processes. But I suppose it doesn't really hurt to use the fancy wrapper in those cases too.

[SparkQA]

Test build #2218 has finished for PR 10313 at commit 2672ddb.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[andrewor14]

Merging into master 1.6. We can add the scalastyle check later.

[tedyu]

There're still 3 references to Runtime.getRuntime.addShutdownHook() in the code base.
Should they be replaced with ShutdownHookManager ?

[andrewor14]

I believe so, and we can also add a scalastyle check that Josh suggested

[tedyu]

Created #10325

[naveenminchu]

@tedyu Thanks for making those changes