[SPARK-46733][CORE] Simplify the BlockManager by the exit operation only depend on interrupt thread.

[beliefer]

What changes were proposed in this pull request?

This PR propose to simplify the BlockManager.

Why are the changes needed?

Currently, close or destroy BlockManager depend on interrupt thread and the volatile variable stopped.
In fact, we can change the stopped to a local variable on stack and let the close operation of BlockManager only depend on interrupt thread.
For further optimization, this PR using running instead of stopped.

Does this PR introduce any user-facing change?

'No'.

How was this patch tested?

GA tests.

Was this patch authored or co-authored using generative AI tooling?

'No'.

[mridulm]

Unlike ReloadingX509TrustManager , here the InterruptedException is caught and ignored in some of the downstream methods: so we cant rely on that pattern here.

[beliefer]

Unlike ReloadingX509TrustManager , here the InterruptedException is caught and ignored in some of the downstream methods: so we cant rely on that pattern here.

I reverted the ContextCleaner.

[mridulm]

Looks reasonable to me.
+CC @dongjoon-hyun as well who reviewed the prev pr.

[beliefer]

@dongjoon-hyun @srowen @LuciferYang

[LuciferYang]

This change of this pr is fine to me.

However, I would like to raise a question: Would it be simpler and clearer to implement the logic for cleaning up the DownloadFile resource using the java.lang.ref.Cleaner mechanism instead of the current approach?

[beliefer]

Java.lang.ref.Cleaner cannot respond to interrupts.

[beliefer]

cc @mridulm
Could you take a review again?

[mridulm]

Since you looked into this - do you have any other thoughts on this @LuciferYang ? Thanks

[LuciferYang]

no more, the change is fine to me

[mridulm]

LGTM

[mridulm]

Merged to master.
Thanks for working on this @beliefer !
Thanks for the review @LuciferYang :-)

[beliefer]

@mridulm @LuciferYang Thank you!