[SPARK-46733][CORE] Simplify the BlockManager by the exit operation o…

…nly depend on interrupt thread

### What changes were proposed in this pull request?
This PR propose to simplify the `BlockManager`.

### Why are the changes needed?
Currently, close or destroy `BlockManager` depend on interrupt thread and the volatile variable `stopped`.
In fact, we can change the `stopped` to a local variable on stack and let the close operation of `BlockManager` only depend on interrupt thread.
For further optimization, this PR using `running` instead of `stopped`.

### Does this PR introduce _any_ user-facing change?
'No'.

### How was this patch tested?
GA tests.

### Was this patch authored or co-authored using generative AI tooling?
'No'.

Closes #44732 from beliefer/simplify-ContextCleaner.

Authored-by: beliefer <beliefer@163.com>
Signed-off-by: Mridul Muralidharan <mridul<at>gmail.com>

core/src/main/scala/org/apache/spark/storage/BlockManager.scala

@@ -2209,7 +2209,6 @@ private[spark] object BlockManager {
       new ConcurrentHashMap)
 
     private val POLL_TIMEOUT = 1000
-    @volatile private var stopped = false
 
     private val cleaningThread = new Thread() { override def run(): Unit = { keepCleaning() } }
     cleaningThread.setDaemon(true)
@@ -2235,13 +2234,13 @@ private[spark] object BlockManager {
     }
 
     def stop(): Unit = {
-      stopped = true
       cleaningThread.interrupt()
       cleaningThread.join()
     }
 
     private def keepCleaning(): Unit = {
-      while (!stopped) {
+      var running = true
+      while (running) {
         try {
           Option(referenceQueue.remove(POLL_TIMEOUT))
             .map(_.asInstanceOf[ReferenceWithCleanup])
@@ -2251,7 +2250,7 @@ private[spark] object BlockManager {
             }
         } catch {
           case _: InterruptedException =>
-            // no-op
+            running = false
           case NonFatal(e) =>
             logError("Error in cleaning thread", e)
         }