Skip to content

Commit

Permalink
[SPARK-46498][CORE] Remove shuffleServiceEnabled from `o.a.spark.ut…
Browse files Browse the repository at this point in the history
…il.Utils#getConfiguredLocalDirs`

### What changes were proposed in this pull request?
This PR cleans up an unused local variables from `org.apache.spark.util.Utils#getConfiguredLocalDirs` function, It is no longer used after the support for Mesos was removed in SPARK-44442 (#43135).

### Why are the changes needed?
Code clean up

### Does this PR introduce _any_ user-facing change?
No

### How was this patch tested?
Pass GitHub Actions

### Was this patch authored or co-authored using generative AI tooling?
No

Closes #44475 from LuciferYang/utils-useless.

Lead-authored-by: yangjie01 <yangjie01@baidu.com>
Co-authored-by: YangJie <yangjie01@baidu.com>
Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
  • Loading branch information
LuciferYang authored and dongjoon-hyun committed Dec 26, 2023
1 parent c0ff0f5 commit 8ea09de
Showing 1 changed file with 1 addition and 2 deletions.
3 changes: 1 addition & 2 deletions core/src/main/scala/org/apache/spark/util/Utils.scala
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ import org.slf4j.Logger

import org.apache.spark._
import org.apache.spark.deploy.SparkHadoopUtil
import org.apache.spark.internal.{config, Logging}
import org.apache.spark.internal.Logging
import org.apache.spark.internal.config._
import org.apache.spark.internal.config.Streaming._
import org.apache.spark.internal.config.Tests.IS_TESTING
Expand Down Expand Up @@ -769,7 +769,6 @@ private[spark] object Utils
* logic of locating the local directories according to deployment mode.
*/
def getConfiguredLocalDirs(conf: SparkConf): Array[String] = {
val shuffleServiceEnabled = conf.get(config.SHUFFLE_SERVICE_ENABLED)
if (isRunningInYarnContainer(conf)) {
// If we are in yarn mode, systems can have different disk layouts so we must set it
// to what Yarn on this system said was available. Note this assumes that Yarn has
Expand Down

0 comments on commit 8ea09de

Please sign in to comment.