Skip to content

Commit

Permalink
Update getLocalDir() documentation.
Browse files Browse the repository at this point in the history
  • Loading branch information
JoshRosen committed Aug 18, 2014
1 parent bb3ad89 commit 399ac25
Show file tree
Hide file tree
Showing 2 changed files with 11 additions and 5 deletions.
14 changes: 10 additions & 4 deletions core/src/main/scala/org/apache/spark/util/Utils.scala
Original file line number Diff line number Diff line change
Expand Up @@ -449,10 +449,16 @@ private[spark] object Utils extends Logging {
}

/**
* Get a temporary directory using Spark's spark.local.dir property, if set. This will always
* return a single directory, even though the spark.local.dir property might be a list of
* multiple paths. If the SPARK_LOCAL_DIRS environment variable is set, then this will return
* a directory from that variable.
* Get the path of temporary directory. Spark's local directories can be configured through
* multiple settings, which are used with the following precedence:
*
* - If called from inside of a YARN container, this will return a directory chosen by YARN.
* - If the SPARK_LOCAL_DIRS environment variable is set, this will return a directory from it.
* - Otherwise, if the spark.local.dir is set, this will return a directory from it.
* - Otherwise, this will return java.io.tmpdir.
*
* Some of these configuration options might be lists of multiple paths, but this method will
* always return a single directory.
*/
def getLocalDir(conf: SparkConf): String = {
getOrCreateLocalRootDirs(conf)(0)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ import org.apache.spark.SparkConf


/**
* Tests for the spark.local.dirs and SPARK_LOCAL_DIRS configuration options.
* Tests for the spark.local.dir and SPARK_LOCAL_DIRS configuration options.
*/
class LocalDirsSuite extends FunSuite {

Expand Down

0 comments on commit 399ac25

Please sign in to comment.