Skip to content

Commit

Permalink
[LIVY-418] Respect spark.pyspark.python for PythonInterpreter
Browse files Browse the repository at this point in the history
## What changes were proposed in this pull request?

`spark.pyspark.python` is introduced in spark 2.1, livy also need to update its code for choosing python version

## How was this patch tested?

Manually verified.

Author: Jeff Zhang <zjffdu@gmail.com>

Closes #65 from zjffdu/LIVY-418.
  • Loading branch information
zjffdu authored and jerryshao committed Nov 23, 2017
1 parent 5471544 commit 64d71aa
Showing 1 changed file with 2 additions and 1 deletion.
Expand Up @@ -44,7 +44,8 @@ import org.apache.livy.sessions._
object PythonInterpreter extends Logging {

def apply(conf: SparkConf, sparkEntries: SparkEntries): Interpreter = {
val pythonExec = sys.env.get("PYSPARK_PYTHON")
val pythonExec = conf.getOption("spark.pyspark.python")
.orElse(sys.env.get("PYSPARK_PYTHON"))
.orElse(sys.props.get("pyspark.python")) // This java property is only used for internal UT.
.getOrElse("python")

Expand Down

0 comments on commit 64d71aa

Please sign in to comment.