Please sign in to comment.
Make Spark's Hadoop token file available to Python method (#1532)
Python code run via horovod.spark.run() needs the Hadoop token file to access secure HDFS. This is made available to the Spark executor via environment variable HADOOP_TOKEN_FILE_LOCATION. SparkTaskService needs to pass this to orted, which runs the Python code. Signed-off-by: Enrico Minack <firstname.lastname@example.org>
- Loading branch information
Showing with 66 additions and 2 deletions.