Skip to content

Commit

Permalink
[SPARK-33363] Add prompt information related to the current task when…
Browse files Browse the repository at this point in the history
… pyspark/sparkR starts

### What changes were proposed in this pull request?
add prompt information about current applicationId, current URL and master info when pyspark / sparkR starts.

### Why are the changes needed?
The information printed when pyspark/sparkR starts does not prompt the basic information of current application, and it is not convenient when used pyspark/sparkR in dos.

### Does this PR introduce _any_ user-facing change?
no

### How was this patch tested?
manual test result shows below:
![pyspark new print](https://user-images.githubusercontent.com/52202080/98274268-2a663f00-1fce-11eb-88ce-964ce90b439e.png)
![sparkR](https://user-images.githubusercontent.com/52202080/98541235-1a01dd00-22ca-11eb-9304-09bcde87b05e.png)

Closes #30266 from akiyamaneko/pyspark-hint-info.

Authored-by: neko <echohlne@gmail.com>
Signed-off-by: HyukjinKwon <gurwls223@apache.org>
  • Loading branch information
echohlne authored and HyukjinKwon committed Nov 10, 2020
1 parent 35ac314 commit 4360c6f
Show file tree
Hide file tree
Showing 2 changed files with 5 additions and 1 deletion.
4 changes: 3 additions & 1 deletion R/pkg/inst/profile/shell.R
Expand Up @@ -43,5 +43,7 @@
cat(" /_/", "\n")
cat("\n")

cat("\nSparkSession available as 'spark'.\n")
cat("\nSparkSession Web UI available at", SparkR::sparkR.uiWebUrl())
cat("\nSparkSession available as 'spark'(master = ", unlist(SparkR::sparkR.conf("spark.master")),
", app id = ", unlist(SparkR::sparkR.conf("spark.app.id")), ").", "\n", sep = "")
}
2 changes: 2 additions & 0 deletions python/pyspark/shell.py
Expand Up @@ -62,6 +62,8 @@
platform.python_version(),
platform.python_build()[0],
platform.python_build()[1]))
print("Spark context Web UI available at %s" % (sc.uiWebUrl))
print("Spark context available as 'sc' (master = %s, app id = %s)." % (sc.master, sc.applicationId))
print("SparkSession available as 'spark'.")

# The ./bin/pyspark script stores the old PYTHONSTARTUP value in OLD_PYTHONSTARTUP,
Expand Down

0 comments on commit 4360c6f

Please sign in to comment.