Skip to content

Commit

Permalink
[SPARK-20120][SQL][FOLLOW-UP] Better way to support spark-sql silent …
Browse files Browse the repository at this point in the history
…mode.

## What changes were proposed in this pull request?

`spark-sql` silent mode will broken if`SPARK_HOME/jars` missing `kubernetes-model-2.0.0.jar`.
This pr use `sc.setLogLevel (<logLevel>)` to implement silent mode.

## How was this patch tested?

manual tests

```
build/sbt -Phive -Phive-thriftserver package
export SPARK_PREPEND_CLASSES=true
./bin/spark-sql -S
```

Author: Yuming Wang <yumwang@ebay.com>

Closes #20274 from wangyum/SPARK-20120-FOLLOW-UP.
  • Loading branch information
wangyum authored and srowen committed May 22, 2018
1 parent d3d1807 commit fc743f7
Showing 1 changed file with 4 additions and 5 deletions.
Expand Up @@ -35,7 +35,7 @@ import org.apache.hadoop.hive.ql.exec.Utilities
import org.apache.hadoop.hive.ql.processors._
import org.apache.hadoop.hive.ql.session.SessionState
import org.apache.hadoop.security.{Credentials, UserGroupInformation}
import org.apache.log4j.{Level, Logger}
import org.apache.log4j.Level
import org.apache.thrift.transport.TSocket

import org.apache.spark.SparkConf
Expand Down Expand Up @@ -300,10 +300,6 @@ private[hive] class SparkSQLCLIDriver extends CliDriver with Logging {

private val console = new SessionState.LogHelper(LOG)

if (sessionState.getIsSilent) {
Logger.getRootLogger.setLevel(Level.WARN)
}

private val isRemoteMode = {
SparkSQLCLIDriver.isRemoteMode(sessionState)
}
Expand All @@ -315,6 +311,9 @@ private[hive] class SparkSQLCLIDriver extends CliDriver with Logging {
// because the Hive unit tests do not go through the main() code path.
if (!isRemoteMode) {
SparkSQLEnv.init()
if (sessionState.getIsSilent) {
SparkSQLEnv.sparkContext.setLogLevel(Level.WARN.toString)
}
} else {
// Hive 1.2 + not supported in CLI
throw new RuntimeException("Remote operations not supported")
Expand Down

0 comments on commit fc743f7

Please sign in to comment.