[SPARK-38713][SQL] Change spark.sessionstate.conf.getConf/setConf operation to spark.conf.get/set#35950
Conversation
|
I don't know this part well, but it seems reasonable to unify the checks here |
sql/core/src/main/scala/org/apache/spark/sql/RuntimeConfig.scala
Outdated
Show resolved
Hide resolved
HyukjinKwon
changed the title
|
Can one of the admins verify this patch? |
|
@cloud-fan @HyukjinKwon Any more question about this pr? |
sql/core/src/main/scala/org/apache/spark/sql/RuntimeConfig.scala
Outdated
Show resolved
Hide resolved
…ation_to_spark_conf
|
thanks, merging to master! |
|
@jackylee-ch seems you link to the wrong JIRA ticket? |
This is a minor change, thus I didn't create a JIRA for it. |
|
At least we shouldn't link to an unrelated JIRA... Please create a new one for this PR as it's not a minor one-line change. |
jackylee-ch
deleted the
change_spark_sessionstate_conf_operation_to_spark_conf
branch
Sorry, I will create one inmediatly. |
|
https://issues.apache.org/jira/browse/SPARK-38713 |
cloud-fan
changed the title
4 participants
Why are the changes needed?
In the sql module, we provide
SparkSession.confas a unified entry forSQLConf.set/get, which can prevent users or logic from modifying StaticSQLConf and Spark configs. However, I foundSparkSession.sessionstate.confis used in some code to getConf or setConf, which can skip the check ofRuntimeConfig.In this PR, we want to unify the behavior of
SQLConf.getConf/setConftoSparkSession.conf.Does this PR introduce any user-facing change?
No.
How was this patch tested?
Origin GA