Skip to content

Commit

Permalink
[SPARK-48946][SQL] NPE in redact method when session is null
Browse files Browse the repository at this point in the history
### What changes were proposed in this pull request?

If we call DataSourceV2ScanExecBase redact method from a thread that don't have a session in thread local we get an NPE. Getting stringRedactionPattern from conf could prevent this problem as conf checks if session is null or not. We also use this in DataSourceScanExec trait.
https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/execution/DataSourceScanExec.scala#L93-L95

### Why are the changes needed?

To prevent NPE when session is null.

### Does this PR introduce _any_ user-facing change?

No

### How was this patch tested?

Existing UTs.

### Was this patch authored or co-authored using generative AI tooling?

No

Closes apache#47419 from mikoszilard/SPARK-48946.

Authored-by: Szilard Miko <smiko@cloudera.com>
Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
  • Loading branch information
mikoszilard authored and jingz-db committed Jul 22, 2024
1 parent cfe2526 commit e7a1105
Showing 1 changed file with 1 addition and 1 deletion.
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@ trait DataSourceV2ScanExecBase extends LeafExecNode {
* Shorthand for calling redact() without specifying redacting rules
*/
protected def redact(text: String): String = {
Utils.redact(session.sessionState.conf.stringRedactionPattern, text)
Utils.redact(conf.stringRedactionPattern, text)
}

override def verboseStringWithOperatorId(): String = {
Expand Down

0 comments on commit e7a1105

Please sign in to comment.