Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[SPARK-26625] Add oauthToken to spark.redaction.regex #23555

Closed
wants to merge 2 commits into from
Closed

[SPARK-26625] Add oauthToken to spark.redaction.regex #23555

wants to merge 2 commits into from

Conversation

vinooganesh
Copy link
Contributor

What changes were proposed in this pull request?

The regex (spark.redaction.regex) that is used to decide which config properties or environment settings are sensitive should also include oauthToken to match spark.kubernetes.authenticate.submission.oauthToken

How was this patch tested?

Simple regex addition - happy to add a test if needed.

@mccheah
Copy link
Contributor

mccheah commented Jan 15, 2019

ok to test

mccheah
mccheah reviewed Jan 15, 2019
View reviewed changes
core/src/main/scala/org/apache/spark/internal/config/package.scala Outdated
@@ -435,7 +435,7 @@ package object config {
"a property key or value, the value is redacted from the environment UI and various logs " +
"like YARN and event logs.")
.regexConf
.createWithDefault("(?i)secret|password".r)
.createWithDefault("(?i)secret|password|oauthToken".r)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we just have token instead of oauthToken?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yep, will update

@SparkQA
Copy link

SparkQA commented Jan 15, 2019

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/7116/

@SparkQA
Copy link

SparkQA commented Jan 15, 2019

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/7116/

@SparkQA
Copy link

SparkQA commented Jan 15, 2019

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/7118/

@SparkQA
Copy link

SparkQA commented Jan 15, 2019

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/7118/

@mccheah
Copy link
Contributor

mccheah commented Jan 15, 2019

Also think the PR title should reference the fact we're adding this to the redaction regex.

cc @vanzin

@SparkQA
Copy link

SparkQA commented Jan 15, 2019

Test build #101274 has finished for PR 23555 at commit e52a9b8.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@vinooganesh vinooganesh changed the title [SPARK-26625] Add oauthToken to spark.kubernetes.authenticate.submission.oauthToken [SPARK-26625] Add oauthToken to spark.redaction.regex Jan 15, 2019
@vinooganesh
Copy link
Contributor Author

Updated the title

@SparkQA
Copy link

SparkQA commented Jan 16, 2019

Test build #101279 has finished for PR 23555 at commit d8bb15b.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@mccheah
Copy link
Contributor

mccheah commented Jan 16, 2019

+1 from me, will leave for a little bit of time but then merge to master.

@mccheah
Copy link
Contributor

mccheah commented Jan 16, 2019

Thanks! merging to master.

@asfgit asfgit closed this in 01301d0 Jan 16, 2019
@robert3005 robert3005 deleted the vinooganesh/SPARK-26625 branch January 28, 2019 10:51
jackylee-ch pushed a commit to jackylee-ch/spark that referenced this pull request Feb 18, 2019
@jackylee-ch
[SPARK-26625] Add oauthToken to spark.redaction.regex
ad0a792 
## What changes were proposed in this pull request?

The regex (spark.redaction.regex) that is used to decide which config properties or environment settings are sensitive should also include oauthToken to match  spark.kubernetes.authenticate.submission.oauthToken

## How was this patch tested?

Simple regex addition - happy to add a test if needed.

Author: Vinoo Ganesh <vganesh@palantir.com>

Closes apache#23555 from vinooganesh/vinooganesh/SPARK-26625.
@dongjoon-hyun
Copy link
Member

Hi, All. I'll backport this to branch-2.4 since this is a security patch.
cc @viirya since he is the release manager of Apache Spark 2.4.8.

dongjoon-hyun pushed a commit that referenced this pull request Mar 21, 2021
@dongjoon-hyun
[SPARK-26625] Add oauthToken to spark.redaction.regex
7879a0c 
## What changes were proposed in this pull request?

The regex (spark.redaction.regex) that is used to decide which config properties or environment settings are sensitive should also include oauthToken to match  spark.kubernetes.authenticate.submission.oauthToken

## How was this patch tested?

Simple regex addition - happy to add a test if needed.

Author: Vinoo Ganesh <vganesh@palantir.com>

Closes #23555 from vinooganesh/vinooganesh/SPARK-26625.

(cherry picked from commit 01301d0)
Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
@dongjoon-hyun dongjoon-hyun mentioned this pull request Mar 21, 2021
Closed
@viirya
Copy link
Member

viirya commented Mar 21, 2021

@dongjoon-hyun Thanks. Looks good to me.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@srowen srowen srowen approved these changes

@mccheah mccheah mccheah approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
6 participants
@vinooganesh @mccheah @SparkQA @dongjoon-hyun @viirya @srowen