Skip to content

YARN-10814. Fallback to RandomSecretProvider if the secret file is empty #3206

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
shuzirra merged 6 commits into from
Jul 30, 2021
Merged

YARN-10814. Fallback to RandomSecretProvider if the secret file is empty #3206

shuzirra merged 6 commits into from
Jul 30, 2021

Conversation

@tomicooler
Copy link
Contributor

@tomicooler tomicooler commented Jul 15, 2021

…pty.

The rest endpoint would be unusable with an empty secret file
(throwing IllegalArgumentExceptions).

Any IO error would have resulted in the same fallback path.

Change-Id: Ieb147a0f6f8f628cdafb3c987d0c9a440fa8c6d0

NOTICE

Please create an issue in ASF JIRA before opening a pull request,
and you need to set the title of the pull request which starts with
the corresponding JIRA issue number. (e.g. HADOOP-XXXXX. Fix a typo in YYY.)
For more details, please see https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute

@tomicooler
YARN-10814. Fallback to RandomSecretProvider if the secret file is em…
7b5b2cb 
…pty.

The rest endpoint would be unusable with an empty secret file
(throwing IllegalArgumentExceptions).

Any IO error would have resulted in the same fallback path.

Change-Id: Ieb147a0f6f8f628cdafb3c987d0c9a440fa8c6d0
@hadoop-yetus
Copy link

hadoop-yetus commented Jul 15, 2021

🎊 +1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 21m 0s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 1s codespell was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 2 new or modified test files.
_ trunk Compile Tests _
+1 💚 mvninstall 33m 6s trunk passed
+1 💚 compile 22m 43s trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚 compile 22m 45s trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚 checkstyle 0m 36s trunk passed
+1 💚 mvnsite 0m 43s trunk passed
+1 💚 javadoc 0m 41s trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚 javadoc 0m 35s trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚 spotbugs 1m 1s trunk passed
+1 💚 shadedclient 18m 35s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 💚 mvninstall 0m 25s the patch passed
+1 💚 compile 26m 59s the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚 javac 26m 59s the patch passed
+1 💚 compile 23m 0s the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚 javac 23m 0s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
-0 ⚠️ checkstyle 0m 33s /results-checkstyle-hadoop-common-project_hadoop-auth.txt hadoop-common-project/hadoop-auth: The patch generated 2 new + 159 unchanged - 1 fixed = 161 total (was 160)
+1 💚 mvnsite 0m 42s the patch passed
+1 💚 javadoc 0m 40s the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚 javadoc 0m 36s the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚 spotbugs 1m 14s the patch passed
+1 💚 shadedclient 19m 12s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 3m 49s hadoop-auth in the patch passed.
+1 💚 asflicense 0m 56s The patch does not generate ASF License warnings.
203m 29s
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3206/1/artifact/out/Dockerfile
GITHUB PR #3206
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname Linux 2e22b0ab2216 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 7b5b2cb
Default Java Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3206/1/testReport/
Max. process+thread count 549 (vs. ulimit of 5500)
modules C: hadoop-common-project/hadoop-auth U: hadoop-common-project/hadoop-auth
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3206/1/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

@tomicooler
fixup! YARN-10814. Fallback to RandomSecretProvider if the secret fil…
79f6066 
…e is empty.

Change-Id: I83d828915e7763e02d30f48f923e2d8ed2b7d146
@hadoop-yetus
Copy link

hadoop-yetus commented Jul 15, 2021

🎊 +1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 30m 32s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 1s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 2 new or modified test files.
_ trunk Compile Tests _
+1 💚 mvninstall 37m 29s trunk passed
+1 💚 compile 27m 8s trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚 compile 19m 2s trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚 checkstyle 0m 42s trunk passed
+1 💚 mvnsite 0m 48s trunk passed
+1 💚 javadoc 0m 44s trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚 javadoc 0m 41s trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚 spotbugs 0m 58s trunk passed
+1 💚 shadedclient 14m 35s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 💚 mvninstall 0m 22s the patch passed
+1 💚 compile 20m 24s the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚 javac 20m 24s the patch passed
+1 💚 compile 18m 15s the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚 javac 18m 15s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 checkstyle 0m 39s hadoop-common-project/hadoop-auth: The patch generated 0 new + 159 unchanged - 1 fixed = 159 total (was 160)
+1 💚 mvnsite 0m 46s the patch passed
+1 💚 javadoc 0m 43s the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚 javadoc 0m 40s the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚 spotbugs 1m 7s the patch passed
+1 💚 shadedclient 14m 26s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 3m 41s hadoop-auth in the patch passed.
+1 💚 asflicense 0m 57s The patch does not generate ASF License warnings.
198m 4s
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3206/2/artifact/out/Dockerfile
GITHUB PR #3206
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname Linux d73b9f9730a2 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 79f6066
Default Java Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3206/2/testReport/
Max. process+thread count 545 (vs. ulimit of 5500)
modules C: hadoop-common-project/hadoop-auth U: hadoop-common-project/hadoop-auth
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3206/2/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

9uapaw
9uapaw requested changes Jul 21, 2021
View reviewed changes
Copy link
Contributor

@9uapaw 9uapaw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the patch @tomicooler. I have found some minor nit, overall it looks good to me!

@tomicooler
fixup! YARN-10814. Fallback to RandomSecretProvider if the secret fil…
8ce9b71 
…e is empty.

Change-Id: Ib92d2daa687cccde8a9d202b647679f55ea126a5
@tomicooler
fixup! YARN-10814. Fallback to RandomSecretProvider if the secret fil…
8e94b83 
…e is empty.

Change-Id: Id54862b6a4ce88ca7b09515ab7399069468c04a2
@tomicooler
fixup! YARN-10814. Fallback to RandomSecretProvider if the secret fil…
e7a9e0a 
…e is empty.

Change-Id: I9cd066e80076b3501cbf7f9e74994a3b9388b033
@hadoop-yetus
Copy link

hadoop-yetus commented Jul 21, 2021

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 36s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 1s codespell was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 2 new or modified test files.
_ trunk Compile Tests _
-1 ❌ mvninstall 9m 19s /branch-mvninstall-root.txt root in trunk failed.
+1 💚 compile 28m 23s trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚 compile 20m 30s trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚 checkstyle 0m 43s trunk passed
+1 💚 mvnsite 0m 50s trunk passed
+1 💚 javadoc 0m 45s trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚 javadoc 0m 44s trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚 spotbugs 1m 3s trunk passed
+1 💚 shadedclient 15m 25s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 💚 mvninstall 0m 23s the patch passed
+1 💚 compile 24m 39s the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚 javac 24m 39s the patch passed
+1 💚 compile 24m 10s the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚 javac 24m 10s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 checkstyle 0m 40s hadoop-common-project/hadoop-auth: The patch generated 0 new + 159 unchanged - 1 fixed = 159 total (was 160)
+1 💚 mvnsite 0m 53s the patch passed
+1 💚 javadoc 0m 46s the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚 javadoc 0m 41s the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚 spotbugs 1m 20s the patch passed
+1 💚 shadedclient 18m 42s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 3m 38s hadoop-auth in the patch passed.
+1 💚 asflicense 1m 2s The patch does not generate ASF License warnings.
158m 43s
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3206/3/artifact/out/Dockerfile
GITHUB PR #3206
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname Linux d5f273530175 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / e7a9e0a
Default Java Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3206/3/testReport/
Max. process+thread count 618 (vs. ulimit of 5500)
modules C: hadoop-common-project/hadoop-auth U: hadoop-common-project/hadoop-auth
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3206/3/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

@tomicooler
Just triggering the yetus precommit job
f9f4bb3 
Change-Id: Ibbfc042d4979bcb72eba799303ee56b67d5d0396
@hadoop-yetus
Copy link

hadoop-yetus commented Jul 21, 2021

🎊 +1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 40s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 1s codespell was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 2 new or modified test files.
_ trunk Compile Tests _
+1 💚 mvninstall 34m 39s trunk passed
+1 💚 compile 25m 5s trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚 compile 20m 36s trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚 checkstyle 0m 44s trunk passed
+1 💚 mvnsite 0m 50s trunk passed
+1 💚 javadoc 0m 45s trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚 javadoc 0m 42s trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚 spotbugs 1m 5s trunk passed
+1 💚 shadedclient 14m 41s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 💚 mvninstall 0m 25s the patch passed
+1 💚 compile 26m 9s the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚 javac 26m 9s the patch passed
+1 💚 compile 21m 3s the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚 javac 21m 3s the patch passed
+1 💚 blanks 0m 1s The patch has no blanks issues.
+1 💚 checkstyle 0m 37s hadoop-common-project/hadoop-auth: The patch generated 0 new + 159 unchanged - 1 fixed = 159 total (was 160)
+1 💚 mvnsite 0m 47s the patch passed
+1 💚 javadoc 0m 44s the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚 javadoc 0m 42s the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚 spotbugs 1m 14s the patch passed
+1 💚 shadedclient 14m 58s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 3m 30s hadoop-auth in the patch passed.
+1 💚 asflicense 0m 59s The patch does not generate ASF License warnings.
174m 27s
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3206/4/artifact/out/Dockerfile
GITHUB PR #3206
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname Linux fa69f3e9c16e 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / f9f4bb3
Default Java Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3206/4/testReport/
Max. process+thread count 678 (vs. ulimit of 5500)
modules C: hadoop-common-project/hadoop-auth U: hadoop-common-project/hadoop-auth
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3206/4/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

@9uapaw
Copy link
Contributor

9uapaw commented Jul 26, 2021

Thanks @tomicooler for the update. Latest revision looks good to me, +1 non-binding.

shuzirra
shuzirra approved these changes Jul 28, 2021
View reviewed changes
Copy link

@shuzirra shuzirra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tomicooler thank you for working on this, nice clean solution, LGTM+1.

@shuzirra shuzirra changed the title YARN-10814. Fallback to RandomSecretProvider if the secret file is em… YARN-10814. Fallback to RandomSecretProvider if the secret file is empty Jul 30, 2021
@shuzirra shuzirra merged commit 798a083 into apache:trunk Jul 30, 2021
kiran-maturi pushed a commit to kiran-maturi/hadoop that referenced this pull request Nov 24, 2021
@tomicooler
YARN-10814. Fallback to RandomSecretProvider if the secret file is em…
87457da 
…pty (apache#3206)

The rest endpoint would be unusable with an empty secret file
(throwing IllegalArgumentExceptions).

Any IO error would have resulted in the same fallback path.

Co-authored-by: Tamas Domok <tdomok@cloudera.com>
jojochuang pushed a commit to jojochuang/hadoop that referenced this pull request May 23, 2023
@tomicooler @brumi1024
COMPX-6683. YARN-10814. Fallback to RandomSecretProvider if the secre…
eacf54d 
…t file is empty (apache#3206)

The rest endpoint would be unusable with an empty secret file
(throwing IllegalArgumentExceptions).

Any IO error would have resulted in the same fallback path.

Co-authored-by: Tamas Domok <tdomok@cloudera.com>
(cherry picked from commit 798a083)
Change-Id: I4eb41064a86be3d8e8a1e7e9abe40841d1680cf1
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@szilard-nemeth szilard-nemeth szilard-nemeth left review comments

@9uapaw 9uapaw 9uapaw requested changes

@zhuqi-lucas zhuqi-lucas zhuqi-lucas approved these changes

@shuzirra shuzirra shuzirra approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@tomicooler @hadoop-yetus @9uapaw @szilard-nemeth @zhuqi-lucas @shuzirra