Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HADOOP-19501. Skip tests that depend on SecurityManager if the JVM does not support it #7507

Open
wants to merge 5 commits into
base: trunk
Choose a base branch
from

Conversation

stoty
Copy link
Contributor

@stoty stoty commented Mar 14, 2025

Description of PR

Due to JEP411 / JEP486 SecurityManager is either completely removed or has to be explicitly re-enabled in newer Java versions.

This patch adds assume() methods to those tests, so that they are skipped instead of erroring out when the
JVM does not let them set the SecurityManager.

A few tests have already been converted not to require SecurityManager, but the SecurityManager setting code
has not been removed. In those cases I have simply removed the SecurityManager-related calls.

How was this patch tested?

This was tested in my JDK23 branch with JDK23, and the standard CI checks for older Javas.

For code changes:

  • Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
  • Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
  • If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
  • If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

@stoty stoty changed the title HADOOP-19501 Skip tests that depend on SecurityManager if the JVM does not support it HADOOP-19501. Skip tests that depend on SecurityManager if the JVM does not support it Mar 14, 2025
@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 30s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 1s codespell was not available.
+0 🆗 detsecrets 0m 1s detect-secrets was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 7 new or modified test files.
_ trunk Compile Tests _
+0 🆗 mvndep 6m 7s Maven dependency ordering for branch
+1 💚 mvninstall 31m 34s trunk passed
+1 💚 compile 15m 38s trunk passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚 compile 13m 41s trunk passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
+1 💚 checkstyle 4m 9s trunk passed
+1 💚 mvnsite 6m 0s trunk passed
+1 💚 javadoc 5m 36s trunk passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚 javadoc 5m 57s trunk passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
+1 💚 spotbugs 9m 56s trunk passed
+1 💚 shadedclient 34m 40s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 31s Maven dependency ordering for patch
+1 💚 mvninstall 3m 41s the patch passed
+1 💚 compile 14m 52s the patch passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚 javac 14m 52s the patch passed
+1 💚 compile 13m 43s the patch passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
+1 💚 javac 13m 43s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
-0 ⚠️ checkstyle 4m 8s /results-checkstyle-root.txt root: The patch generated 2 new + 160 unchanged - 2 fixed = 162 total (was 162)
+1 💚 mvnsite 6m 3s the patch passed
+1 💚 javadoc 5m 33s the patch passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚 javadoc 6m 4s the patch passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
+1 💚 spotbugs 11m 12s the patch passed
+1 💚 shadedclient 34m 31s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 1m 53s hadoop-hdfs in the patch passed.
+1 💚 unit 111m 34s hadoop-yarn-server-resourcemanager in the patch passed.
-1 ❌ unit 27m 35s /patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt hadoop-yarn-server-nodemanager in the patch passed.
+1 💚 unit 129m 22s hadoop-mapreduce-client-jobclient in the patch passed.
+1 💚 unit 26m 3s hadoop-distcp in the patch passed.
+1 💚 unit 17m 3s hadoop-gridmix in the patch passed.
+1 💚 asflicense 1m 14s The patch does not generate ASF License warnings.
555m 43s
Reason Tests
Failed junit tests hadoop.yarn.server.nodemanager.containermanager.logaggregation.TestLogAggregationService
Subsystem Report/Notes
Docker ClientAPI=1.48 ServerAPI=1.48 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7507/2/artifact/out/Dockerfile
GITHUB PR #7507
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname Linux a16c748b3d8d 5.15.0-134-generic #145-Ubuntu SMP Wed Feb 12 20:08:39 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / ef27d7d
Default Java Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7507/2/testReport/
Max. process+thread count 1248 (vs. ulimit of 5500)
modules C: hadoop-hdfs-project/hadoop-hdfs hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient hadoop-tools/hadoop-distcp hadoop-tools/hadoop-gridmix U: .
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7507/2/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 33s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 1s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 7 new or modified test files.
_ trunk Compile Tests _
+0 🆗 mvndep 6m 26s Maven dependency ordering for branch
+1 💚 mvninstall 32m 6s trunk passed
+1 💚 compile 15m 35s trunk passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚 compile 13m 42s trunk passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
+1 💚 checkstyle 4m 11s trunk passed
+1 💚 mvnsite 6m 11s trunk passed
+1 💚 javadoc 5m 35s trunk passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚 javadoc 5m 59s trunk passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
+1 💚 spotbugs 10m 3s trunk passed
+1 💚 shadedclient 34m 22s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 32s Maven dependency ordering for patch
+1 💚 mvninstall 3m 40s the patch passed
+1 💚 compile 15m 4s the patch passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚 javac 15m 4s the patch passed
+1 💚 compile 13m 28s the patch passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
+1 💚 javac 13m 28s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
-0 ⚠️ checkstyle 4m 6s /results-checkstyle-root.txt root: The patch generated 2 new + 160 unchanged - 2 fixed = 162 total (was 162)
+1 💚 mvnsite 6m 5s the patch passed
+1 💚 javadoc 5m 37s the patch passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚 javadoc 5m 57s the patch passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
+1 💚 spotbugs 11m 6s the patch passed
+1 💚 shadedclient 34m 20s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 1m 54s hadoop-hdfs in the patch passed.
+1 💚 unit 112m 13s hadoop-yarn-server-resourcemanager in the patch passed.
-1 ❌ unit 27m 43s /patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt hadoop-yarn-server-nodemanager in the patch passed.
+1 💚 unit 129m 40s hadoop-mapreduce-client-jobclient in the patch passed.
+1 💚 unit 26m 41s hadoop-distcp in the patch passed.
+1 💚 unit 16m 57s hadoop-gridmix in the patch passed.
+1 💚 asflicense 1m 15s The patch does not generate ASF License warnings.
558m 0s
Reason Tests
Failed junit tests hadoop.yarn.server.nodemanager.containermanager.logaggregation.TestLogAggregationService
Subsystem Report/Notes
Docker ClientAPI=1.48 ServerAPI=1.48 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7507/1/artifact/out/Dockerfile
GITHUB PR #7507
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname Linux 3ee3531b6efd 5.15.0-131-generic #141-Ubuntu SMP Fri Jan 10 21:18:28 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / ef27d7d
Default Java Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7507/1/testReport/
Max. process+thread count 1245 (vs. ulimit of 5500)
modules C: hadoop-hdfs-project/hadoop-hdfs hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient hadoop-tools/hadoop-distcp hadoop-tools/hadoop-gridmix U: .
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7507/1/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@hadoop-yetus
Copy link

🎊 +1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 51s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 1s codespell was not available.
+0 🆗 detsecrets 0m 1s detect-secrets was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 7 new or modified test files.
_ trunk Compile Tests _
+0 🆗 mvndep 6m 0s Maven dependency ordering for branch
+1 💚 mvninstall 36m 33s trunk passed
+1 💚 compile 17m 45s trunk passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚 compile 15m 10s trunk passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
+1 💚 checkstyle 4m 38s trunk passed
+1 💚 mvnsite 5m 56s trunk passed
+1 💚 javadoc 5m 35s trunk passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚 javadoc 5m 42s trunk passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
+1 💚 spotbugs 9m 53s trunk passed
+1 💚 shadedclient 39m 35s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 32s Maven dependency ordering for patch
+1 💚 mvninstall 3m 38s the patch passed
+1 💚 compile 16m 56s the patch passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚 javac 16m 56s the patch passed
+1 💚 compile 14m 56s the patch passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
+1 💚 javac 14m 56s the patch passed
+1 💚 blanks 0m 1s The patch has no blanks issues.
-0 ⚠️ checkstyle 4m 34s /results-checkstyle-root.txt root: The patch generated 2 new + 160 unchanged - 2 fixed = 162 total (was 162)
+1 💚 mvnsite 5m 52s the patch passed
+1 💚 javadoc 5m 29s the patch passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚 javadoc 5m 42s the patch passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
+1 💚 spotbugs 10m 59s the patch passed
+1 💚 shadedclient 40m 21s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 2m 17s hadoop-hdfs in the patch passed.
+1 💚 unit 115m 0s hadoop-yarn-server-resourcemanager in the patch passed.
+1 💚 unit 27m 19s hadoop-yarn-server-nodemanager in the patch passed.
+1 💚 unit 126m 23s hadoop-mapreduce-client-jobclient in the patch passed.
+1 💚 unit 26m 0s hadoop-distcp in the patch passed.
+1 💚 unit 15m 57s hadoop-gridmix in the patch passed.
+1 💚 asflicense 1m 10s The patch does not generate ASF License warnings.
577m 39s
Subsystem Report/Notes
Docker ClientAPI=1.48 ServerAPI=1.48 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7507/3/artifact/out/Dockerfile
GITHUB PR #7507
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname Linux 9f409cda3b1d 5.15.0-131-generic #141-Ubuntu SMP Fri Jan 10 21:18:28 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 8b7539a
Default Java Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7507/3/testReport/
Max. process+thread count 1130 (vs. ulimit of 5500)
modules C: hadoop-hdfs-project/hadoop-hdfs hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient hadoop-tools/hadoop-distcp hadoop-tools/hadoop-gridmix U: .
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7507/3/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@hadoop-yetus
Copy link

🎊 +1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 50s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 7 new or modified test files.
_ trunk Compile Tests _
+0 🆗 mvndep 5m 41s Maven dependency ordering for branch
+1 💚 mvninstall 36m 21s trunk passed
+1 💚 compile 17m 38s trunk passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚 compile 14m 57s trunk passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
+1 💚 checkstyle 4m 37s trunk passed
+1 💚 mvnsite 5m 59s trunk passed
+1 💚 javadoc 5m 33s trunk passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚 javadoc 5m 44s trunk passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
+1 💚 spotbugs 9m 57s trunk passed
+1 💚 shadedclient 39m 4s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 31s Maven dependency ordering for patch
+1 💚 mvninstall 3m 40s the patch passed
+1 💚 compile 16m 40s the patch passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚 javac 16m 40s the patch passed
+1 💚 compile 14m 54s the patch passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
+1 💚 javac 14m 54s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 checkstyle 4m 34s root: The patch generated 0 new + 160 unchanged - 2 fixed = 160 total (was 162)
+1 💚 mvnsite 5m 51s the patch passed
+1 💚 javadoc 5m 30s the patch passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚 javadoc 5m 42s the patch passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
+1 💚 spotbugs 10m 57s the patch passed
+1 💚 shadedclient 39m 25s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 1m 50s hadoop-hdfs in the patch passed.
+1 💚 unit 114m 14s hadoop-yarn-server-resourcemanager in the patch passed.
+1 💚 unit 28m 22s hadoop-yarn-server-nodemanager in the patch passed.
+1 💚 unit 127m 13s hadoop-mapreduce-client-jobclient in the patch passed.
+1 💚 unit 26m 24s hadoop-distcp in the patch passed.
+1 💚 unit 16m 23s hadoop-gridmix in the patch passed.
+1 💚 asflicense 1m 11s The patch does not generate ASF License warnings.
576m 31s
Subsystem Report/Notes
Docker ClientAPI=1.48 ServerAPI=1.48 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7507/4/artifact/out/Dockerfile
GITHUB PR #7507
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname Linux ef16a0172d8c 5.15.0-131-generic #141-Ubuntu SMP Fri Jan 10 21:18:28 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 99470a0
Default Java Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7507/4/testReport/
Max. process+thread count 1131 (vs. ulimit of 5500)
modules C: hadoop-hdfs-project/hadoop-hdfs hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient hadoop-tools/hadoop-distcp hadoop-tools/hadoop-gridmix U: .
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7507/4/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@stoty
Copy link
Contributor Author

stoty commented Mar 27, 2025

Can you please take a look @slfan1989 @cnauroth ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants