Skip to content

HADOOP-18006. maven-enforcer-plugin's execution of banned-illegal-imports gets overridden in child poms#3648

Merged
tasanuma merged 2 commits intoapache:trunkfrom
virajjasani:HADOOP-18006-trunk
Nov 15, 2021
Merged

HADOOP-18006. maven-enforcer-plugin's execution of banned-illegal-imports gets overridden in child poms#3648
tasanuma merged 2 commits intoapache:trunkfrom
virajjasani:HADOOP-18006-trunk

Conversation

@virajjasani
Copy link
Contributor

Description of PR

When we specify any maven plugin with execution tag in the parent as well as child modules, child module plugin overrides parent plugin. For instance, when banned-illegal-imports is applied for any child module with only one banned import (let’s say Preconditions), then only that banned import is covered by that child module and all imports defined in parent module (e.g Sets, Lists etc) are overridden and they are no longer applied.
As of today, hadoop-hdfs module will not complain about Sets even if i import it from guava banned imports but on the other hand, hadoop-yarn module doesn’t have any child level banned-illegal-imports defined so yarn modules will fail if Sets guava import is used.
So going forward, it would be good to replace guava imports with Hadoop’s own imports module-by-module and only at the end, we should add new entry to parent pom banned-illegal-imports list.

For code changes:

  • Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?

@tasanuma
Copy link
Member

@virajjasani Thanks for reporting the problem. It seems we have no choice but remove them. +1, pending Jenkins.

So going forward, it would be good to replace guava imports with Hadoop’s own imports module-by-module and only at the end, we should add new entry to parent pom banned-illegal-imports list.

+1 for this plan.

@amahussein Could you review it too?

@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 1m 4s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 1s codespell was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 1 new or modified test files.
_ trunk Compile Tests _
+0 🆗 mvndep 12m 28s Maven dependency ordering for branch
+1 💚 mvninstall 26m 43s trunk passed
+1 💚 compile 29m 26s trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚 compile 24m 20s trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚 checkstyle 3m 51s trunk passed
+1 💚 mvnsite 9m 27s trunk passed
+1 💚 javadoc 7m 38s trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚 javadoc 8m 39s trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚 spotbugs 16m 30s trunk passed
+1 💚 shadedclient 23m 43s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 27s Maven dependency ordering for patch
+1 💚 mvninstall 6m 25s the patch passed
+1 💚 compile 28m 53s the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚 javac 28m 53s the patch passed
+1 💚 compile 24m 16s the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚 javac 24m 16s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 checkstyle 3m 55s the patch passed
+1 💚 mvnsite 9m 17s the patch passed
+1 💚 xml 0m 12s The patch has no ill-formed XML file.
+1 💚 javadoc 7m 33s the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚 javadoc 8m 33s the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚 spotbugs 18m 43s the patch passed
+1 💚 shadedclient 24m 8s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 3m 37s hadoop-auth in the patch passed.
+1 💚 unit 18m 8s hadoop-common in the patch passed.
+1 💚 unit 0m 51s hadoop-nfs in the patch passed.
+1 💚 unit 3m 40s hadoop-kms in the patch passed.
+1 💚 unit 1m 19s hadoop-registry in the patch passed.
+1 💚 unit 2m 33s hadoop-hdfs-client in the patch passed.
+1 💚 unit 371m 27s hadoop-hdfs in the patch passed.
-1 ❌ unit 9m 6s /patch-unit-hadoop-hdfs-project_hadoop-hdfs-httpfs.txt hadoop-hdfs-httpfs in the patch passed.
+1 💚 unit 3m 35s hadoop-hdfs-nfs in the patch passed.
-1 ❌ unit 39m 22s /patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt hadoop-hdfs-rbf in the patch passed.
+1 💚 asflicense 2m 8s The patch does not generate ASF License warnings.
756m 4s
Reason Tests
Failed junit tests hadoop.fs.http.client.TestHttpFSFWithWebhdfsFileSystem
hadoop.fs.http.client.TestHttpFSFWithSWebhdfsFileSystem
hadoop.hdfs.rbfbalance.TestRouterDistCpProcedure
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3648/3/artifact/out/Dockerfile
GITHUB PR #3648
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell xml spotbugs checkstyle
uname Linux 4607a65d7c43 4.15.0-153-generic #160-Ubuntu SMP Thu Jul 29 06:54:29 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 4e9f4e4
Default Java Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3648/3/testReport/
Max. process+thread count 2501 (vs. ulimit of 5500)
modules C: hadoop-common-project/hadoop-auth hadoop-common-project/hadoop-common hadoop-common-project/hadoop-nfs hadoop-common-project/hadoop-kms hadoop-common-project/hadoop-registry hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-httpfs hadoop-hdfs-project/hadoop-hdfs-nfs hadoop-hdfs-project/hadoop-hdfs-rbf U: .
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3648/3/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 1m 8s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 3s codespell was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 1 new or modified test files.
_ trunk Compile Tests _
+0 🆗 mvndep 12m 35s Maven dependency ordering for branch
+1 💚 mvninstall 26m 40s trunk passed
+1 💚 compile 29m 32s trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚 compile 24m 30s trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚 checkstyle 4m 0s trunk passed
+1 💚 mvnsite 9m 30s trunk passed
+1 💚 javadoc 7m 40s trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚 javadoc 8m 39s trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚 spotbugs 16m 36s trunk passed
+1 💚 shadedclient 24m 11s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 25s Maven dependency ordering for patch
+1 💚 mvninstall 6m 29s the patch passed
+1 💚 compile 28m 29s the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚 javac 28m 30s the patch passed
+1 💚 compile 24m 19s the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚 javac 24m 19s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 checkstyle 3m 43s the patch passed
+1 💚 mvnsite 9m 18s the patch passed
+1 💚 xml 0m 13s The patch has no ill-formed XML file.
+1 💚 javadoc 7m 27s the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚 javadoc 8m 35s the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚 spotbugs 18m 49s the patch passed
+1 💚 shadedclient 24m 39s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 3m 36s hadoop-auth in the patch passed.
+1 💚 unit 18m 5s hadoop-common in the patch passed.
+1 💚 unit 0m 54s hadoop-nfs in the patch passed.
+1 💚 unit 3m 40s hadoop-kms in the patch passed.
+1 💚 unit 1m 19s hadoop-registry in the patch passed.
+1 💚 unit 2m 28s hadoop-hdfs-client in the patch passed.
+1 💚 unit 375m 36s hadoop-hdfs in the patch passed.
-1 ❌ unit 8m 59s /patch-unit-hadoop-hdfs-project_hadoop-hdfs-httpfs.txt hadoop-hdfs-httpfs in the patch passed.
+1 💚 unit 3m 38s hadoop-hdfs-nfs in the patch passed.
-1 ❌ unit 38m 52s /patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt hadoop-hdfs-rbf in the patch passed.
+1 💚 asflicense 1m 20s The patch does not generate ASF License warnings.
760m 4s
Reason Tests
Failed junit tests hadoop.fs.http.client.TestHttpFSFWithWebhdfsFileSystem
hadoop.fs.http.client.TestHttpFSFWithSWebhdfsFileSystem
hadoop.hdfs.rbfbalance.TestRouterDistCpProcedure
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3648/2/artifact/out/Dockerfile
GITHUB PR #3648
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell xml spotbugs checkstyle
uname Linux 23a62e88d2b6 4.15.0-153-generic #160-Ubuntu SMP Thu Jul 29 06:54:29 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 4e9f4e4
Default Java Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3648/2/testReport/
Max. process+thread count 2374 (vs. ulimit of 5500)
modules C: hadoop-common-project/hadoop-auth hadoop-common-project/hadoop-common hadoop-common-project/hadoop-nfs hadoop-common-project/hadoop-kms hadoop-common-project/hadoop-registry hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-httpfs hadoop-hdfs-project/hadoop-hdfs-nfs hadoop-hdfs-project/hadoop-hdfs-rbf U: .
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3648/2/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

@amahussein amahussein self-requested a review November 13, 2021 13:38
@amahussein
Copy link
Contributor

Thanks @virajjasani for reporting this.
+1 LGTM

@virajjasani
Copy link
Contributor Author

Thanks for the reviews @amahussein @tasanuma

@tasanuma tasanuma merged commit e14a2dc into apache:trunk Nov 15, 2021
@tasanuma
Copy link
Member

Merged it. Thanks for your contribution, @virajjasani. Thanks for your review, @amahussein.

HarshitGupta11 pushed a commit to HarshitGupta11/hadoop that referenced this pull request Nov 28, 2022
…orts gets overridden in child poms (apache#3648)

Reviewed-by: Ahmed Hussein <ahussein@apache.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants