Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

HADOOP-18916. Exclude all module-info classes #6131

Merged
merged 2 commits into from
Oct 13, 2023
Merged

HADOOP-18916. Exclude all module-info classes #6131

merged 2 commits into from
Oct 13, 2023

Conversation

pjfanning
Copy link
Contributor

@pjfanning pjfanning commented Sep 30, 2023 •
edited
Loading

Description of PR

How was this patch tested?

For code changes:

  • Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
  • Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
  • If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
  • If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 47s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+0 🆗 xmllint 0m 0s xmllint was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
-1 ❌ test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+0 🆗 mvndep 13m 55s Maven dependency ordering for branch
+1 💚 mvninstall 36m 2s trunk passed
+1 💚 compile 0m 22s trunk passed with JDK Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04
+1 💚 compile 0m 22s trunk passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
+1 💚 mvnsite 0m 47s trunk passed
+1 💚 javadoc 0m 45s trunk passed with JDK Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04
+1 💚 javadoc 0m 42s trunk passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
+1 💚 shadedclient 90m 13s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 28s Maven dependency ordering for patch
+1 💚 mvninstall 23m 45s the patch passed
+1 💚 compile 0m 15s the patch passed with JDK Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04
+1 💚 javac 0m 15s the patch passed
+1 💚 compile 0m 15s the patch passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
+1 💚 javac 0m 15s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 mvnsite 0m 30s the patch passed
+1 💚 javadoc 0m 28s the patch passed with JDK Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04
+1 💚 javadoc 0m 27s the patch passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
+1 💚 shadedclient 41m 49s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 0m 17s hadoop-client-runtime in the patch passed.
+1 💚 unit 0m 17s hadoop-client-minicluster in the patch passed.
+1 💚 asflicense 0m 34s The patch does not generate ASF License warnings.
163m 3s
Subsystem Report/Notes
Docker ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6131/1/artifact/out/Dockerfile
GITHUB PR #6131
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell detsecrets xmllint
uname Linux d032fa14a007 4.15.0-213-generic #224-Ubuntu SMP Mon Jun 19 13:30:12 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 1148f5d
Default Java Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6131/1/testReport/
Max. process+thread count 583 (vs. ulimit of 5500)
modules C: hadoop-client-modules/hadoop-client-runtime hadoop-client-modules/hadoop-client-minicluster U: hadoop-client-modules
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6131/1/console
versions git=2.25.1 maven=3.6.3
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@pjfanning
Copy link
Contributor Author

@ayushtkn @steveloughran this issue is affecting a number of backport PRs that I have made for branch3.3. The idea here is to get this PR merged to trunk and backported to branch3.3.

An example broken build (a PR targeted to branch3.3) - https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6151/3/artifact/out/patch-javadoc-root.txt

@ayushtkn
Copy link
Member

ayushtkn commented Oct 7, 2023

Why does these issues don't happen on trunk, There must be some commit present in trunk which prevents these, can we just backport that ticket?

@pjfanning
Copy link
Contributor Author

@ayushtkn the thing is that more and more jars are having module-info classes added to them. To have the hadoop build choose 2 sets of coordinates and ignore the rest seems like a bad idea.

I've wasted tonnes of time on 3.3 builds recently and am not going to spend any more time on them.

ayushtkn
ayushtkn approved these changes Oct 11, 2023
View reviewed changes
Copy link
Member

@ayushtkn ayushtkn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Honestly, to me this change looks fair enough, we need this, every second dependency upgrade is breaking on branch-3 & if we don't find a way, branch-3 would stop getting some relevant changes....

I am good to commit this, but good if get one more approval

@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 49s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 1s codespell was not available.
+0 🆗 detsecrets 0m 1s detect-secrets was not available.
+0 🆗 xmllint 0m 1s xmllint was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
-1 ❌ test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+0 🆗 mvndep 16m 14s Maven dependency ordering for branch
+1 💚 mvninstall 36m 19s trunk passed
+1 💚 compile 0m 23s trunk passed with JDK Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04
+1 💚 compile 0m 22s trunk passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
+1 💚 mvnsite 0m 47s trunk passed
+1 💚 javadoc 0m 45s trunk passed with JDK Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04
+1 💚 javadoc 0m 42s trunk passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
+1 💚 shadedclient 93m 8s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 26s Maven dependency ordering for patch
+1 💚 mvninstall 20m 22s the patch passed
+1 💚 compile 0m 15s the patch passed with JDK Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04
+1 💚 javac 0m 15s the patch passed
+1 💚 compile 0m 15s the patch passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
+1 💚 javac 0m 15s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 mvnsite 0m 31s the patch passed
+1 💚 javadoc 0m 28s the patch passed with JDK Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04
+1 💚 javadoc 0m 27s the patch passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
+1 💚 shadedclient 39m 56s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 0m 24s hadoop-client-runtime in the patch passed.
+1 💚 unit 0m 23s hadoop-client-minicluster in the patch passed.
+1 💚 asflicense 0m 35s The patch does not generate ASF License warnings.
160m 53s
Subsystem Report/Notes
Docker ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6131/2/artifact/out/Dockerfile
GITHUB PR #6131
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell detsecrets xmllint
uname Linux 4d563fddf2a7 4.15.0-213-generic #224-Ubuntu SMP Mon Jun 19 13:30:12 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 0e22fd0
Default Java Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6131/2/testReport/
Max. process+thread count 578 (vs. ulimit of 5500)
modules C: hadoop-client-modules/hadoop-client-runtime hadoop-client-modules/hadoop-client-minicluster U: hadoop-client-modules
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6131/2/console
versions git=2.25.1 maven=3.6.3
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

steveloughran
steveloughran approved these changes Oct 13, 2023
View reviewed changes
Copy link
Contributor

@steveloughran steveloughran left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM
+1

@steveloughran steveloughran merged commit 3d7b58d into apache:trunk Oct 13, 2023
1 of 3 checks passed
@steveloughran steveloughran mentioned this pull request Oct 13, 2023
Merged
pjfanning added a commit to pjfanning/hadoop that referenced this pull request Oct 13, 2023
@pjfanning
HADOOP-18916. Exclude all module-info classes from uber jars (apache#…
ff65f89 
…6131)

Removes java9 and java11 from all modules pulled into the hadoop-client
and hadoop-client-minicluster modules.

Contributed by PJ Fanning
ayushtkn pushed a commit that referenced this pull request Oct 14, 2023
@pjfanning
HADOOP-18916. Exclude all module-info classes from uber jars (#6131) (#…
553b8ff 
…6188)

Removes java9 and java11 from all modules pulled into the hadoop-client
and hadoop-client-minicluster modules.

Contributed by PJ Fanning
jiajunmao pushed a commit to jiajunmao/hadoop-MLEC that referenced this pull request Feb 6, 2024
@pjfanning @jiajunmao
HADOOP-18916. Exclude all module-info classes from uber jars (apache#…
b541910 
…6131)


Removes java9 and java11 from all modules pulled into the hadoop-client
and hadoop-client-minicluster modules.

Contributed by PJ Fanning
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@steveloughran steveloughran steveloughran approved these changes

@ayushtkn ayushtkn ayushtkn approved these changes

Assignees
No one assigned
Labels
build trunk
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@pjfanning @hadoop-yetus @ayushtkn @steveloughran