Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Related jiras that update our handling of Hadoop transitive dependencies #192

Merged
merged 4 commits into from
Apr 26, 2019

Conversation

busbey
Copy link
Contributor

@busbey busbey commented Apr 25, 2019

This includes two general fixes that will be needed by all branch-2 releases

  • HBASE-22312 when built against hadoop 3 our hbase-shaded-mapreduce module incorrectly includes the mapreduce client's transitive dependencies
  • HBASE-22314 when built against hadoop 3 our hbase-shaded-client-byo-hadoop module incorrectly includes the hadoop client's transitive dependencies

and also two fixes that will only be needed for upcoming minor releases, since they fix problems that depend on which version(s) of Hadoop 3 end up getting released and built against. These are both modified versions of patches provided by other contributors.

  • HBASE-22109 Update hbase shaded client for new transitive dependencies of guava after hadoop update
  • HBASE-22087 Update LICENSE/shading for the dependencies from the latest Hadoop trunk

busbey and others added 4 commits April 25, 2019 13:38
…s as provided scope to avoid inclusion of unnecessary transitive depednencies
…s of guava after hadoop update

* Hadoop updated to Guava 27.0-jre
* Guava 27 adds error checker framework as a transitive dependency
* update relocation rules to relocate it for the shaded client that includes hadoop

Co-authored-by: Sean Busbey <busbey@apache.org>
Signed-off-by: Sean Busbey <busbey@apache.org>
…st Hadoop trunk.

When building with latest hadoop:
* JLine is now in the assembly so update licensing
* shaded client w/hadoop needs relocation for a couple new hadoop transitives

Co-authored-by: Sean Busbey <busbey@apache.org>
Signed-off-by: Sean Busbey <busbey@apache.org>
@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
0 reexec 35 Docker mode activated.
_ Prechecks _
+1 @author 0 The patch does not contain any @author tags.
-0 test4tests 0 The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ master Compile Tests _
0 mvndep 25 Maven dependency ordering for branch
+1 mvninstall 253 master passed
+1 compile 65 master passed
+1 shadedjars 264 branch has no errors when building our shaded downstream artifacts.
+1 javadoc 47 master passed
_ Patch Compile Tests _
0 mvndep 14 Maven dependency ordering for patch
+1 mvninstall 238 the patch passed
+1 compile 66 the patch passed
+1 javac 66 the patch passed
+1 whitespace 0 The patch has no whitespace issues.
+1 xml 5 The patch has no ill-formed XML file.
+1 shadedjars 256 patch has no errors when building our shaded downstream artifacts.
+1 hadoopcheck 494 Patch does not cause any errors with Hadoop 2.7.4 or 3.0.0.
+1 javadoc 47 the patch passed
_ Other Tests _
+1 unit 12 hbase-resource-bundle in the patch passed.
+1 unit 23 hbase-shaded in the patch passed.
+1 unit 15 hbase-shaded-client-byo-hadoop in the patch passed.
+1 unit 16 hbase-shaded-mapreduce in the patch passed.
+1 asflicense 41 The patch does not generate ASF License warnings.
1999
Subsystem Report/Notes
Docker Client=17.05.0-ce Server=17.05.0-ce base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-192/1/artifact/out/Dockerfile
GITHUB PR #192
Optional Tests dupname asflicense javac javadoc unit xml shadedjars hadoopcheck compile
uname Linux 0741eedfde1b 3.13.0-153-generic #203-Ubuntu SMP Thu Jun 14 08:52:28 UTC 2018 x86_64 GNU/Linux
Build tool maven
Personality /testptch/patchprocess/precommit/personality/provided.sh
git revision master / ec36372
maven version: Apache Maven 3.5.4 (1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z)
Default Java 1.8.0_181
Test Results https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-192/1/testReport/
Max. process+thread count 86 (vs. ulimit of 10000)
modules C: hbase-resource-bundle hbase-shaded hbase-shaded/hbase-shaded-client-byo-hadoop hbase-shaded/hbase-shaded-mapreduce U: .
Console output https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-192/1/console
Powered by Apache Yetus 0.9.0 http://yetus.apache.org

This message was automatically generated.

Copy link
Contributor

@willshen willshen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

<url>https://opensource.org/licenses/BSD-3-Clause</url>
<distribution>repo</distribution>
<comments>
Copyright (c) 2002-2018, the original author or authors.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just wondering, does it matter that we are in 2019 now?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not unless the version of jline we end up packaging is from 2019.

Copy link
Contributor

@saintstack saintstack left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems fine to me. Looks like a horrid debug affair (that stuff around dnsjava utility stuff).

@busbey busbey merged commit 962585d into apache:master Apr 26, 2019
@busbey busbey deleted the hadoop-update-prereqs branch April 26, 2019 20:03
@busbey
Copy link
Contributor Author

busbey commented Apr 26, 2019

arg. merge button doesn't include review sign off in the git commit information. 😿

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
5 participants