HBASE-26838 Junit jar is not included in the hbase tar ball, causing …

[wchevreuil]

…issues for some hbase tools that do rely on it

[Apache9]

Why these two blank lines?

[wchevreuil]

Let me remove it.

[Apache9]

Will we load the libraries under this package?

And better also include hamcrest here.

[wchevreuil]

Yes, these libraries would be packed under a "lib\test" folder, and would not be added to hbase classpath automatically (the hbase script, by default, puts "lib*" on the CP).

[Apache9]

I mean we should add these libraries?

[wchevreuil]

You mean add to hbase classpath by default? Junit used to be shipped until it was scoped for test only on all modules, at least since release 2.4 (In 2.2 it was still shipped). In general, HBase processes seem to work pretty fine without it on the CP. The only setback is for the test tooling, such as IntegrationTestIngest, which will now fail without junit lib in the classpath.

[Apache9]

So how do we setback when executing IntegrationTestIngest? Manually?

[wchevreuil]

Yes, this could be easily done as HBASE_CLASSPATH="$HBASE_HOME/lib/tests/*" hbase org.apache.hadoop.hbase.IntegrationTestIngest -m slowDeterministic, since we already provide the lib within the tarball. We could update the ref guide section accordingly.

[Apache9]

I wonder what is the problem if we always add these jars to our classpath? At lease when running tools(not daemon services)?

[joshelser]

Ehh, usually it's been a thing that causes people to ask questions rather than something actually breaking. Apparently others think that we shouldnt' have this on the $CLASSPATH given they removed them.

[ndimiduk]

For me, it is a security and release hygiene issue to ship test dependencies in the runtime. It is problematic for me that some of our user utilities have runtime dependency on test jars. Only mildly related: I'm curious how much extra space those test dependencies consume in our release artifacts.

Short of the restructuring that I suggested, I am okay with this approach of isolating these test dependencies in their own directory and conditionally including them at runtime. If there are classes that we know require them, our bin/hbase script can intelligently add them to the classpath for those classes. Doing so will preserve backward compatibility while also keeping the extra jars out of the daemon services' classpaths.

[wchevreuil]

Ok, let me add it to the CP of the tools that require it, in bin/hbase script.

[ndimiduk]

I find all this very problematic. Our IntegrationTest classes are in a /src/test/java path, which maven associates with the test-jar artifact. I think what you need to do is include the test artifact as a dependency of the assembly plugin, so that all test-scope dependencies are included. However, as I pointed out in this comment on #4016, maven doesn't pack transitive dependencies of test-jar artifacts. In general, it's an anti-pattern to have any dependency at all on test-jars.

If we want to ship the IntegrationTest classes in our binary distribution, what we need to do is restructure hbase-it so that the IntegrationTest jars are in src/main/java. (We also need to mark them with some InterfaceAudience decoration, because they are in fact part of our public contract...) Restructuring the project like this will will have the happy benefit of allowing the maven assembly plugin to do the work for us.

[wchevreuil]

If we want to ship the IntegrationTest classes in our binary distribution, what we need to do is restructure hbase-it so that the IntegrationTest jars are in src/main/java. (We also need to mark them with some InterfaceAudience decoration, because they are in fact part of our public contract...) Restructuring the project like this will will have the happy benefit of allowing the maven assembly plugin to do the work for us.

Well, we already ship those and document its use on the ref guide. The problem is that we also make it possible to be run as junit tests, through maven builds, so I think that's why those are originally placed under src/test?

[joshelser]

Assuming these show up only in the right place, this changeset looks correct (meaning, I didn't build and validate what you did -- I trust you here :))

[joshelser]

Ehh, usually it's been a thing that causes people to ask questions rather than something actually breaking. Apparently others think that we shouldnt' have this on the $CLASSPATH given they removed them.

[wchevreuil]

@ndimiduk , I believe all classes on hbase-it which are exposed for running standalone follow this naming pattern. These are spread over few different packages.

[ndimiduk]

There's quite a few more classes than that.

$ grep 'public static void main' -cirIF hbase-it/src --include *.java | grep -v IntegrationTest | grep -v Action | wc -l
      46

[wchevreuil]

Yes, I had noticed before that there were few others with a main method, but the only ones really relying on junit are the "IntegrationTest" ones.

[Apache-HBase]

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 19s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+0 🆗	shelldocs	0m 0s	Shelldocs was not available.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+0 🆗	mvndep	0m 12s	Maven dependency ordering for branch
+1 💚	mvninstall	2m 41s	master passed
+1 💚	compile	8m 14s	master passed
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 10s	Maven dependency ordering for patch
+1 💚	mvninstall	2m 55s	the patch passed
+1 💚	compile	8m 44s	the patch passed
+1 💚	javac	8m 44s	the patch passed
+1 💚	shellcheck	0m 2s	There were no new shellcheck issues.
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	xml	0m 3s	The patch has no ill-formed XML file.
+1 💚	hadoopcheck	13m 38s	Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1.
		_ Other Tests _
+1 💚	asflicense	0m 21s	The patch does not generate ASF License warnings.
		44m 14s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4223/4/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#4223
Optional Tests	dupname asflicense shellcheck shelldocs javac hadoopcheck xml compile
uname	Linux c5bfe5a486f8 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / ab4351a
Default Java	AdoptOpenJDK-1.8.0_282-b08
Max. process+thread count	138 (vs. ulimit of 30000)
modules	C: hbase-assembly . U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4223/4/console
versions	git=2.17.1 maven=3.6.3 shellcheck=0.4.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

[Apache-HBase]

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 43s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+0 🆗	mvndep	0m 31s	Maven dependency ordering for branch
+1 💚	mvninstall	2m 32s	master passed
+1 💚	compile	1m 39s	master passed
+1 💚	shadedjars	3m 45s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	1m 51s	master passed
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 10s	Maven dependency ordering for patch
+1 💚	mvninstall	2m 31s	the patch passed
+1 💚	compile	1m 38s	the patch passed
+1 💚	javac	1m 38s	the patch passed
+1 💚	shadedjars	3m 45s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	1m 51s	the patch passed
		_ Other Tests _
+1 💚	unit	229m 34s	root in the patch passed.
		254m 56s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4223/4/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#4223
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 0e29fef53bea 5.4.0-1054-aws #57~18.04.1-Ubuntu SMP Thu Jul 15 03:21:36 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / ab4351a
Default Java	AdoptOpenJDK-11.0.10+9
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4223/4/testReport/
Max. process+thread count	4855 (vs. ulimit of 30000)
modules	C: hbase-assembly . U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4223/4/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

[busbey]

I don't mean to thrash things around. But I read through the changes here and the discussion around how to tell which classes in the test jars are magically detected for including this classpath and it all feels prohibitively brittle.

If I'm a developer adding in something I want run as a test on a live cluster how do I know what I have to do to get it to work here?

If I'm an operator how am I supposed to reason about which utilities get included for running with these "hbase test dependencies"?

I looked at the ref guide and I see two candidate starting points for how I'd figure this out.

1. The troubleshooting section "Running unit or integration tests"
2. The operations section "Tools and utilities"

But neither of them actually have any relevant details.

I know this has accidentally worked in the past by "just" running the class. Could we instead be intentional about things?

What if we add a new command to the hbase script similar to the one for ltt that is for running integration tests specifically? Something like it-test? That could add in the needed classpath. I am indifferent to wether folks specify which test via a fully qualified classname or a short class name. We could have it list the available classes when none is given. Then a small update to the ref guide that points folks to that listing and I guess javadocs for the listed classes would give a much needed foothold to someone who want to use this stuff.

I don't know what the other classes with main methods are in hbase-it are, but if they are supposed to be run against a cluster we ought to be able to similarly categorize them. Not everything with a main method is meant for running on a cluster, but there should be some kind of note about that in the javadocs for the class or method.

[wchevreuil]

If I'm a developer adding in something I want run as a test on a live cluster how do I know what I have to do to get it to work here?

Yeah, not much defences here other than documenting it.

If I'm an operator how am I supposed to reason about which utilities get included for running with these "hbase test dependencies"?

If we ever figure out in the hbase script all classes that would need the test dependencies, then operators shouldn't worry about it.

I know this has accidentally worked in the past by "just" running the class. Could we instead be intentional about things?

Not only that, but now the sections about running IT and chaos monkey in the ref guide are broken, because they explicitly mention some classes to be run from hbase script.

What if we add a new command to the hbase script similar to the one for ltt that is for running integration tests specifically? Something like it-test? That could add in the needed classpath.

We could do it, but would break compatibility. For example, imagine some automated QA process that invokes a set of these IT classes. Until 2.4, it was working by just running hbase IT_CLASS_NAME, but now it would require code changes.

Then a small update to the ref guide that points folks to that listing and I guess javadocs for the listed classes would give a much needed foothold to someone who want to use this stuff.

Yeah, looks like docs update is a must. Could we go with it by adding developer guidelines notices on the ref guide, javadocs and hbase-it readme that in case of any junit (or other test only lib) dependent exposed tool addition to hbase-it would require review of classpath definition in hbasescript?

[joshelser]

what we need to do is restructure hbase-it so that the IntegrationTest jars are in src/main/java.

I don't mean to thrash things around. But I read through the changes here and the discussion around how to tell which classes in the test jars are magically detected for including this classpath and it all feels prohibitively brittle.

I just want to come with some humility to ask that we don't look past fixing this immediate problem (to restore how things previously could work) to build an ideal solution. I would very much love to have our integration test code exposed via src/main/java so that maven transitive dependencies work, and JUnit wrappers so that they can be automatically invoked by a maven lifecycle phase.

I also know that just because something previously could work in a certain way, doesn't mean that it was intentional to work that way (as Busbey pointed out -- this isn't 100% clear in the Book). I think we're all on the same page that this is a gap and it would be great to make meaningful changes to close those gaps. I just want to reinforce that we have a number of test suites at $dayjob which have now been broken for some time without any remediation. Wellington's current changes will help solve that problem and we can take our time to figure out how we want standalone tests to work in HBase with less urgency attached to it.

[wchevreuil]

Ping @busbey @ndimiduk , considering mine and Josh's latest comments observations, would you still see this approach as -1, or can we go ahead with this solution for now, provided that I add reg-guide/readme comments/javadocs explaining the pre-condition for test libs to be automatically loaded into the CP by hbase script?

[ndimiduk]

Yep, in the sprit of "get it mostly fixed," I think what you have is pretty good. I did a quick search -- I think you need to include mockito in the test jars as well -- if it's not already.

$ grep 'import ' -rFh . --include '*.java' | sort -u | grep -v org\.apache | grep -v java
import com.codahale.metrics.Histogram;
import io.opentelemetry.api.common.AttributeKey;
import io.opentelemetry.api.common.Attributes;
import io.opentelemetry.api.trace.Span;
import io.opentelemetry.context.Scope;
import org.junit.After;
import org.junit.AfterClass;
import org.junit.Assert;
import org.junit.Before;
import org.junit.BeforeClass;
import org.junit.ClassRule;
import org.junit.Ignore;
import org.junit.Rule;
import org.junit.Test;
import org.junit.experimental.categories.Category;
import org.junit.internal.TextListener;
import org.junit.rules.ExternalResource;
import org.junit.rules.TestName;
import org.junit.rules.TestRule;
import org.junit.rules.Timeout;
import org.junit.runner.JUnitCore;
import org.junit.runner.Result;
import org.mockito.Mockito;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertFalse;
import static org.junit.Assert.assertNotNull;
import static org.junit.Assert.assertTrue;
import static org.junit.Assert.fail;
import static org.junit.Assume.assumeFalse;
import static org.junit.Assume.assumeTrue;

[joshelser]

I think what you have is pretty good. I did a quick search -- I think you need to include mockito in the test jars as well -- if it's not already.

That's a good catch :)

[joshelser]

Looking at your current PR branch, hbase-it/src/test/java/org/apache/hadoop/hbase/chaos/actions/TestChangeSplitPolicyAction.java is the only use of Mockito and I don't think this one has standalone "merit" to it. That said, including Mockito in this limited scope is a low-risk failsafe, IMO.

If you can update the client.xml with the same assembly descriptor change as hadoop-three-compat.xml has, I think your change is even better :).

trivially hbase org.apache.hadoop.hbase.IntegrationTestIngest fails without this change and will run with it.

[joshelser]

Just noticed this gets missed in the client tarball. Just copy this same block in hbase-assembly/src/main/assembly/client.xml

[busbey]

Ping @busbey @ndimiduk , considering mine and Josh's latest comments observations, would you still see this approach as -1, or can we go ahead with this solution for now, provided that I add reg-guide/readme comments/javadocs explaining the pre-condition for test libs to be automatically loaded into the CP by hbase script?

yeah that's fine by me.

[wchevreuil]

Thanks @joshelser @busbey and @ndimiduk ! Had pushed two additional commits, with updates to the ref-guide, adding mockito lib, and adding the test libs to the client tar ball.

[Apache-HBase]

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 41s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+0 🆗	shelldocs	0m 1s	Shelldocs was not available.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+0 🆗	mvndep	0m 21s	Maven dependency ordering for branch
+1 💚	mvninstall	2m 18s	master passed
+1 💚	compile	6m 5s	master passed
+0 🆗	refguide	2m 5s	branch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect.
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 13s	Maven dependency ordering for patch
+1 💚	mvninstall	2m 5s	the patch passed
+1 💚	compile	6m 0s	the patch passed
-0 ⚠️	javac	6m 0s	root generated 1 new + 1471 unchanged - 1 fixed = 1472 total (was 1472)
+1 💚	shellcheck	0m 1s	There were no new shellcheck issues.
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	xml	0m 3s	The patch has no ill-formed XML file.
+0 🆗	refguide	1m 36s	patch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect.
+1 💚	hadoopcheck	11m 53s	Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1.
		_ Other Tests _
+1 💚	asflicense	0m 16s	The patch does not generate ASF License warnings.
		38m 52s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4223/5/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#4223
Optional Tests	dupname asflicense shellcheck shelldocs javac hadoopcheck xml compile refguide
uname	Linux 709fdf305a45 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / 828ea91
Default Java	AdoptOpenJDK-1.8.0_282-b08
refguide	https://nightlies.apache.org/hbase/HBase-PreCommit-GitHub-PR/PR-4223/5/yetus-general-check/output/branch-site/book.html
javac	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4223/5/artifact/yetus-general-check/output/diff-compile-javac-root.txt
refguide	https://nightlies.apache.org/hbase/HBase-PreCommit-GitHub-PR/PR-4223/5/yetus-general-check/output/patch-site/book.html
Max. process+thread count	140 (vs. ulimit of 30000)
modules	C: hbase-assembly . U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4223/5/console
versions	git=2.17.1 maven=3.6.3 shellcheck=0.4.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

[joshelser]

Thanks for getting parity with the client tarball. LGTM

[Apache-HBase]

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 17s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+0 🆗	mvndep	0m 14s	Maven dependency ordering for branch
+1 💚	mvninstall	2m 40s	master passed
+1 💚	compile	1m 54s	master passed
+1 💚	shadedjars	3m 44s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	2m 9s	master passed
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 15s	Maven dependency ordering for patch
+1 💚	mvninstall	2m 35s	the patch passed
+1 💚	compile	1m 54s	the patch passed
+1 💚	javac	1m 54s	the patch passed
+1 💚	shadedjars	3m 40s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	2m 8s	the patch passed
		_ Other Tests _
+1 💚	unit	246m 18s	root in the patch passed.
		271m 40s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4223/5/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#4223
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux ae3e366d4156 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / 828ea91
Default Java	AdoptOpenJDK-11.0.10+9
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4223/5/testReport/
Max. process+thread count	4460 (vs. ulimit of 30000)
modules	C: hbase-assembly . U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4223/5/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

[Apache-HBase]

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 23s	Docker mode activated.
-0 ⚠️	yetus	0m 2s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+0 🆗	mvndep	0m 20s	Maven dependency ordering for branch
+1 💚	mvninstall	2m 14s	master passed
+1 💚	compile	1m 33s	master passed
+1 💚	shadedjars	3m 38s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	1m 35s	master passed
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 15s	Maven dependency ordering for patch
+1 💚	mvninstall	2m 12s	the patch passed
+1 💚	compile	1m 32s	the patch passed
+1 💚	javac	1m 32s	the patch passed
+1 💚	shadedjars	3m 40s	patch has no errors when building our shaded downstream artifacts.
-0 ⚠️	javadoc	1m 23s	root generated 9 new + 49 unchanged - 9 fixed = 58 total (was 58)
		_ Other Tests _
+1 💚	unit	326m 16s	root in the patch passed.
		348m 34s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4223/5/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#4223
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 6c7b2055a399 5.4.0-96-generic #109-Ubuntu SMP Wed Jan 12 16:49:16 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / 828ea91
Default Java	AdoptOpenJDK-1.8.0_282-b08
javadoc	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4223/5/artifact/yetus-jdk8-hadoop3-check/output/diff-javadoc-javadoc-root.txt
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4223/5/testReport/
Max. process+thread count	4699 (vs. ulimit of 30000)
modules	C: hbase-assembly . U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4223/5/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

[Apache-HBase]

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 42s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+0 🆗	shelldocs	0m 0s	Shelldocs was not available.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+0 🆗	mvndep	0m 26s	Maven dependency ordering for branch
+1 💚	mvninstall	2m 3s	master passed
+1 💚	compile	5m 58s	master passed
+0 🆗	refguide	1m 37s	branch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect.
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 13s	Maven dependency ordering for patch
+1 💚	mvninstall	2m 7s	the patch passed
+1 💚	compile	5m 54s	the patch passed
+1 💚	javac	5m 54s	the patch passed
+1 💚	shellcheck	0m 1s	There were no new shellcheck issues.
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	xml	0m 3s	The patch has no ill-formed XML file.
+0 🆗	refguide	1m 36s	patch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect.
+1 💚	hadoopcheck	11m 37s	Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1.
		_ Other Tests _
+1 💚	asflicense	0m 15s	The patch does not generate ASF License warnings.
		37m 26s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4223/6/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#4223
Optional Tests	dupname asflicense shellcheck shelldocs javac hadoopcheck xml compile refguide
uname	Linux 8f910519edb0 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / 828ea91
Default Java	AdoptOpenJDK-1.8.0_282-b08
refguide	https://nightlies.apache.org/hbase/HBase-PreCommit-GitHub-PR/PR-4223/6/yetus-general-check/output/branch-site/book.html
refguide	https://nightlies.apache.org/hbase/HBase-PreCommit-GitHub-PR/PR-4223/6/yetus-general-check/output/patch-site/book.html
Max. process+thread count	138 (vs. ulimit of 30000)
modules	C: hbase-assembly . U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4223/6/console
versions	git=2.17.1 maven=3.6.3 shellcheck=0.4.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

[Apache-HBase]

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 16s	Docker mode activated.
-0 ⚠️	yetus	0m 4s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+0 🆗	mvndep	0m 19s	Maven dependency ordering for branch
+1 💚	mvninstall	2m 29s	master passed
+1 💚	compile	1m 49s	master passed
+1 💚	shadedjars	3m 38s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	2m 7s	master passed
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 15s	Maven dependency ordering for patch
+1 💚	mvninstall	2m 31s	the patch passed
+1 💚	compile	1m 50s	the patch passed
+1 💚	javac	1m 50s	the patch passed
+1 💚	shadedjars	3m 41s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	2m 4s	the patch passed
		_ Other Tests _
+1 💚	unit	267m 14s	root in the patch passed.
		291m 51s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4223/6/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#4223
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 752e1d114193 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / 828ea91
Default Java	AdoptOpenJDK-11.0.10+9
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4223/6/testReport/
Max. process+thread count	4497 (vs. ulimit of 30000)
modules	C: hbase-assembly . U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4223/6/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

[Apache-HBase]

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 50s	Docker mode activated.
-0 ⚠️	yetus	0m 4s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+0 🆗	mvndep	0m 58s	Maven dependency ordering for branch
+1 💚	mvninstall	2m 40s	master passed
+1 💚	compile	1m 42s	master passed
+1 💚	shadedjars	4m 17s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	1m 51s	master passed
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 15s	Maven dependency ordering for patch
+1 💚	mvninstall	2m 26s	the patch passed
+1 💚	compile	1m 40s	the patch passed
+1 💚	javac	1m 40s	the patch passed
+1 💚	shadedjars	4m 16s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	1m 46s	the patch passed
		_ Other Tests _
+1 💚	unit	408m 18s	root in the patch passed.
		434m 6s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4223/6/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#4223
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 13af496b71fc 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / 828ea91
Default Java	AdoptOpenJDK-1.8.0_282-b08
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4223/6/testReport/
Max. process+thread count	4715 (vs. ulimit of 30000)
modules	C: hbase-assembly . U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4223/6/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

[ndimiduk]

I have one concern with the mockito change. Everything else looks great.

[ndimiduk]

I think you should not remove scope:test from this managedDependency -- this definition is pulled into many many module. Instead, can you override the dependency scope in abase-assembly/pom.xml to be scope:compile ?

[wchevreuil]

Done.

[ndimiduk]

Sorry, that should have been with "request changes".

[ndimiduk]

This works? Great!

Because compile is the default scope, someone later may be tempted to remove this distinction. Maybe add a comment to future do-gooders?

[wchevreuil]

Done.

[Apache-HBase]

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 11s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+0 🆗	mvndep	0m 18s	Maven dependency ordering for branch
+1 💚	mvninstall	2m 39s	master passed
+1 💚	compile	1m 51s	master passed
+1 💚	shadedjars	3m 42s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	2m 16s	master passed
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 16s	Maven dependency ordering for patch
+1 💚	mvninstall	3m 32s	the patch passed
+1 💚	compile	2m 24s	the patch passed
+1 💚	javac	2m 24s	the patch passed
+1 💚	shadedjars	4m 20s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	2m 41s	the patch passed
		_ Other Tests _
-1 ❌	unit	3m 23s	root in the patch failed.
		29m 45s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4223/7/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#4223
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 4e809a1301d3 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / 805c346
Default Java	AdoptOpenJDK-11.0.10+9
unit	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4223/7/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-root.txt
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4223/7/testReport/
Max. process+thread count	190 (vs. ulimit of 30000)
modules	C: hbase-assembly . U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4223/7/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

[Apache-HBase]

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 10s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+0 🆗	shelldocs	0m 0s	Shelldocs was not available.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+0 🆗	mvndep	0m 20s	Maven dependency ordering for branch
+1 💚	mvninstall	2m 33s	master passed
+1 💚	compile	6m 19s	master passed
+0 🆗	refguide	2m 28s	branch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect.
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 14s	Maven dependency ordering for patch
+1 💚	mvninstall	2m 13s	the patch passed
+1 💚	compile	6m 15s	the patch passed
-0 ⚠️	javac	6m 15s	root generated 7 new + 1462 unchanged - 10 fixed = 1469 total (was 1472)
+1 💚	shellcheck	0m 2s	There were no new shellcheck issues.
+1 💚	whitespace	0m 1s	The patch has no whitespace issues.
+1 💚	xml	0m 3s	The patch has no ill-formed XML file.
+0 🆗	refguide	1m 52s	patch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect.
+1 💚	hadoopcheck	11m 44s	Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1.
		_ Other Tests _
+1 💚	asflicense	0m 21s	The patch does not generate ASF License warnings.
		41m 12s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4223/7/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#4223
Optional Tests	dupname asflicense shellcheck shelldocs javac hadoopcheck xml compile refguide
uname	Linux d8c00a5c2e83 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / 805c346
Default Java	AdoptOpenJDK-1.8.0_282-b08
refguide	https://nightlies.apache.org/hbase/HBase-PreCommit-GitHub-PR/PR-4223/7/yetus-general-check/output/branch-site/book.html
javac	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4223/7/artifact/yetus-general-check/output/diff-compile-javac-root.txt
refguide	https://nightlies.apache.org/hbase/HBase-PreCommit-GitHub-PR/PR-4223/7/yetus-general-check/output/patch-site/book.html
Max. process+thread count	139 (vs. ulimit of 30000)
modules	C: hbase-assembly . U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4223/7/console
versions	git=2.17.1 maven=3.6.3 shellcheck=0.4.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

[Apache-HBase]

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 53s	Docker mode activated.
-0 ⚠️	yetus	0m 2s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+0 🆗	mvndep	0m 23s	Maven dependency ordering for branch
+1 💚	mvninstall	2m 35s	master passed
+1 💚	compile	1m 42s	master passed
+1 💚	shadedjars	4m 15s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	1m 48s	master passed
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 14s	Maven dependency ordering for patch
+1 💚	mvninstall	2m 24s	the patch passed
+1 💚	compile	1m 39s	the patch passed
+1 💚	javac	1m 39s	the patch passed
+1 💚	shadedjars	4m 20s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	1m 57s	the patch passed
		_ Other Tests _
-1 ❌	unit	391m 33s	root in the patch failed.
		417m 2s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4223/7/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#4223
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux c98c6b727dbd 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / 805c346
Default Java	AdoptOpenJDK-1.8.0_282-b08
unit	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4223/7/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-root.txt
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4223/7/testReport/
Max. process+thread count	2436 (vs. ulimit of 30000)
modules	C: hbase-assembly . U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4223/7/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.