PHOENIX-6222 Bump default HBase version to 2.2.6 and 2.3.3

[stoty]

No description provided.

[stoty]

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 31s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s	The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
		_ master Compile Tests _
+1 💚	mvninstall	12m 43s	master passed
+1 💚	compile	1m 21s	master passed
+1 💚	javadoc	1m 5s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	7m 29s	the patch passed
+1 💚	compile	1m 16s	the patch passed
+1 💚	javac	1m 16s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	xml	0m 1s	The patch has no ill-formed XML file.
+1 💚	javadoc	1m 5s	the patch passed
		_ Other Tests _
-1 ❌	unit	96m 37s	root in the patch failed.
+1 💚	asflicense	0m 30s	The patch does not generate ASF License warnings.
		125m 0s

Subsystem	Report/Notes
Docker	ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-964/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#964
Optional Tests	dupname asflicense javac javadoc unit xml compile
uname	Linux db58e27ec126 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev/phoenix-personality.sh
git revision	master / 6cc9d50
Default Java	Private Build-1.8.0_242-8u242-b08-0ubuntu3~16.04-b08
unit	https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-964/1/artifact/yetus-general-check/output/patch-unit-root.txt
Test Results	https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-964/1/testReport/
Max. process+thread count	7105 (vs. ulimit of 30000)
modules	C: . U: .
Console output	https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-964/1/console
versions	git=2.7.4 maven=3.3.9
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

[joshelser]

Corresponding Jira issue for this one? What are we getting with the newer versions? (i.e. do we explicitly require fixes from them)

[stoty]

Corresponding Jira issue for this one?

Sorry, cut-and-paste error. I've updated the commit and PR message.

What are we getting with the newer versions? (i.e. do we explicitly require fixes from them)

No, I haven't identified specific fixes for known problems.
IMO we should generally try to use the latest HBase patch releases for the usual reasons.

I made the PR now, because we are nearing 5.1, and I want to see the test results for the current HBase versions.

• Do they fix existing flakey tests ( like the SplitSystemCatalog setup failures we see with 2.3 )
• Do they introduce any new problems that we want to follow up on before release ?

[stoty]

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 29s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s	The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
		_ master Compile Tests _
+1 💚	mvninstall	12m 55s	master passed
+1 💚	compile	1m 19s	master passed
+1 💚	javadoc	1m 5s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	7m 26s	the patch passed
+1 💚	compile	1m 18s	the patch passed
+1 💚	javac	1m 18s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	xml	0m 1s	The patch has no ill-formed XML file.
+1 💚	javadoc	1m 4s	the patch passed
		_ Other Tests _
-1 ❌	unit	97m 8s	root in the patch failed.
+1 💚	asflicense	0m 32s	The patch does not generate ASF License warnings.
		125m 40s

Subsystem	Report/Notes
Docker	ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-964/2/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#964
Optional Tests	dupname asflicense javac javadoc unit xml compile
uname	Linux dc9b05500180 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev/phoenix-personality.sh
git revision	master / 6cc9d50
Default Java	Private Build-1.8.0_242-8u242-b08-0ubuntu3~16.04-b08
unit	https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-964/2/artifact/yetus-general-check/output/patch-unit-root.txt
Test Results	https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-964/2/testReport/
Max. process+thread count	6862 (vs. ulimit of 30000)
modules	C: . U: .
Console output	https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-964/2/console
versions	git=2.7.4 maven=3.3.9
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

[virajjasani]

+1 (non-binding) for using latest patch releases.
However, as for the tests, more often we don't get to see full test runs because tests like GlobalMutableNonTxIndexWithLazyPostBatchWriteIT, IndexUsageIT crashes VM and then we see multiple skipped tests. For instance, test runs for both builds of this PR so far have more than 100 tests skipped. As of today, do we have any workaround for VM crash or do we know of all tests that usually cause such trouble? Do we generally follow any recommendation like running all tests locally as part of HBase version upgrade to be more confident that all tests are passing (including flaky ones and the ones that are skipped in precommit builds)?

I am just thinking from 5.1 and 4.16 release stability viewpoint that at least if we get to see results of all tests, upgrading HBase versions might not be that much of a big deal and we could see all test results. Thought?

[stoty]

Thanks @virajjasani.

The precommit tests are irrelevant for this patch, as those are run with 2.1.9, which is the almost certainly the final 2.1 release, and hasn't been changed.

I consider HBase patch release upgrades to be generally low risk, unless there is a breaking API change, which usually manifests at compile time.

ATM precommit runs the full test suite, it only skips modules if there are no changes in the module, nor in its dependencies.

Flakey tests are a huge problem, and we really should take them more seriously as a project.

I haven't had a chance to look at recent flakeys, and I cannot promise that I will in near future, but I REALLY would like to have the tests stabilized by the time we release 5.1 .

[virajjasani]

My bad, I forgot for a moment that 2.1 is the default profile. Still I believe it's worth getting this patch in, at least multibranch master will start running tests for 2.3 profile with 2.3.3 release line and we can keep an eye for some days.

I REALLY would like to have the tests stabilized by the time we release 5.1 .

I agree, will be happy to help on this.
Thanks

[virajjasani]

@stoty we should also bump provided dependencies of phoenix-hbase-compat-x.y.z's hbase-client, hbase-common, hbase-server?
e.g bump hbase-server to 2.3.3 here https://github.com/apache/phoenix/blob/master/phoenix-hbase-compat-2.3.0/pom.xml#L70-L71
Thought?

[stoty]

@stoty we should also bump provided dependencies of phoenix-hbase-compat-x.y.z's hbase-client, hbase-common, hbase-server?
e.g bump hbase-server to 2.3.3 here https://github.com/apache/phoenix/blob/master/phoenix-hbase-compat-2.3.0/pom.xml#L70-L71
Thought?

No.
Those are intentionally set to the lowest release of the HBase API variant that we are targeting-
While updating it wouldn't break the code, it is not necessary, and would cause confusion.
If there was an API change that affected us, we'd have to add a new a compatibility module instead.

The provided HBase dependency versions here will get overridden with the proper versions from the phoenix parent module.

[virajjasani]

If there was an API change that affected us, we'd have to add a new a compatibility module instead.

Got it, makes sense, we are good then. Thank you @stoty !

[virajjasani]

The provided HBase dependency versions here will get overridden with the proper versions from the phoenix parent module.

That's correct, I confirmed that a couple of days back by running dependency:tree.