HBASE-22690 Deprecate / Remove OfflineMetaRepair in hbase-2+ #380

brfrn169 · 2019-07-14T01:24:10Z

This is for branch-2. I changed OfflineMetaRepair to output a message pointing user to hbck2 operator tools and removed some unnecessary logics in HBaseFsck.

busbey

We can't point folks to unreleased code, so please just reference "hbck2 from hbase-operator-tools" and link to the explanation in the ref guide.

HorizonNet · 2019-07-14T16:33:36Z

hbase-server/src/main/java/org/apache/hadoop/hbase/util/hbck/OfflineMetaRepair.java

- * mentioned.
- *
- * hbase org.apache.hadoop.hbase.util.hbck.OfflineMetaRepair ...
+ * We've deprecated this tool in hbase-2+ because it destroyed the hbase2 meta table.


Could you please move this to the deprecated Javadoc annotation, stating the earliest version it was deprecated and in which version it should be removed?

wchevreuil · 2019-07-15T14:07:47Z

Other than the suggestions from @busbey and @HorizonNet , lgtm. Tested that hbck readonly options still work, and trying to run OMR gives the warning message without doing anything.

saintstack · 2019-07-15T16:37:14Z

Should we squash this PR @brfrn169 given you opened new one sir. Does new one address above comments? Thanks.

Apache-HBase · 2019-07-15T18:58:24Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
0	reexec	220	Docker mode activated.
		_ Prechecks _
+1	hbaseanti	0	Patch does not have any anti-patterns.
+1	@author	0	The patch does not contain any @author tags.
-0	test4tests	0	The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
		_ branch-2 Compile Tests _
+1	mvninstall	260	branch-2 passed
+1	compile	59	branch-2 passed
+1	checkstyle	73	branch-2 passed
+1	shadedjars	234	branch has no errors when building our shaded downstream artifacts.
+1	findbugs	165	branch-2 passed
+1	javadoc	36	branch-2 passed
		_ Patch Compile Tests _
+1	mvninstall	234	the patch passed
+1	compile	53	the patch passed
+1	javac	53	the patch passed
-1	checkstyle	68	hbase-server: The patch generated 4 new + 103 unchanged - 4 fixed = 107 total (was 107)
+1	whitespace	0	The patch has no whitespace issues.
+1	shadedjars	246	patch has no errors when building our shaded downstream artifacts.
+1	hadoopcheck	709	Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2.
+1	findbugs	170	the patch passed
+1	javadoc	36	the patch passed
		_ Other Tests _
-1	unit	19226	hbase-server in the patch failed.
+1	asflicense	30	The patch does not generate ASF License warnings.
		22126

Reason	Tests
Failed junit tests	hadoop.hbase.client.TestRestoreSnapshotFromClientSimple
	hadoop.hbase.client.TestSnapshotDFSTemporaryDirectory
	hadoop.hbase.client.TestFromClientSideWithCoprocessor
	hadoop.hbase.replication.TestReplicationKillSlaveRS
	hadoop.hbase.client.TestAsyncTableAdminApi
	hadoop.hbase.client.TestCloneSnapshotFromClientNormal
	hadoop.hbase.replication.TestMasterReplication
	hadoop.hbase.client.TestSnapshotTemporaryDirectory
	hadoop.hbase.client.TestFromClientSide
	hadoop.hbase.client.TestSnapshotTemporaryDirectoryWithRegionReplicas
	hadoop.hbase.client.replication.TestReplicationAdminWithClusters

Subsystem	Report/Notes
Docker	Client=18.09.7 Server=18.09.7 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-380/1/artifact/out/Dockerfile
GITHUB PR	#380
Optional Tests	dupname asflicense javac javadoc unit findbugs shadedjars hadoopcheck hbaseanti checkstyle compile
uname	Linux bb5bc6b2b717 4.4.0-139-generic #165-Ubuntu SMP Wed Oct 24 10:58:50 UTC 2018 x86_64 GNU/Linux
Build tool	maven
Personality	/testptch/patchprocess/precommit/personality/provided.sh
git revision	branch-2 / `b656944`
maven	version: Apache Maven 3.5.4 (1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z)
Default Java	1.8.0_181
findbugs	v3.1.11
checkstyle	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-380/1/artifact/out/diff-checkstyle-hbase-server.txt
unit	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-380/1/artifact/out/patch-unit-hbase-server.txt
Test Results	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-380/1/testReport/
Max. process+thread count	4575 (vs. ulimit of 10000)
modules	C: hbase-server U: hbase-server
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-380/1/console
Powered by	Apache Yetus 0.9.0 http://yetus.apache.org

This message was automatically generated.

brfrn169 · 2019-07-16T01:41:36Z

Thank you for reviewing guys.

I modified the patch for the review. Could you please review it? @busbey @HorizonNet

brfrn169 · 2019-07-16T01:48:09Z

Should we squash this PR @brfrn169 given you opened new one sir. Does new one address above comments? Thanks.

@saintstack The new one means this PR, right?
#381

That is for the master branch. In that PR, I removed OfflineMetaRepair completely whereas in this PR (for branch-2), I changed OfflineMetaRepair to output a message pointing user to hbck2 operator tools.

My intention is to deprecate in 2.x and to remove in 3.0.

Apache-HBase · 2019-07-16T15:13:50Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
0	reexec	244	Docker mode activated.
		_ Prechecks _
+1	hbaseanti	0	Patch does not have any anti-patterns.
+1	@author	0	The patch does not contain any @author tags.
-0	test4tests	0	The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
		_ branch-2 Compile Tests _
+1	mvninstall	455	branch-2 passed
+1	compile	68	branch-2 passed
+1	checkstyle	94	branch-2 passed
+1	shadedjars	320	branch has no errors when building our shaded downstream artifacts.
+1	findbugs	212	branch-2 passed
+1	javadoc	42	branch-2 passed
		_ Patch Compile Tests _
+1	mvninstall	297	the patch passed
+1	compile	68	the patch passed
+1	javac	68	the patch passed
-1	checkstyle	82	hbase-server: The patch generated 4 new + 103 unchanged - 4 fixed = 107 total (was 107)
+1	whitespace	0	The patch has no whitespace issues.
+1	shadedjars	311	patch has no errors when building our shaded downstream artifacts.
+1	hadoopcheck	914	Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2.
+1	findbugs	191	the patch passed
+1	javadoc	40	the patch passed
		_ Other Tests _
-1	unit	16102	hbase-server in the patch failed.
+1	asflicense	42	The patch does not generate ASF License warnings.
		19859

Reason	Tests
Failed junit tests	hadoop.hbase.master.procedure.TestSCPWithReplicasWithoutZKCoordinated

Subsystem	Report/Notes
Docker	Client=18.09.7 Server=18.09.7 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-380/2/artifact/out/Dockerfile
GITHUB PR	#380
Optional Tests	dupname asflicense javac javadoc unit findbugs shadedjars hadoopcheck hbaseanti checkstyle compile
uname	Linux cf3abb244e15 4.4.0-143-generic #169-Ubuntu SMP Thu Feb 7 07:56:38 UTC 2019 x86_64 GNU/Linux
Build tool	maven
Personality	/testptch/patchprocess/precommit/personality/provided.sh
git revision	branch-2 / `76c080b`
maven	version: Apache Maven 3.5.4 (1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z)
Default Java	1.8.0_181
findbugs	v3.1.11
checkstyle	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-380/2/artifact/out/diff-checkstyle-hbase-server.txt
unit	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-380/2/artifact/out/patch-unit-hbase-server.txt
Test Results	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-380/2/testReport/
Max. process+thread count	4887 (vs. ulimit of 10000)
modules	C: hbase-server U: hbase-server
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-380/2/console
Powered by	Apache Yetus 0.9.0 http://yetus.apache.org

This message was automatically generated.

brfrn169 · 2019-07-16T23:04:03Z

I fixed some checkstyle warnings.

Apache-HBase · 2019-07-17T10:56:06Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
0	reexec	34	Docker mode activated.
		_ Prechecks _
+1	hbaseanti	0	Patch does not have any anti-patterns.
+1	@author	0	The patch does not contain any @author tags.
-0	test4tests	0	The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
		_ branch-2 Compile Tests _
+1	mvninstall	238	branch-2 passed
+1	compile	52	branch-2 passed
+1	checkstyle	70	branch-2 passed
+1	shadedjars	240	branch has no errors when building our shaded downstream artifacts.
+1	findbugs	151	branch-2 passed
+1	javadoc	34	branch-2 passed
		_ Patch Compile Tests _
+1	mvninstall	231	the patch passed
+1	compile	51	the patch passed
+1	javac	51	the patch passed
+1	checkstyle	67	hbase-server: The patch generated 0 new + 101 unchanged - 6 fixed = 101 total (was 107)
+1	whitespace	0	The patch has no whitespace issues.
+1	shadedjars	241	patch has no errors when building our shaded downstream artifacts.
+1	hadoopcheck	715	Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2.
+1	findbugs	142	the patch passed
+1	javadoc	32	the patch passed
		_ Other Tests _
+1	unit	7803	hbase-server in the patch passed.
+1	asflicense	27	The patch does not generate ASF License warnings.
		10444

Subsystem	Report/Notes
Docker	Client=18.09.7 Server=18.09.7 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-380/3/artifact/out/Dockerfile
GITHUB PR	#380
Optional Tests	dupname asflicense javac javadoc unit findbugs shadedjars hadoopcheck hbaseanti checkstyle compile
uname	Linux 9cc36205b412 4.4.0-139-generic #165-Ubuntu SMP Wed Oct 24 10:58:50 UTC 2018 x86_64 GNU/Linux
Build tool	maven
Personality	/testptch/patchprocess/precommit/personality/provided.sh
git revision	branch-2 / `6c834b9`
maven	version: Apache Maven 3.5.4 (1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z)
Default Java	1.8.0_181
findbugs	v3.1.11
Test Results	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-380/3/testReport/
Max. process+thread count	4637 (vs. ulimit of 10000)
modules	C: hbase-server U: hbase-server
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-380/3/console
Powered by	Apache Yetus 0.9.0 http://yetus.apache.org

This message was automatically generated.

brfrn169 · 2019-07-19T06:34:21Z

Ping @busbey @HorizonNet @saintstack

saintstack

Looks excellent to me

Apache-HBase · 2019-07-20T23:19:08Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
0	reexec	85	Docker mode activated.
		_ Prechecks _
+1	hbaseanti	0	Patch does not have any anti-patterns.
+1	@author	0	The patch does not contain any @author tags.
-0	test4tests	0	The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
		_ branch-2 Compile Tests _
+1	mvninstall	267	branch-2 passed
+1	compile	58	branch-2 passed
+1	checkstyle	89	branch-2 passed
+1	shadedjars	285	branch has no errors when building our shaded downstream artifacts.
+1	findbugs	216	branch-2 passed
+1	javadoc	87	branch-2 passed
		_ Patch Compile Tests _
+1	mvninstall	278	the patch passed
+1	compile	61	the patch passed
+1	javac	61	the patch passed
+1	checkstyle	84	hbase-server: The patch generated 0 new + 101 unchanged - 6 fixed = 101 total (was 107)
+1	whitespace	0	The patch has no whitespace issues.
+1	shadedjars	266	patch has no errors when building our shaded downstream artifacts.
+1	hadoopcheck	817	Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2.
+1	findbugs	165	the patch passed
+1	javadoc	35	the patch passed
		_ Other Tests _
-1	unit	15719	hbase-server in the patch failed.
+1	asflicense	32	The patch does not generate ASF License warnings.
		18914

Reason	Tests
Failed junit tests	hadoop.hbase.client.TestFromClientSide
	hadoop.hbase.quotas.TestClusterScopeQuotaThrottle

Subsystem	Report/Notes
Docker	Client=18.09.7 Server=18.09.7 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-380/4/artifact/out/Dockerfile
GITHUB PR	#380
Optional Tests	dupname asflicense javac javadoc unit findbugs shadedjars hadoopcheck hbaseanti checkstyle compile
uname	Linux b508e2f9e409 4.15.0-52-generic #56-Ubuntu SMP Tue Jun 4 22:49:08 UTC 2019 x86_64 GNU/Linux
Build tool	maven
Personality	/testptch/patchprocess/precommit/personality/provided.sh
git revision	branch-2 / `6fb03e6`
maven	version: Apache Maven 3.5.4 (1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z)
Default Java	1.8.0_181
findbugs	v3.1.11
unit	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-380/4/artifact/out/patch-unit-hbase-server.txt
Test Results	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-380/4/testReport/
Max. process+thread count	4626 (vs. ulimit of 10000)
modules	C: hbase-server U: hbase-server
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-380/4/console
Powered by	Apache Yetus 0.9.0 http://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2019-07-22T12:27:02Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
0	reexec	250	Docker mode activated.
		_ Prechecks _
+1	dupname	0	No case conflicting files found.
+1	hbaseanti	0	Patch does not have any anti-patterns.
+1	@author	0	The patch does not contain any @author tags.
-0	test4tests	0	The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
		_ branch-2 Compile Tests _
+1	mvninstall	404	branch-2 passed
+1	compile	77	branch-2 passed
+1	checkstyle	106	branch-2 passed
+1	shadedjars	331	branch has no errors when building our shaded downstream artifacts.
+1	javadoc	45	branch-2 passed
0	spotbugs	246	Used deprecated FindBugs config; considering switching to SpotBugs.
+1	findbugs	244	branch-2 passed
		_ Patch Compile Tests _
+1	mvninstall	327	the patch passed
+1	compile	64	the patch passed
+1	javac	64	the patch passed
+1	checkstyle	91	hbase-server: The patch generated 0 new + 101 unchanged - 6 fixed = 101 total (was 107)
+1	whitespace	0	The patch has no whitespace issues.
+1	shadedjars	289	patch has no errors when building our shaded downstream artifacts.
+1	hadoopcheck	1022	Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2.
+1	javadoc	40	the patch passed
+1	findbugs	232	the patch passed
		_ Other Tests _
-1	unit	18925	hbase-server in the patch failed.
+1	asflicense	31	The patch does not generate ASF License warnings.
		22889

Reason	Tests
Failed junit tests	hadoop.hbase.client.TestAsyncSnapshotAdminApi
	hadoop.hbase.client.TestFromClientSideWithCoprocessor
	hadoop.hbase.client.TestSnapshotTemporaryDirectoryWithRegionReplicas
	hadoop.hbase.client.TestFromClientSide

Subsystem	Report/Notes
Docker	Client=18.09.8 Server=18.09.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-380/5/artifact/out/Dockerfile
GITHUB PR	#380
Optional Tests	dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile
uname	Linux 78b37afb3f2b 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 GNU/Linux
Build tool	maven
Personality	/home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-380/out/precommit/personality/provided.sh
git revision	branch-2 / `c54085a`
Default Java	1.8.0_181
unit	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-380/5/artifact/out/patch-unit-hbase-server.txt
Test Results	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-380/5/testReport/
Max. process+thread count	4433 (vs. ulimit of 10000)
modules	C: hbase-server U: hbase-server
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-380/5/console
versions	git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11
Powered by	Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

brfrn169 · 2019-07-22T13:48:52Z

Thank you for reviewing @saintstack. I'll commit this.

Apache-HBase · 2019-07-22T14:55:16Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
0	reexec	0	Docker mode activated.
-1	patch	13	#380 does not apply to branch-2. Rebase required? Wrong Branch? See https://yetus.apache.org/documentation/in-progress/precommit-patchnames for help.

Subsystem	Report/Notes
GITHUB PR	#380
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-380/6/console
versions	git=2.7.4
Powered by	Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2019-07-22T17:17:53Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
0	reexec	0	Docker mode activated.
-1	patch	12	#380 does not apply to branch-2. Rebase required? Wrong Branch? See https://yetus.apache.org/documentation/in-progress/precommit-patchnames for help.

Subsystem	Report/Notes
GITHUB PR	#380
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-380/1/console
versions	git=2.7.4
Powered by	Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2019-07-22T22:40:51Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
0	reexec	0	Docker mode activated.
-1	patch	14	#380 does not apply to branch-2. Rebase required? Wrong Branch? See https://yetus.apache.org/documentation/in-progress/precommit-patchnames for help.

Subsystem	Report/Notes
GITHUB PR	#380
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-380/2/console
versions	git=2.7.4
Powered by	Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

HBASE-22690 Deprecate / Remove OfflineMetaRepair in hbase-2+

cb9de4c

brfrn169 requested review from saintstack and Apache9 July 14, 2019 01:24

busbey requested changes Jul 14, 2019

View reviewed changes

HorizonNet requested changes Jul 14, 2019

View reviewed changes

HBASE-22690 Deprecate / Remove OfflineMetaRepair in hbase-2+

8d1ec71

HBASE-22690 Deprecate / Remove OfflineMetaRepair in hbase-2+

3b51434

brfrn169 requested review from HorizonNet and busbey July 18, 2019 04:58

saintstack approved these changes Jul 19, 2019

View reviewed changes

brfrn169 closed this Jul 23, 2019

brfrn169 self-assigned this Jul 18, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HBASE-22690 Deprecate / Remove OfflineMetaRepair in hbase-2+ #380

HBASE-22690 Deprecate / Remove OfflineMetaRepair in hbase-2+ #380

brfrn169 commented Jul 14, 2019 •

edited

Loading

busbey left a comment

HorizonNet Jul 14, 2019

wchevreuil commented Jul 15, 2019

saintstack commented Jul 15, 2019

Apache-HBase commented Jul 15, 2019

brfrn169 commented Jul 16, 2019

brfrn169 commented Jul 16, 2019

Apache-HBase commented Jul 16, 2019

brfrn169 commented Jul 16, 2019

Apache-HBase commented Jul 17, 2019

brfrn169 commented Jul 19, 2019

saintstack left a comment

Apache-HBase commented Jul 20, 2019

Apache-HBase commented Jul 22, 2019

brfrn169 commented Jul 22, 2019

Apache-HBase commented Jul 22, 2019

Apache-HBase commented Jul 22, 2019

Apache-HBase commented Jul 22, 2019

HBASE-22690 Deprecate / Remove OfflineMetaRepair in hbase-2+ #380

HBASE-22690 Deprecate / Remove OfflineMetaRepair in hbase-2+ #380

Conversation

brfrn169 commented Jul 14, 2019 • edited Loading

busbey left a comment

Choose a reason for hiding this comment

HorizonNet Jul 14, 2019

Choose a reason for hiding this comment

wchevreuil commented Jul 15, 2019

saintstack commented Jul 15, 2019

Apache-HBase commented Jul 15, 2019

brfrn169 commented Jul 16, 2019

brfrn169 commented Jul 16, 2019

Apache-HBase commented Jul 16, 2019

brfrn169 commented Jul 16, 2019

Apache-HBase commented Jul 17, 2019

brfrn169 commented Jul 19, 2019

saintstack left a comment

Choose a reason for hiding this comment

Apache-HBase commented Jul 20, 2019

Apache-HBase commented Jul 22, 2019

brfrn169 commented Jul 22, 2019

Apache-HBase commented Jul 22, 2019

Apache-HBase commented Jul 22, 2019

Apache-HBase commented Jul 22, 2019

brfrn169 commented Jul 14, 2019 •

edited

Loading