Skip to content
Permalink
Browse files
HBASE-23112 [hbase-operator-tools] fixMeta in hbck2 is porcelain, in …
…hbck1 it was plumbing; fix (#40)

Removed 'porcelain vs plubming' analogy. Its fine when we are talking
amongst ourselves but it confuses describing hbck2, a jumble of tools
doing various things.

Signed-off-by: Daisuke Kobayashi <poleon.kd@gmail.com>
  • Loading branch information
saintstack committed Oct 9, 2019
1 parent e4f73fb commit d5a843f5fe50fba6c11d7c95012726689bed583d
Showing 2 changed files with 59 additions and 33 deletions.
@@ -20,32 +20,33 @@

_HBCK2_ is the repair tool for Apache HBase clusters.

Problems in operation are bugs.
The need for an _HBCK2_ fix is meant as workaround until the bug is fixed and
deployed in a new hbase version.
Problems in operation are bugs. The need for an _HBCK2_ fix
is meant as workaround until the bug is fixed and deployed
in a new hbase version.

## _HBCK2_ vs _hbck1_
HBCK2 is the successor to [hbck](https://hbase.apache.org/book.html#hbck.in.depth),
the repair tool that shipped with _hbase-1.x_ (A.K.A _hbck1_). Use _HBCK2_ in place of
_hbck1_ making repairs against hbase-2.x clusters. _hbck1_ should not be run against an
hbase-2.x install. It may do damage. While _hbck1_ is still bundled inside hbase-2.x
-- to minimize surprise -- it is deprecated, to be removed in _hbase-3.x_. It's
-- to minimize surprise -- it is deprecated, to be removed in _hbase-3.x_. Its
write-facility (`-fix`) has been removed. It can report on the state of an hbase-2.x
cluster but its assessments will be inaccurate since it does not understand the internal
workings of an hbase-2.x.

_HBCK2_ does not work the way _hbck1_ used to. See the next section for how.
_HBCK2_ does not work the way _hbck1_ used to, even for the case where commands are
similarly named across the two versions. See the next section for how the tools
differ.

## Philosophy
_HBCK2_ performs a single discrete 'fix' task each time it is run. It does not presume
_HBCK2_ performs a single, discrete task each time it is run. It does not presume
a tool can analyze all about the running cluster and then repair 'all problems' found as
_hbck1_ used suggest. _HBCK2_ is a tool that is more in the vein of
[`plumbing` than `porcelain`](https://git-scm.com/book/en/v2/Git-Internals-Plumbing-and-Porcelain).
_hbck1_ used suggest.

The _HBCK2_ tool makes fixes. For listings of inconsistencies or blockages in the running cluster,
_HBCK2_ is for fixes. For listings of inconsistencies or blockages in the running cluster,
you go elsewhere, to the logs and UI of the running cluster Master. Once an issue has been identified,
you use the _HBCK2_ tool to ask the Master to effect fixes or to skip-over bad state. Asking the
Master for problems and to make fixes rather than try and effect the repair locally in a fix-it
Master to make the fixes rather than try and effect the repair locally in a fix-it
tool's context is another important difference between _HBCK2_ and _hbck1_. More on how this
interactive fix-it process works and on _HBCK2_ workings can be found in sections that follow.

@@ -102,11 +103,10 @@ Command:
addFsRegionsMissingInMeta <NAMESPACE|NAMESPACE:TABLENAME>...
Options:
-d,--force_disable aborts fix for table if disable fails.
To be used when some regions may be missing from hbase:meta
but their directories are present in HDFS. This is a 'lighter'
version of 'OfflineMetaRepair' tool commonly used for similar
issues in hbase-1.x. This command needs hbase:meta to be online.
For each table name passed as parameter, it performs a diff
To be used when regions missing from hbase:meta but directories
are present still in HDFS. Can happen if user has run _hbck1_
'OfflineMetaRepair' against an hbase-2.x cluster. Needs hbase:meta
to be online. For each table name passed as parameter, performs diff
between regions available in hbase:meta and region dirs on HDFS.
Then for dirs with no hbase:meta matches, it reads the 'regioninfo'
metadata file and re-creates given region in hbase:meta. Regions are
@@ -115,15 +115,16 @@ Command:
regions online, run the HBCK2 'assigns'command printed when this
command-run completes.
NOTE: If using hbase releases older than 2.3.0, a rolling restart of
HMasters is needed prior to executing the provided 'assigns' command.
HMasters is needed prior to executing the set of 'assigns' output.
An example adding missing regions for tables 'tbl_1' in the default
namespace, 'tbl_2' in namespace 'n1' and for all tables from
namespace 'n2':
$ HBCK2 addFsRegionsMissingInMeta default:tbl_1 n1:tbl_2 n2
Returns HBCK2 an 'assigns' command with all re-inserted regions.
SEE ALSO: reportMissingRegionsInMeta
SEE ALSO: fixMeta
assigns [OPTIONS] <ENCODED_REGIONNAME>...
assigns [OPTIONS] <ENCODED_REGIONNAME>...
Options:
-o,--override override ownership by another procedure
A 'raw' assign that can be used even during Master initialization (if
@@ -158,8 +159,19 @@ Command:
only! Modified regions need to be reopened to pick-up changes.
fixMeta
Do a server-side fixing of bad or inconsistent state in hbase:meta.
Repairs 'holes' and 'overlaps' in hbase:meta.
Do a server-side fix of bad or inconsistent state in hbase:meta.
Available in hbase 2.2.1/2.1.6 or newer versions. Master UI has
matching, new 'HBCK Report' tab that dumps reports generated by
most recent run of _catalogjanitor_ and a new 'HBCK Chore'. It
is critical that hbase:meta first be made healthy before making
any other repairs. Fixes 'holes', 'overlaps', etc., creating
(empty) region directories in HDFS to match regions added to
hbase:meta. Command is NOT the same as the old _hbck1_ command
named similarily. Works against the reports generated by the last
catalog_janitor and hbck chore runs. If nothing to fix, run is a
noop. Otherwise, if 'HBCK Report' UI reports problems, a run of
fixMeta will clear up hbase:meta issues. See 'HBase HBCK' UI
for how to generate new report.
SEE ALSO: reportMissingRegionsInMeta
replication [OPTIONS] [<TABLENAME>...]
@@ -170,11 +182,12 @@ Command:
purge if '--fix'.
reportMissingRegionsInMeta <NAMESPACE|NAMESPACE:TABLENAME>...
To be used when some regions may be missing from hbase:meta
but their directories are present in HDFS. This is a checking only
To be used when regions missing from hbase:meta but directories
are present still in HDFS. Can happen if user has run _hbck1_
'OfflineMetaRepair' against an hbase-2.x cluster. This is a CHECK only
method, designed for reporting purposes and doesn't perform any
fixes, providing a view of which regions (if any) would get re-added
to meta, grouped by respective table/namespace. To effectively
to hbase:meta, grouped by respective table/namespace. To effectively
re-add regions in meta, run addFsRegionsMissingInMeta.
This command needs hbase:meta to be online. For each namespace/table
passed as parameter, it performs a diff between regions available in
@@ -399,11 +399,10 @@ private static void usageAddFsRegionsMissingInMeta(PrintWriter writer) {
+ "NAMESPACE:TABLENAME>...");
writer.println(" Options:");
writer.println(" -d,--force_disable aborts fix for table if disable fails.");
writer.println(" To be used when some regions may be missing from hbase:meta");
writer.println(" but their directories are present in HDFS. This is a 'lighter'");
writer.println(" version of 'OfflineMetaRepair' tool commonly used for similar");
writer.println(" issues in hbase-1.x. This command needs hbase:meta to be online.");
writer.println(" For each table name passed as parameter, it performs a diff");
writer.println(" To be used when regions missing from hbase:meta but directories");
writer.println(" are present still in HDFS. Can happen if user has run _hbck1_");
writer.println(" 'OfflineMetaRepair' against an hbase-2.x cluster. Needs hbase:meta");
writer.println(" to be online. For each table name passed as parameter, performs diff");
writer.println(" between regions available in hbase:meta and region dirs on HDFS.");
writer.println(" Then for dirs with no hbase:meta matches, it reads the 'regioninfo'");
writer.println(" metadata file and re-creates given region in hbase:meta. Regions are");
@@ -412,14 +411,15 @@ private static void usageAddFsRegionsMissingInMeta(PrintWriter writer) {
writer.println(" regions online, run the HBCK2 'assigns'command printed when this");
writer.println(" command-run completes.");
writer.println(" NOTE: If using hbase releases older than 2.3.0, a rolling restart of");
writer.println(" HMasters is needed prior to executing the provided 'assigns' command.");
writer.println(" HMasters is needed prior to executing the set of 'assigns' output.");
writer.println(" An example adding missing regions for tables 'tbl_1' in the default");
writer.println(" namespace, 'tbl_2' in namespace 'n1' and for all tables from");
writer.println(" namespace 'n2':");
writer.println(" $ HBCK2 " + ADD_MISSING_REGIONS_IN_META_FOR_TABLES +
" default:tbl_1 n1:tbl_2 n2");
writer.println(" Returns HBCK2 an 'assigns' command with all re-inserted regions.");
writer.println(" SEE ALSO: " + REPORT_MISSING_REGIONS_IN_META);
writer.println(" SEE ALSO: " + FIX_META);
}

private static void usageAssigns(PrintWriter writer) {
@@ -464,8 +464,20 @@ private static void usageFilesystem(PrintWriter writer) {

private static void usageFixMeta(PrintWriter writer) {
writer.println(" " + FIX_META);
writer.println(" Do a server-side fixing of bad or inconsistent state in hbase:meta.");
writer.println(" Repairs 'holes' and 'overlaps' in hbase:meta.");
writer.println(" Do a server-side fix of bad or inconsistent state in hbase:meta.");
writer.println(" Available in hbase 2.2.1/2.1.6 or newer versions. Master UI has");
writer.println(" matching, new 'HBCK Report' tab that dumps reports generated by");
writer.println(" most recent run of _catalogjanitor_ and a new 'HBCK Chore'. It");
writer.println(" is critical that hbase:meta first be made healthy before making");
writer.println(" any other repairs. Fixes 'holes', 'overlaps', etc., creating");
writer.println(" (empty) region directories in HDFS to match regions added to");
writer.println(" hbase:meta. Command is NOT the same as the old _hbck1_ command");
writer.println(" named similarily. Works against the reports generated by the last");
writer.println(" catalog_janitor and hbck chore runs. If nothing to fix, run is a");
writer.println(" noop. Otherwise, if 'HBCK Report' UI reports problems, a run of");
writer.println(" " + FIX_META +
" will clear up hbase:meta issues. See 'HBase HBCK' UI");
writer.println(" for how to generate new report.");
writer.println(" SEE ALSO: " + REPORT_MISSING_REGIONS_IN_META);
}

@@ -481,11 +493,12 @@ private static void usageReplication(PrintWriter writer) {
private static void usageReportMissingRegionsInMeta(PrintWriter writer) {
writer.println(" " + REPORT_MISSING_REGIONS_IN_META + " <NAMESPACE|"
+ "NAMESPACE:TABLENAME>...");
writer.println(" To be used when some regions may be missing from hbase:meta");
writer.println(" but their directories are present in HDFS. This is a checking only");
writer.println(" To be used when regions missing from hbase:meta but directories");
writer.println(" are present still in HDFS. Can happen if user has run _hbck1_");
writer.println(" 'OfflineMetaRepair' against an hbase-2.x cluster. This is a CHECK only");
writer.println(" method, designed for reporting purposes and doesn't perform any");
writer.println(" fixes, providing a view of which regions (if any) would get re-added");
writer.println(" to meta, grouped by respective table/namespace. To effectively");
writer.println(" to hbase:meta, grouped by respective table/namespace. To effectively");
writer.println(" re-add regions in meta, run " + ADD_MISSING_REGIONS_IN_META_FOR_TABLES +
".");
writer.println(" This command needs hbase:meta to be online. For each namespace/table");

0 comments on commit d5a843f

Please sign in to comment.