HBASE-23112 [hbase-operator-tools] fixMeta in hbck2 is porcelain, in … #40
Conversation
|
Refer to this link for build results (access rights to CI server needed): |
| fixMeta | ||
| Do a server-side fixing of bad or inconsistent state in hbase:meta. | ||
| Repairs 'holes' and 'overlaps' in hbase:meta. | ||
| Critical that hbase:meta is first made healthy before making any | ||
| other repairs. Fix 'holes', 'overlaps', etc. Is NOT same as the | ||
| _hbck1_ command named similarily. | ||
| SEE ALSO: reportMissingRegionsInMeta |
There was a problem hiding this comment.
Can we also describe how to recognize the issue which requires to involve this command?
There was a problem hiding this comment.
We can, at least in newer hbases where the holes and overlaps are listed in the UI hbck report. Let me add a note.
There was a problem hiding this comment.
Hmmm... This usage note needed a good bit more. See what you think of the latest @daisukebe ? Thanks boss.
There was a problem hiding this comment.
Thanks Stack! Reviewed the latest one and I'm still wondering if the Master UI is the only one way to recognize the inconsistency. Is there an equivalent tool developed in the command line?
Fixes 'holes', 'overlaps', etc
Does it make sense to you to change the sentence to Fixes 'holes', 'overlaps' in HDFS, etc?
There was a problem hiding this comment.
There is no reporting apart from Master UI in hbase2. No equivalent tool. We'd have to write one for those older releases. It would be different though because the report would not be generated by the catalogjanitor but by a tool reading hbase:meta remote running the code we recently added to catalogjanitor.
So, the fixMeta fixes holes and overlaps in meta. It does not fix holes and overlaps in HDFS -- intentionally. Idea is you'd fix meta then if any orphan data in HDFS, you'd ask HBase to adopt it using the bulk load tool to load it up (The filesystem command would note the orphan data/regions).
Shout if I should add more.
There was a problem hiding this comment.
You are right in that it creates an empty region to match the hole or overlap found in hbase:meta.
What it does not do is look around in hdfs to try and match hdfs content w/ the hole that is being plugged or moving data around in hdfs to fill the overlap.
I think talking of HDFS side-effects when describing fixMeta will give the wrong impression, that we will be doing reconcile w/ HDFS content which we are explicitly not doing in this command.
Not sure how better to say this boss.
There was a problem hiding this comment.
Ah ohk. To me, it looks the fixHoles and fixOverlaps methods are doing what -fixHdfsHoles and -fixHdfsOverlaps used to do, respectively, in hbck1, hence I supposed it'd be better to document it that fixMeta touches and modifies HDFS. However, you are expecting more than that when mentioning Fixes 'holes', 'overlaps' in HDFS, etc, correct?
There was a problem hiding this comment.
Let me add a note that empty dirs are created in HDFS. Makes sense.
There was a problem hiding this comment.
Here is what I changed it to in coming patch...
" Do a server-side fix of bad or inconsistent state in hbase:meta.
Available in hbase 2.2.1/2.1.6 or newer versions. Master UI has
matching, new 'HBCK Report' tab that dumps reports generated by
most recent run of catalogjanitor and a new 'HBCK Chore'. It
is critical that hbase:meta first be made healthy before making
any other repairs. Fixes 'holes', 'overlaps', etc., creating
(empty) region directories in HDFS to match regions added to
hbase:meta. Command is NOT the same as the old hbck1 command
named similarily. Works against the reports generated by the last
catalog_janitor and hbck chore runs. If nothing to fix, run is a
noop. Otherwise, if 'HBCK Report' UI reports problems, a run of
fixMeta will clear up hbase:meta issues. See 'HBase HBCK' UI
for how to generate new report.
SEE ALSO: reportMissingRegionsInMeta"
There was a problem hiding this comment.
Thanks for making the change! Looks more user friendly. +1 to the latest patch.
saintstack
force-pushed
the
HBASE-23112
branch
from
3544169 to
9a77c8c
Compare
|
Refer to this link for build results (access rights to CI server needed): |
…hbck1 it was plumbing; fix Removed 'porcelain vs plubming' analogy. Its fine when we are talking amongst ourselves but it confuses describing hbck2, a jumble of tools doing various things.
saintstack
force-pushed
the
HBASE-23112
branch
from
9a77c8c to
eedf87a
Compare
|
Refer to this link for build results (access rights to CI server needed): |
|
Thanks for the reivews @daisukebe |
3 participants
…hbck1 it was plumbing; fix
Removed 'porcelain vs plubming' analogy. Its fine when we are talking
amongst ourselves but it confuses describing hbck2, a jumble of tools
doing various things.