Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HBASE-23315 Miscellaneous HBCK Report page cleanup #847

Closed
wants to merge 1 commit into from

Conversation

saintstack
Copy link
Contributor

  • Add a bit of javadoc around SerialReplicationChecker.
  • Miniscule edit to the profiler jsp page and then a bit of doc on how to make it work that might help.
  • Add some detail if NPE getting BitSetNode to help w/ debug.
  • Change HbckChore to log region names instead of encoded names; helps doing diagnostics; can take region name and query in shell to find out all about the region according to hbase:meta.
  • Add some fix-it help inline in the HBCK Report page – how to fix.
  • Add counts in procedures page so can see if making progress; move listing of WALs to end of the page.

@Apache-HBase
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 0m 36s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 1s No case conflicting files found.
+1 💚 hbaseanti 0m 0s Patch does not have any anti-patterns.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
-0 ⚠️ test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ branch-2 Compile Tests _
+0 🆗 mvndep 0m 14s Maven dependency ordering for branch
+1 💚 mvninstall 5m 10s branch-2 passed
+1 💚 compile 2m 5s branch-2 passed
+1 💚 checkstyle 2m 23s branch-2 passed
+1 💚 shadedjars 4m 16s branch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 1m 37s branch-2 passed
+0 🆗 spotbugs 3m 9s Used deprecated FindBugs config; considering switching to SpotBugs.
+1 💚 findbugs 5m 25s branch-2 passed
_ Patch Compile Tests _
+0 🆗 mvndep 0m 15s Maven dependency ordering for patch
+1 💚 mvninstall 4m 45s the patch passed
+1 💚 compile 2m 2s the patch passed
+1 💚 javac 2m 2s the patch passed
-1 ❌ checkstyle 0m 15s hbase-procedure: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
-1 ❌ checkstyle 1m 17s hbase-server: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
+1 💚 whitespace 0m 0s The patch has no whitespace issues.
+1 💚 shadedjars 4m 12s patch has no errors when building our shaded downstream artifacts.
+1 💚 hadoopcheck 15m 29s Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2.
+1 💚 javadoc 1m 33s the patch passed
-1 ❌ findbugs 3m 30s hbase-server generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0)
_ Other Tests _
+1 💚 unit 3m 34s hbase-client in the patch passed.
+1 💚 unit 1m 13s hbase-http in the patch passed.
+1 💚 unit 3m 32s hbase-procedure in the patch passed.
-1 ❌ unit 162m 46s hbase-server in the patch failed.
+1 💚 asflicense 2m 9s The patch does not generate ASF License warnings.
238m 55s
Reason Tests
FindBugs module:hbase-server
Null pointer dereference of hri in org.apache.hadoop.hbase.master.HbckChore.loadRegionsFromFS() Dereferenced at HbckChore.java:in org.apache.hadoop.hbase.master.HbckChore.loadRegionsFromFS() Dereferenced at HbckChore.java:[line 267]
Load of known null value in org.apache.hadoop.hbase.master.HbckChore.loadRegionsFromFS() At HbckChore.java:in org.apache.hadoop.hbase.master.HbckChore.loadRegionsFromFS() At HbckChore.java:[line 267]
Failed junit tests hadoop.hbase.master.assignment.TestHbckChore
hadoop.hbase.client.TestScannersFromClientSide2
Subsystem Report/Notes
Docker Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-847/1/artifact/out/Dockerfile
GITHUB PR #847
Optional Tests dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile
uname Linux e1eb5bdd5651 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux
Build tool maven
Personality /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-847/out/precommit/personality/provided.sh
git revision branch-2 / 44c8b58
Default Java 1.8.0_181
checkstyle https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-847/1/artifact/out/diff-checkstyle-hbase-procedure.txt
checkstyle https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-847/1/artifact/out/diff-checkstyle-hbase-server.txt
findbugs https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-847/1/artifact/out/new-findbugs-hbase-server.html
unit https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-847/1/artifact/out/patch-unit-hbase-server.txt
Test Results https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-847/1/testReport/
Max. process+thread count 4809 (vs. ulimit of 10000)
modules C: hbase-client hbase-http hbase-procedure hbase-server U: .
Console output https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-847/1/console
versions git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11
Powered by Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

 * Add a bit of javadoc around SerialReplicationChecker.
 * Miniscule edit to the profiler jsp page and then a bit of doc on how to make it work that might help.
 * Add some detail if NPE getting BitSetNode to help w/ debug.
 * Change HbckChore to log region names instead of encoded names; helps doing diagnostics; can take region name and query in shell to find out all about the region according to hbase:meta.
 * Add some fix-it help inline in the HBCK Report page – how to fix.
 * Add counts in procedures page so can see if making progress; move listing of WALs to end of the page.
@Apache-HBase
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 0m 37s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+1 💚 hbaseanti 0m 0s Patch does not have any anti-patterns.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 1 new or modified test files.
_ branch-2 Compile Tests _
+0 🆗 mvndep 0m 15s Maven dependency ordering for branch
+1 💚 mvninstall 5m 15s branch-2 passed
+1 💚 compile 2m 3s branch-2 passed
+1 💚 checkstyle 2m 26s branch-2 passed
+1 💚 shadedjars 4m 10s branch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 1m 34s branch-2 passed
+0 🆗 spotbugs 3m 31s Used deprecated FindBugs config; considering switching to SpotBugs.
+1 💚 findbugs 5m 45s branch-2 passed
_ Patch Compile Tests _
+0 🆗 mvndep 0m 15s Maven dependency ordering for patch
+1 💚 mvninstall 4m 52s the patch passed
+1 💚 compile 2m 2s the patch passed
+1 💚 javac 2m 2s the patch passed
-1 ❌ checkstyle 1m 18s hbase-server: The patch generated 2 new + 19 unchanged - 0 fixed = 21 total (was 19)
+1 💚 whitespace 0m 0s The patch has no whitespace issues.
+1 💚 shadedjars 4m 10s patch has no errors when building our shaded downstream artifacts.
+1 💚 hadoopcheck 15m 25s Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2.
+1 💚 javadoc 1m 33s the patch passed
+1 💚 findbugs 6m 3s the patch passed
_ Other Tests _
+1 💚 unit 3m 33s hbase-client in the patch passed.
+1 💚 unit 1m 11s hbase-http in the patch passed.
+1 💚 unit 3m 31s hbase-procedure in the patch passed.
+1 💚 unit 162m 36s hbase-server in the patch passed.
+1 💚 asflicense 2m 7s The patch does not generate ASF License warnings.
238m 58s
Subsystem Report/Notes
Docker Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-847/2/artifact/out/Dockerfile
GITHUB PR #847
Optional Tests dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile
uname Linux c4192503e130 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux
Build tool maven
Personality /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-847/out/precommit/personality/provided.sh
git revision branch-2 / 1f9f45c
Default Java 1.8.0_181
checkstyle https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-847/2/artifact/out/diff-checkstyle-hbase-server.txt
Test Results https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-847/2/testReport/
Max. process+thread count 4694 (vs. ulimit of 10000)
modules C: hbase-client hbase-http hbase-procedure hbase-server U: .
Console output https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-847/2/console
versions git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11
Powered by Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

@saintstack
Copy link
Contributor Author

Merging these small changes by hand with fix for the above checkstyle too-long-lines complaint

@saintstack saintstack closed this Nov 19, 2019
<p>
<span>
The below are Regions we've lost account of. To be safe, run bulk load of any data found in these Region orphan directories back into the HBase cluster.
First make sure hbase:meta is in healthy state; run 'hbkc2 fixMeta' to be sure. Once this is done, per Region below, run a bulk
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"hbck2 fixMeta"

<span>
The below are Regions we've lost account of. To be safe, run bulk load of any data found in these Region orphan directories back into the HBase cluster.
First make sure hbase:meta is in healthy state; run 'hbkc2 fixMeta' to be sure. Once this is done, per Region below, run a bulk
load -- '$ hbase completebulkload REGION_DIR_PATH TABLE_NAME' -- and then delete the desiccated directory content (HFiles are removed upon successful load; all that is left are empty directories
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yuck. In a table with lots of busted regions, this completeBulkLoad would be tedious. Maybe we need a new hbck2 bulkloadOrphanedRegions command that can identify the orphans, bulk load them, and clean up the husks.

@@ -81,11 +81,14 @@
<th>Errors</th>
<th>Parameters</th>
</tr>
<% for (Procedure<?> proc : procedures) {
<%
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can't believe we're still actively maintaining JSP in 2020.

@ndimiduk
Copy link
Member

Belated +1.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants