Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HBASE-24425 Run hbck_chore_run and catalogjanitor_run on draw of 'HBC… #1771

Merged
merged 1 commit into from
May 27, 2020

Conversation

saintstack
Copy link
Contributor

…K Report' page

@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 0m 28s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
_ branch-2 Compile Tests _
_ Patch Compile Tests _
-0 ⚠️ whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
_ Other Tests _
+1 💚 asflicense 0m 13s The patch does not generate ASF License warnings.
2m 0s
Subsystem Report/Notes
Docker Client=19.03.9 Server=19.03.9 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1771/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR #1771
Optional Tests dupname asflicense
uname Linux 128a9edf8c6d 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision branch-2 / 0392977
whitespace https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1771/1/artifact/yetus-general-check/output/whitespace-eol.txt
Max. process+thread count 42 (vs. ulimit of 12500)
modules C: hbase-server U: hbase-server
Console output https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1771/1/console
versions git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered by Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 0m 35s Docker mode activated.
-0 ⚠️ yetus 0m 6s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ branch-2 Compile Tests _
+1 💚 mvninstall 4m 17s branch-2 passed
-0 ⚠️ javadoc 0m 41s hbase-server in branch-2 failed.
_ Patch Compile Tests _
+1 💚 mvninstall 3m 59s the patch passed
-0 ⚠️ javadoc 0m 39s hbase-server in the patch failed.
_ Other Tests _
+1 💚 unit 120m 33s hbase-server in the patch passed.
132m 56s
Subsystem Report/Notes
Docker Client=19.03.9 Server=19.03.9 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1771/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR #1771
Optional Tests javac javadoc unit
uname Linux 9521dab4ac3a 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision branch-2 / 0392977
Default Java 2020-01-14
javadoc https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1771/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt
javadoc https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1771/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt
Test Results https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1771/1/testReport/
Max. process+thread count 4546 (vs. ulimit of 12500)
modules C: hbase-server U: hbase-server
Console output https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1771/1/console
versions git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered by Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 0m 34s Docker mode activated.
-0 ⚠️ yetus 0m 5s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ branch-2 Compile Tests _
+1 💚 mvninstall 3m 41s branch-2 passed
+1 💚 javadoc 0m 38s branch-2 passed
_ Patch Compile Tests _
+1 💚 mvninstall 3m 12s the patch passed
+1 💚 javadoc 0m 34s the patch passed
_ Other Tests _
+1 💚 unit 130m 56s hbase-server in the patch passed.
141m 47s
Subsystem Report/Notes
Docker Client=19.03.9 Server=19.03.9 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1771/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile
GITHUB PR #1771
Optional Tests javac javadoc unit
uname Linux a91b4d6eefe5 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision branch-2 / 0392977
Default Java 1.8.0_232
Test Results https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1771/1/testReport/
Max. process+thread count 3881 (vs. ulimit of 12500)
modules C: hbase-server U: hbase-server
Console output https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1771/1/console
versions git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered by Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

Copy link
Contributor

@HorizonNet HorizonNet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. branch-2 only?

Copy link
Contributor

@virajjasani virajjasani left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left few comments. Looks good.

<%
final String cacheParameterValue = request.getParameter("cache");
boolean cache = Boolean.valueOf(cacheParameterValue);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Since we need boolean, we can directly use: Boolean.parseBoolean(cacheParameterValue)

if (!cache) {
// Run the two reporters inline w/ drawing of the page. If exception, will show in page draw.
master.getMasterRpcServices().runHbckChore(null, null);
master.getMasterRpcServices().runCatalogScan(null, null);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MasterRpcServices throw ServiceException for both runHbckChore and runCatalogScan. Maybe we can handle it and print some nice error message?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah. Was thinking about it. Though the rare SE not the end of the world. Added something at your prompting.... it looks like this when generated...

   if (!Boolean.parseBoolean(cacheParameterValue)) {
     // Run the two reporters inline w/ drawing of the page. If exception, will show in page draw.
     try {
       master.getMasterRpcServices().runHbckChore(null, null);
     } catch (org.apache.hbase.thirdparty.com.google.protobuf.ServiceException se) {
       out.write("Failed generating a new hbck_chore report; using cache; try again or run hbck_chore_run in the shell: " + se.getMessage() + "\n");
     }
     try {
       master.getMasterRpcServices().runCatalogScan(null, null);
     } catch (org.apache.hbase.thirdparty.com.google.protobuf.ServiceException se) {
       out.write("Failed generating a new catalogjanitor report; using cache; try again or run catalogjanitor_run in the shell: " + se.getMessage() + "\n");
     }
   }

@@ -37,9 +37,17 @@
<%@ page import="org.apache.hadoop.hbase.util.Pair" %>
<%@ page import="org.apache.hadoop.hbase.master.CatalogJanitor" %>
<%@ page import="org.apache.hadoop.hbase.master.CatalogJanitor.Report" %>
<%@ page import="org.apache.hadoop.hbase.util.Threads" %>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not being used it seems.

@virajjasani
Copy link
Contributor

LGTM. branch-2 only?

Looks like a clean apply to master and branch-2.3 👍

@saintstack
Copy link
Contributor Author

Address @virajjasani suggestions.

@HorizonNet I pushed against branch-2 since that is what I know and its stable.... will forward port when all is good.

Thanks for the reviews.

@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 0m 34s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
_ branch-2 Compile Tests _
_ Patch Compile Tests _
-0 ⚠️ whitespace 0m 0s The patch has 3 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
_ Other Tests _
+1 💚 asflicense 0m 11s The patch does not generate ASF License warnings.
2m 3s
Subsystem Report/Notes
Docker Client=19.03.9 Server=19.03.9 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1771/2/artifact/yetus-general-check/output/Dockerfile
GITHUB PR #1771
Optional Tests dupname asflicense
uname Linux eb69aa032478 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision branch-2 / 3ef44a4
whitespace https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1771/2/artifact/yetus-general-check/output/whitespace-eol.txt
Max. process+thread count 43 (vs. ulimit of 12500)
modules C: hbase-server U: hbase-server
Console output https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1771/2/console
versions git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered by Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 0m 45s Docker mode activated.
-0 ⚠️ yetus 0m 6s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ branch-2 Compile Tests _
+1 💚 mvninstall 5m 35s branch-2 passed
-0 ⚠️ javadoc 0m 49s hbase-server in branch-2 failed.
_ Patch Compile Tests _
+1 💚 mvninstall 5m 18s the patch passed
-0 ⚠️ javadoc 0m 49s hbase-server in the patch failed.
_ Other Tests _
+1 💚 unit 135m 45s hbase-server in the patch passed.
151m 23s
Subsystem Report/Notes
Docker Client=19.03.9 Server=19.03.9 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1771/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR #1771
Optional Tests javac javadoc unit
uname Linux d962d21e4272 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision branch-2 / 3ef44a4
Default Java 2020-01-14
javadoc https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1771/2/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt
javadoc https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1771/2/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt
Test Results https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1771/2/testReport/
Max. process+thread count 3537 (vs. ulimit of 12500)
modules C: hbase-server U: hbase-server
Console output https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1771/2/console
versions git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered by Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 1m 25s Docker mode activated.
-0 ⚠️ yetus 0m 6s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ branch-2 Compile Tests _
+1 💚 mvninstall 3m 57s branch-2 passed
+1 💚 javadoc 0m 40s branch-2 passed
_ Patch Compile Tests _
+1 💚 mvninstall 3m 38s the patch passed
+1 💚 javadoc 0m 35s the patch passed
_ Other Tests _
+1 💚 unit 211m 50s hbase-server in the patch passed.
225m 13s
Subsystem Report/Notes
Docker Client=19.03.9 Server=19.03.9 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1771/2/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile
GITHUB PR #1771
Optional Tests javac javadoc unit
uname Linux aafe54afd2ad 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision branch-2 / 3ef44a4
Default Java 1.8.0_232
Test Results https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1771/2/testReport/
Max. process+thread count 2602 (vs. ulimit of 12500)
modules C: hbase-server U: hbase-server
Console output https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1771/2/console
versions git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered by Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

@saintstack saintstack merged commit f8a5cc9 into apache:branch-2 May 27, 2020
saintstack added a commit that referenced this pull request May 27, 2020
…K Report' page (#1771)


Signed-off-by: clarax
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
Signed-off-by: Huaxiang Sun <huaxiangsun@apache.com>
saintstack added a commit that referenced this pull request May 27, 2020
…K Report' page (#1771)


Signed-off-by: clarax
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
Signed-off-by: Huaxiang Sun <huaxiangsun@apache.com>
clarax pushed a commit to clarax/hbase that referenced this pull request Nov 15, 2020
…K Report' page (apache#1771)


Signed-off-by: clarax
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
Signed-off-by: Huaxiang Sun <huaxiangsun@apache.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
6 participants