Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HBASE-24528 : BalancerDecision queue implementation in HMaster with Admin API #2261

Closed
wants to merge 22 commits into from

Conversation

virajjasani
Copy link
Contributor

@virajjasani virajjasani commented Aug 15, 2020

  • Admin API getLogEntries() for ring buffer use-cases: so far, provides balancerDecision and slowLogResponse
  • Refactor RPC call for similar use-cases
  • Single RPC API getLogEntries() for both Master.proto and Admin.proto

@virajjasani virajjasani changed the title HBASE-24528 : BalancerDecision implementation with Admin API to retrieve data from HMaster queue HBASE-24528 : BalancerDecision queue implementation in HMaster with Admin API to retrieve data Aug 15, 2020
@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 7m 58s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 prototool 0m 0s prototool was not available.
+1 💚 hbaseanti 0m 0s Patch does not have any anti-patterns.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
_ master Compile Tests _
+0 🆗 mvndep 0m 22s Maven dependency ordering for branch
+1 💚 mvninstall 4m 13s master passed
+1 💚 checkstyle 2m 28s master passed
+0 🆗 refguide 5m 33s branch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect.
+1 💚 spotbugs 20m 18s master passed
_ Patch Compile Tests _
+0 🆗 mvndep 0m 14s Maven dependency ordering for patch
+1 💚 mvninstall 4m 17s the patch passed
-0 ⚠️ checkstyle 2m 44s root: The patch generated 17 new + 387 unchanged - 0 fixed = 404 total (was 387)
-0 ⚠️ rubocop 0m 14s The patch generated 4 new + 364 unchanged - 0 fixed = 368 total (was 364)
+1 💚 whitespace 0m 0s The patch has no whitespace issues.
+1 💚 xml 0m 2s The patch has no ill-formed XML file.
+0 🆗 refguide 6m 9s patch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect.
+1 💚 hadoopcheck 13m 22s Patch does not cause any errors with Hadoop 3.1.2 3.2.1.
+1 💚 hbaseprotoc 7m 52s the patch passed
+1 💚 spotbugs 23m 22s the patch passed
_ Other Tests _
+1 💚 asflicense 1m 24s The patch does not generate ASF License warnings.
110m 39s
Subsystem Report/Notes
Docker Client=19.03.12 Server=19.03.12 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR #2261
Optional Tests dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle refguide xml cc hbaseprotoc prototool rubocop
uname Linux a6136c5885a1 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / c81ef73
refguide https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/1/artifact/yetus-general-check/output/branch-site/book.html
checkstyle https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/1/artifact/yetus-general-check/output/diff-checkstyle-root.txt
rubocop https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/1/artifact/yetus-general-check/output/diff-patch-rubocop.txt
refguide https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/1/artifact/yetus-general-check/output/patch-site/book.html
Max. process+thread count 122 (vs. ulimit of 12500)
modules C: hbase-protocol-shaded hbase-common hbase-client hbase-server hbase-thrift hbase-shell . U: .
Console output https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/1/console
versions git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 rubocop=0.80.0
Powered by Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

@virajjasani virajjasani changed the title HBASE-24528 : BalancerDecision queue implementation in HMaster with Admin API to retrieve data HBASE-24528 : BalancerDecision queue implementation in HMaster with Admin API Aug 15, 2020
@Apache-HBase
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 4m 1s Docker mode activated.
-0 ⚠️ yetus 0m 3s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+0 🆗 mvndep 0m 22s Maven dependency ordering for branch
+1 💚 mvninstall 4m 9s master passed
+1 💚 compile 2m 47s master passed
+1 💚 shadedjars 5m 48s branch has no errors when building our shaded downstream artifacts.
-0 ⚠️ javadoc 0m 15s root in master failed.
-0 ⚠️ javadoc 0m 24s hbase-client in master failed.
-0 ⚠️ javadoc 0m 17s hbase-common in master failed.
-0 ⚠️ javadoc 0m 38s hbase-server in master failed.
-0 ⚠️ javadoc 0m 51s hbase-thrift in master failed.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 13s Maven dependency ordering for patch
+1 💚 mvninstall 4m 2s the patch passed
+1 💚 compile 2m 45s the patch passed
+1 💚 javac 2m 45s the patch passed
+1 💚 shadedjars 5m 41s patch has no errors when building our shaded downstream artifacts.
-0 ⚠️ javadoc 0m 17s hbase-common in the patch failed.
-0 ⚠️ javadoc 0m 25s hbase-client in the patch failed.
-0 ⚠️ javadoc 0m 38s hbase-server in the patch failed.
-0 ⚠️ javadoc 0m 50s hbase-thrift in the patch failed.
-0 ⚠️ javadoc 0m 15s root in the patch failed.
_ Other Tests _
-1 ❌ unit 137m 23s root in the patch failed.
176m 10s
Subsystem Report/Notes
Docker Client=19.03.12 Server=19.03.12 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR #2261
Optional Tests javac javadoc unit shadedjars compile
uname Linux f14e36d274da 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / c81ef73
Default Java 2020-01-14
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-root.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-client.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-common.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-thrift.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-common.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-client.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-thrift.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-root.txt
unit https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-root.txt
Test Results https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/1/testReport/
Max. process+thread count 4628 (vs. ulimit of 12500)
modules C: hbase-protocol-shaded hbase-common hbase-client hbase-server hbase-thrift hbase-shell . U: .
Console output https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/1/console
versions git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered by Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 7m 58s Docker mode activated.
-0 ⚠️ yetus 0m 3s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+0 🆗 mvndep 0m 21s Maven dependency ordering for branch
+1 💚 mvninstall 4m 17s master passed
+1 💚 compile 2m 40s master passed
+1 💚 shadedjars 6m 4s branch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 5m 1s master passed
_ Patch Compile Tests _
+0 🆗 mvndep 0m 13s Maven dependency ordering for patch
+1 💚 mvninstall 4m 7s the patch passed
+1 💚 compile 2m 48s the patch passed
+1 💚 javac 2m 48s the patch passed
+1 💚 shadedjars 6m 20s patch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 4m 52s the patch passed
_ Other Tests _
-1 ❌ unit 322m 30s root in the patch failed.
370m 14s
Subsystem Report/Notes
Docker Client=19.03.12 Server=19.03.12 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR #2261
Optional Tests javac javadoc unit shadedjars compile
uname Linux af906ce8d83f 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / c81ef73
Default Java 1.8.0_232
unit https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/1/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-root.txt
Test Results https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/1/testReport/
Max. process+thread count 3021 (vs. ulimit of 12500)
modules C: hbase-protocol-shaded hbase-common hbase-client hbase-server hbase-thrift hbase-shell . U: .
Console output https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/1/console
versions git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered by Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 1m 27s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 prototool 0m 0s prototool was not available.
+1 💚 hbaseanti 0m 0s Patch does not have any anti-patterns.
+1 💚 @author 0m 1s The patch does not contain any @author tags.
_ master Compile Tests _
+0 🆗 mvndep 0m 32s Maven dependency ordering for branch
+1 💚 mvninstall 3m 57s master passed
+1 💚 checkstyle 2m 22s master passed
+0 🆗 refguide 5m 55s branch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect.
+1 💚 spotbugs 20m 45s master passed
_ Patch Compile Tests _
+0 🆗 mvndep 0m 13s Maven dependency ordering for patch
+1 💚 mvninstall 4m 29s the patch passed
-0 ⚠️ checkstyle 2m 57s root: The patch generated 17 new + 387 unchanged - 0 fixed = 404 total (was 387)
-0 ⚠️ rubocop 0m 13s The patch generated 4 new + 364 unchanged - 0 fixed = 368 total (was 364)
+1 💚 whitespace 0m 0s The patch has no whitespace issues.
+1 💚 xml 0m 2s The patch has no ill-formed XML file.
+0 🆗 refguide 6m 20s patch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect.
+1 💚 hadoopcheck 13m 33s Patch does not cause any errors with Hadoop 3.1.2 3.2.1.
+1 💚 hbaseprotoc 8m 23s the patch passed
+1 💚 spotbugs 24m 1s the patch passed
_ Other Tests _
+1 💚 asflicense 1m 34s The patch does not generate ASF License warnings.
106m 52s
Subsystem Report/Notes
Docker Client=19.03.12 Server=19.03.12 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/2/artifact/yetus-general-check/output/Dockerfile
GITHUB PR #2261
Optional Tests dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle refguide xml cc hbaseprotoc prototool rubocop
uname Linux a0e5ba5dcb3c 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / c81ef73
refguide https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/2/artifact/yetus-general-check/output/branch-site/book.html
checkstyle https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/2/artifact/yetus-general-check/output/diff-checkstyle-root.txt
rubocop https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/2/artifact/yetus-general-check/output/diff-patch-rubocop.txt
refguide https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/2/artifact/yetus-general-check/output/patch-site/book.html
Max. process+thread count 122 (vs. ulimit of 12500)
modules C: hbase-protocol-shaded hbase-common hbase-hadoop-compat hbase-client hbase-server hbase-thrift hbase-shell . U: .
Console output https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/2/console
versions git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 rubocop=0.80.0
Powered by Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 1m 3s Docker mode activated.
-0 ⚠️ yetus 0m 3s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+0 🆗 mvndep 0m 21s Maven dependency ordering for branch
+1 💚 mvninstall 4m 41s master passed
+1 💚 compile 3m 1s master passed
+1 💚 shadedjars 6m 29s branch has no errors when building our shaded downstream artifacts.
-0 ⚠️ javadoc 0m 13s root in master failed.
-0 ⚠️ javadoc 0m 25s hbase-client in master failed.
-0 ⚠️ javadoc 0m 15s hbase-common in master failed.
-0 ⚠️ javadoc 0m 18s hbase-hadoop-compat in master failed.
-0 ⚠️ javadoc 0m 39s hbase-server in master failed.
-0 ⚠️ javadoc 0m 57s hbase-thrift in master failed.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 12s Maven dependency ordering for patch
+1 💚 mvninstall 4m 28s the patch passed
+1 💚 compile 3m 4s the patch passed
+1 💚 javac 3m 4s the patch passed
+1 💚 shadedjars 6m 20s patch has no errors when building our shaded downstream artifacts.
-0 ⚠️ javadoc 0m 15s hbase-common in the patch failed.
-0 ⚠️ javadoc 0m 17s hbase-hadoop-compat in the patch failed.
-0 ⚠️ javadoc 0m 25s hbase-client in the patch failed.
-0 ⚠️ javadoc 0m 43s hbase-server in the patch failed.
-0 ⚠️ javadoc 0m 58s hbase-thrift in the patch failed.
-0 ⚠️ javadoc 0m 14s root in the patch failed.
_ Other Tests _
+1 💚 unit 240m 14s root in the patch passed.
279m 10s
Subsystem Report/Notes
Docker Client=19.03.12 Server=19.03.12 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR #2261
Optional Tests javac javadoc unit shadedjars compile
uname Linux 9a5b74e5ccd7 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / c81ef73
Default Java 2020-01-14
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/2/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-root.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/2/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-client.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/2/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-common.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/2/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-hadoop-compat.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/2/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/2/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-thrift.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/2/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-common.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/2/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-hadoop-compat.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/2/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-client.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/2/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/2/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-thrift.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/2/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-root.txt
Test Results https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/2/testReport/
Max. process+thread count 4446 (vs. ulimit of 12500)
modules C: hbase-protocol-shaded hbase-common hbase-hadoop-compat hbase-client hbase-server hbase-thrift hbase-shell . U: .
Console output https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/2/console
versions git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered by Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

Copy link
Contributor

@apurtell apurtell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Most of this is boilerplate and mechanism, which I largely don't have concerns about, but I do have a concern with lack of generalization in some places. I think we want to take steps now to reduce future work especially API modification/deprecation pains. What do you think?

* @return list of balancer decision records
* @throws IOException if a remote or network exception occurs
*/
List<BalancerDecisionRecords> getBalancerDecisions() throws IOException;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just call this BalancerDecision ? Of course it's a record, or an item, or an element (choose your term for individual item in a collection) so that just adds letters for no clearer meaning.

private static final Gson GSON = GsonUtil.createGson()
.setPrettyPrinting()
.registerTypeAdapter(OnlineLogRecord.class, (JsonSerializer<OnlineLogRecord>)
(slowLogPayload, type, jsonSerializationContext) -> {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this is being used for more than just the slowlog, this parameter slowLogPayload should be renamed. (First question that comes to mind is what does slow log have to do with the balancer). Call it logPayload?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't believe this miss! This Gson serialization function was a copy paste :|

@@ -1057,4 +1057,9 @@ public void updateRSGroupConfig(String groupName, Map<String, String> configurat
throws IOException {
get(admin.updateRSGroupConfig(groupName, configuration));
}

@Override
public List<BalancerDecisionRecords> getBalancerDecisions() throws IOException {
Copy link
Contributor

@apurtell apurtell Aug 17, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Something to consider: Rather than adding new API for every ringbuffer backed type, since the ringbuffers are named, can we just have one API that retrieves records from a buffer specified by name?

E.g.

public List<LogEntry> getLogEntries(String name)

Then, LogEntry is a generic type capable of accepting any protobuf encoding. Then, we derive new types from LogEntry such as BalancerDecision. Have a static method in LogEntry for instantiating the subclasses by reflection based on what type is communicated by the protobuf.

If LogEntry is too generic a name, consider RingEntry. (I'm not the best at naming, maybe someone else has a better idea...)

It is a lot easier to add or remove specialized classes as these things evolve than add or remove methods from public/stable admin APIs.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would mean you'd have to go back and modify any admin API related to the slow log, which is fine, and desirable (if you accept the premise of this feedback)

Copy link
Contributor

@apurtell apurtell Aug 17, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we have this type of API, it should be possible to provide a limit, e.g.

public List<LogEntry> getLogEntries(String name, int limit);

so that a client that is memory constrained (or wants to be frugal) doesn't have to worry about invoking this and maybe getting back a list of 5000 entries or whatever, by passing in a limit of 100, or 10, or ...

It would be fine to also provide a method that doesn't accept a limit, for convenience. Prerequisite: The methods are generic enough so we don't add a pair (or more!) for every type. E.g.

public List<LogEntry> getLogEntries(String name) { this(name, Integer.MAX_VALUE); }

Copy link
Contributor Author

@virajjasani virajjasani Aug 17, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Definitely, adding new APIs not desirable, however few reasons why I had to choose this way:

  1. slowLog released in 2.3 already.
  2. slowLog is in each RS and the API has flexibility to choose serverName to get records from (filter by serverName) whereas balancer is HMaster queue meaning activeMasterStub is to be identified by client, hence no need of serverName filter.
  3. client has to identify whom to make a call - HMaster or RegionServer (or limited no of RS) based on use-case, not a big deal but maybe too much logic for client interface?
  4. genericAPI might need to include abstract request and response classes, and respective use-cases to ensure if filter is required at server side (not a big deal)

For 1), I believe trunk and 2.4.0 both can have generic API end-point.
2) and 3) seem bit of headache for single API.

Copy link
Contributor

@apurtell apurtell Aug 17, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree, that's the trade off here. Some pain now for flexibility later, or not much pain and no flexibility now or later.

New APIs for each new type is not a wrong choice per se. A benefit to API per ring type is pressure from API compat concerns and method proliferation will put a lot of pressure on new potential use cases for this mechanism, will set a really high bar. Maybe too high, hence my concerns.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So just to be clear, please take some time to think about this, but it is not a request for change per se.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, but if you dont' accept the majority of this feedback, which would be fine, we at least need the limit parameter. And also add one for the slow log if there isn't one there.

Copy link
Contributor Author

@virajjasani virajjasani Aug 17, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree method proliferation is a concern, something to consider and once we come to a decision, there is no way back it seems.
For limit param, we have limit in slowLog use-case because client is calling every single RS. Here, since it's just HMaster, the real limit should be kept in HMaster's ring buffer size. However, memory constraints for client makes limiting useful even for single server.

* History of balancer decisions taken for region movements.
*/
@InterfaceAudience.Private
final public class BalancerDecisionRecords {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just BalancerDecision? 'Records' doesn't add any meaning.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure

@@ -1277,4 +1278,9 @@ public void updateRSGroupConfig(String groupName, Map<String, String> configurat
throws IOException {
throw new NotImplementedException("updateRSGroupConfig not supported in ThriftAdmin");
}

@Override
public List<BalancerDecisionRecords> getBalancerDecisions() throws IOException {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we had one admin API for master and regionservers that returned protobuf encoded entries, then the thrift API could support it. First cut could just pass through the encoded PB.

balancer_decisions_resp_arr << balancer_dec_resp.toJsonPrettyPrint
}
puts 'Retrieved BalancerDecision Responses from HMaster'
puts balancer_decisions_resp_arr
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only return the array here, this may be used for programmatic things. Put the part that prints the result into the command impl which invokes this admin.rb function.

@@ -30,7 +30,8 @@
public class NamedQueuePayload {

public enum NamedQueueEvent {
SLOW_LOG
SLOW_LOG,
Copy link
Contributor

@apurtell apurtell Aug 17, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wish I had caught this earlier.
If we have to use an Enum here, add a constructor that defines an ordinal for each type so we can maintain compatibility by instantiating by our ordinal, e.g.

NamedQueueEvent(int ordinal) { ... }

SLOW_LOG(1),
BALANCE_DECISION(2),.
...

NamedQueueEvent getByOrdinal(int ordinal)

@virajjasani
Copy link
Contributor Author

virajjasani commented Aug 18, 2020

I believe we should have generic Admin API for similar use-cases. For existing slowLog use-case, I can make it default implementation pointing to generic API and provide deprecation notice (with removal in 4.0.0 release).

  1. Addressed concerns other than providing generic Admin API for ring buffer use-cases.
  2. Will add generic API changes as separate commit so that it becomes easier to review commit by commit on PR.
  3. I can take up thrift changes as a follow up? (to avoid too many changes going in same PR)

@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 1m 24s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 1s No case conflicting files found.
+0 🆗 prototool 0m 1s prototool was not available.
+1 💚 hbaseanti 0m 0s Patch does not have any anti-patterns.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
_ master Compile Tests _
+0 🆗 mvndep 0m 30s Maven dependency ordering for branch
+1 💚 mvninstall 4m 4s master passed
+1 💚 checkstyle 2m 28s master passed
+0 🆗 refguide 5m 55s branch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect.
+1 💚 spotbugs 21m 6s master passed
-0 ⚠️ patch 11m 36s Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 13s Maven dependency ordering for patch
+1 💚 mvninstall 4m 16s the patch passed
-0 ⚠️ checkstyle 2m 51s root: The patch generated 19 new + 436 unchanged - 1 fixed = 455 total (was 437)
-0 ⚠️ rubocop 0m 24s The patch generated 5 new + 566 unchanged - 2 fixed = 571 total (was 568)
+1 💚 whitespace 0m 0s The patch has no whitespace issues.
+1 💚 xml 0m 1s The patch has no ill-formed XML file.
+0 🆗 refguide 6m 15s patch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect.
+1 💚 hadoopcheck 13m 26s Patch does not cause any errors with Hadoop 3.1.2 3.2.1.
+1 💚 hbaseprotoc 8m 22s the patch passed
+1 💚 spotbugs 23m 59s the patch passed
_ Other Tests _
+1 💚 asflicense 1m 31s The patch does not generate ASF License warnings.
106m 57s
Subsystem Report/Notes
Docker Client=19.03.12 Server=19.03.12 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/3/artifact/yetus-general-check/output/Dockerfile
GITHUB PR #2261
Optional Tests dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle refguide xml cc hbaseprotoc prototool rubocop
uname Linux 144e1f1e63c8 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / ea26463
refguide https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/3/artifact/yetus-general-check/output/branch-site/book.html
checkstyle https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/3/artifact/yetus-general-check/output/diff-checkstyle-root.txt
rubocop https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/3/artifact/yetus-general-check/output/diff-patch-rubocop.txt
refguide https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/3/artifact/yetus-general-check/output/patch-site/book.html
Max. process+thread count 123 (vs. ulimit of 12500)
modules C: hbase-protocol-shaded hbase-common hbase-hadoop-compat hbase-client hbase-server hbase-thrift hbase-shell . U: .
Console output https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/3/console
versions git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 rubocop=0.80.0
Powered by Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 2m 17s Docker mode activated.
-0 ⚠️ yetus 0m 3s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+0 🆗 mvndep 0m 23s Maven dependency ordering for branch
+1 💚 mvninstall 4m 27s master passed
+1 💚 compile 3m 4s master passed
+1 💚 shadedjars 6m 24s branch has no errors when building our shaded downstream artifacts.
-0 ⚠️ javadoc 0m 15s hbase-common in master failed.
-0 ⚠️ javadoc 0m 17s hbase-hadoop-compat in master failed.
-0 ⚠️ javadoc 0m 24s hbase-client in master failed.
-0 ⚠️ javadoc 0m 42s hbase-server in master failed.
-0 ⚠️ javadoc 0m 57s hbase-thrift in master failed.
-0 ⚠️ javadoc 0m 13s root in master failed.
-0 ⚠️ patch 10m 43s Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 12s Maven dependency ordering for patch
+1 💚 mvninstall 4m 39s the patch passed
+1 💚 compile 3m 7s the patch passed
+1 💚 javac 3m 7s the patch passed
+1 💚 shadedjars 6m 24s patch has no errors when building our shaded downstream artifacts.
-0 ⚠️ javadoc 0m 15s hbase-common in the patch failed.
-0 ⚠️ javadoc 0m 17s hbase-hadoop-compat in the patch failed.
-0 ⚠️ javadoc 0m 25s hbase-client in the patch failed.
-0 ⚠️ javadoc 0m 40s hbase-server in the patch failed.
-0 ⚠️ javadoc 0m 58s hbase-thrift in the patch failed.
-0 ⚠️ javadoc 0m 13s root in the patch failed.
_ Other Tests _
+1 💚 unit 243m 40s root in the patch passed.
285m 9s
Subsystem Report/Notes
Docker Client=19.03.12 Server=19.03.12 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR #2261
Optional Tests javac javadoc unit shadedjars compile
uname Linux 753447dee88d 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / ea26463
Default Java 2020-01-14
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/3/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-common.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/3/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-hadoop-compat.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/3/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-client.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/3/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/3/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-thrift.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/3/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-root.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/3/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-common.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/3/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-hadoop-compat.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/3/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-client.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/3/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/3/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-thrift.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/3/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-root.txt
Test Results https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/3/testReport/
Max. process+thread count 4430 (vs. ulimit of 12500)
modules C: hbase-protocol-shaded hbase-common hbase-hadoop-compat hbase-client hbase-server hbase-thrift hbase-shell . U: .
Console output https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/3/console
versions git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered by Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

@virajjasani
Copy link
Contributor Author

@apurtell
Update so far:
Addressed all concerns including generic Admin API for future use-cases.

@Apache-HBase
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 1m 23s Docker mode activated.
-0 ⚠️ yetus 0m 4s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+0 🆗 mvndep 0m 31s Maven dependency ordering for branch
+1 💚 mvninstall 4m 4s master passed
+1 💚 compile 2m 39s master passed
+1 💚 shadedjars 6m 5s branch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 5m 5s master passed
-0 ⚠️ patch 12m 15s Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 13s Maven dependency ordering for patch
+1 💚 mvninstall 3m 55s the patch passed
+1 💚 compile 2m 58s the patch passed
+1 💚 javac 2m 58s the patch passed
+1 💚 shadedjars 6m 49s patch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 5m 10s the patch passed
_ Other Tests _
-1 ❌ unit 320m 9s root in the patch failed.
361m 48s
Subsystem Report/Notes
Docker Client=19.03.12 Server=19.03.12 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR #2261
Optional Tests javac javadoc unit shadedjars compile
uname Linux 35e267241f42 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / ea26463
Default Java 1.8.0_232
unit https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/3/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-root.txt
Test Results https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/3/testReport/
Max. process+thread count 2859 (vs. ulimit of 12500)
modules C: hbase-protocol-shaded hbase-common hbase-hadoop-compat hbase-client hbase-server hbase-thrift hbase-shell . U: .
Console output https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/3/console
versions git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered by Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 3m 26s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 prototool 0m 0s prototool was not available.
+1 💚 hbaseanti 0m 0s Patch does not have any anti-patterns.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
_ master Compile Tests _
+0 🆗 mvndep 0m 22s Maven dependency ordering for branch
+1 💚 mvninstall 4m 11s master passed
+1 💚 checkstyle 2m 24s master passed
+0 🆗 refguide 5m 28s branch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect.
+1 💚 spotbugs 20m 58s master passed
-0 ⚠️ patch 2m 27s Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 13s Maven dependency ordering for patch
+1 💚 mvninstall 4m 19s the patch passed
-0 ⚠️ checkstyle 2m 49s root: The patch generated 26 new + 437 unchanged - 0 fixed = 463 total (was 437)
-0 ⚠️ rubocop 0m 26s The patch generated 5 new + 594 unchanged - 2 fixed = 599 total (was 596)
+1 💚 whitespace 0m 0s The patch has no whitespace issues.
+1 💚 xml 0m 1s The patch has no ill-formed XML file.
+0 🆗 refguide 6m 17s patch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect.
+1 💚 hadoopcheck 13m 31s Patch does not cause any errors with Hadoop 3.1.2 3.2.1.
+1 💚 hbaseprotoc 8m 17s the patch passed
+1 💚 spotbugs 23m 57s the patch passed
_ Other Tests _
+1 💚 asflicense 1m 34s The patch does not generate ASF License warnings.
108m 37s
Subsystem Report/Notes
Docker Client=19.03.12 Server=19.03.12 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/4/artifact/yetus-general-check/output/Dockerfile
GITHUB PR #2261
Optional Tests dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle refguide xml cc hbaseprotoc prototool rubocop
uname Linux fa91c148e4a2 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / 98e3584
refguide https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/4/artifact/yetus-general-check/output/branch-site/book.html
checkstyle https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/4/artifact/yetus-general-check/output/diff-checkstyle-root.txt
rubocop https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/4/artifact/yetus-general-check/output/diff-patch-rubocop.txt
refguide https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/4/artifact/yetus-general-check/output/patch-site/book.html
Max. process+thread count 122 (vs. ulimit of 12500)
modules C: hbase-protocol-shaded hbase-common hbase-hadoop-compat hbase-client hbase-server hbase-thrift hbase-shell . U: .
Console output https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/4/console
versions git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 rubocop=0.80.0
Powered by Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 1m 29s Docker mode activated.
-0 ⚠️ yetus 0m 4s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+0 🆗 mvndep 0m 24s Maven dependency ordering for branch
+1 💚 mvninstall 4m 22s master passed
+1 💚 compile 2m 57s master passed
+1 💚 shadedjars 5m 59s branch has no errors when building our shaded downstream artifacts.
-0 ⚠️ javadoc 0m 15s root in master failed.
-0 ⚠️ javadoc 0m 24s hbase-client in master failed.
-0 ⚠️ javadoc 0m 18s hbase-common in master failed.
-0 ⚠️ javadoc 0m 19s hbase-hadoop-compat in master failed.
-0 ⚠️ javadoc 0m 38s hbase-server in master failed.
-0 ⚠️ javadoc 0m 50s hbase-thrift in master failed.
-0 ⚠️ patch 10m 16s Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 13s Maven dependency ordering for patch
+1 💚 mvninstall 4m 4s the patch passed
+1 💚 compile 2m 49s the patch passed
+1 💚 javac 2m 49s the patch passed
+1 💚 shadedjars 5m 53s patch has no errors when building our shaded downstream artifacts.
-0 ⚠️ javadoc 0m 17s hbase-common in the patch failed.
-0 ⚠️ javadoc 0m 18s hbase-hadoop-compat in the patch failed.
-0 ⚠️ javadoc 0m 24s hbase-client in the patch failed.
-0 ⚠️ javadoc 0m 39s hbase-server in the patch failed.
-0 ⚠️ javadoc 0m 53s hbase-thrift in the patch failed.
-0 ⚠️ javadoc 0m 15s root in the patch failed.
_ Other Tests _
+1 💚 unit 165m 20s root in the patch passed.
203m 57s
Subsystem Report/Notes
Docker Client=19.03.12 Server=19.03.12 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/4/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR #2261
Optional Tests javac javadoc unit shadedjars compile
uname Linux 578ffe23e4c4 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / 98e3584
Default Java 2020-01-14
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/4/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-root.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/4/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-client.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/4/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-common.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/4/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-hadoop-compat.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/4/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/4/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-thrift.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/4/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-common.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/4/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-hadoop-compat.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/4/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-client.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/4/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/4/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-thrift.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/4/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-root.txt
Test Results https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/4/testReport/
Max. process+thread count 6395 (vs. ulimit of 12500)
modules C: hbase-protocol-shaded hbase-common hbase-hadoop-compat hbase-client hbase-server hbase-thrift hbase-shell . U: .
Console output https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/4/console
versions git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered by Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

Copy link
Contributor

@apurtell apurtell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good, some changes requested for consideration, and some nits

@@ -64,6 +66,7 @@
import org.apache.hadoop.hbase.snapshot.UnknownSnapshotException;
import org.apache.hadoop.hbase.util.Bytes;
import org.apache.hadoop.hbase.util.Pair;
import org.apache.hbase.thirdparty.org.apache.commons.collections4.CollectionUtils;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a nonshaded collections util available? Might be an issue when backporting. Just a nit.

* @return Log entries representing online records from servers
* @throws IOException if a remote or network exception occurs
*/
List<LogEntry> getLogEntries(LogRequest logRequest) throws IOException;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This needs an optional parameter to allow constrained clients to limit the size of the returned list of LogEntry.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can consider limit as also one of the request attribute right? Hence, we are leaving upto individual use-cases to provide limit and both slowLog and balancerDecision do have limit provided as part of their request payload and both use-cases consider limit:
(All references are for 1. BalancerDecision followed by 2. SlowLog)

1.
public class BalancerDecisionRequest extends LogRequest {

  private int limit = 250;

  public int getLimit() {
...
...

2.
public class LogQueryFilter extends LogRequest {
...
...
  private int limit = 10;
...
...

And, here on the server side, we consider limiting records for both use-cases:

1.
public class BalancerDecisionQueueService implements NamedQueueService {
...
...
  @Override
  public NamedQueueGetResponse getNamedQueueRecords(NamedQueueGetRequest request) {
...
...
    int limit = Math.min(request.getBalancerDecisionRequest().getLimit(), balancerDecisions.size());
    // filter limit if provided
    balancerDecisions = balancerDecisions.subList(0, limit);
...
...

2.
public class LogHandlerUtils {
...
...
  public static List<TooSlowLog.SlowLogPayload> getFilteredLogs(
      AdminProtos.SlowLogResponseRequest request, List<TooSlowLog.SlowLogPayload> logPayloadList) {
...
...
    int limit = Math.min(request.getLimit(), logPayloadList.size());
    return logPayloadList.subList(0, limit);
  }

Also, this is where we convert Admin request payload to Protobuf request payload before passing on to servers over network call:

1.
  private CompletableFuture<List<LogEntry>> getBalancerDecisions(
      BalancerDecisionRequest balancerDecisionRequest) {
    return this.<List<LogEntry>>newMasterCaller()
      .action((controller, stub) ->
        this.call(controller, stub,
          MasterProtos.BalancerDecisionRequest.newBuilder()
/** => **/            .setLimit(balancerDecisionRequest.getLimit()).build(),
          MasterService.Interface::getBalancerDecisions, ProtobufUtil::toBalancerDecisionResponse))
      .call();
  }

2.
public final class RequestConverter {
...
...
  public static SlowLogResponseRequest buildSlowLogResponseRequest(
      final LogQueryFilter logQueryFilter) {
...
...
    return builder.setLimit(logQueryFilter.getLimit()).build();
  }

Copy link
Contributor

@apurtell apurtell Sep 2, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please define a limit parameter for the Admin API. I'm open to other suggestions, but by "individual use cases" I believe you mean the RPC message classes, and those are not the user facing API, they are an implementation detail.

As an alternative you can do the same thing as you did for the SlowLog API where a filter can be supplied, and one thing the filter interface lets you do is specify a limit.

I won't approve this without a limit option of some kind in the public user facing admin API.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is our Admin endpoint:

List<LogEntry> getLogEntries(LogRequest logRequest) throws IOException;

Now when client uses this to get slowLog response, they can provide limit in specific payload which extends LogRequest and send it over, similarly they can provide limit in BalancerDecisionRequest which extends LogRequest and get the response accordingly, both request payloads that extend LogRequest are anyways public interfaces for clients. Now, client would rather provide limit in request payload than as separate argument on this Admin endpoint (method signature). Does that sound good?

@@ -1057,4 +1051,9 @@ public void updateRSGroupConfig(String groupName, Map<String, String> configurat
throws IOException {
get(admin.updateRSGroupConfig(groupName, configuration));
}

@Override
public List<LogEntry> getLogEntries(LogRequest logRequest) throws IOException {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment on admin interface has implications here. (This needs an optional parameter to allow constrained clients to limit the size of the returned list of LogEntry.)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above, limit is included in request payload by individual use-cases.

* @param logRequest request payload with possible filters
* @return Log entries representing online records from servers
*/
CompletableFuture<List<LogEntry>> getLogEntries(LogRequest logRequest);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment on admin interface has implications here. (This needs an optional parameter to allow constrained clients to limit the size of the returned list of LogEntry.)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above, limit is included in request payload by individual use-cases.

Copy link
Contributor

@apurtell apurtell Sep 2, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please define a limit parameter for the Admin API. I'm open to other suggestions, but by "individual use cases" I believe you mean the RPC message classes, and those are not user facing public API. The user facing API here is the Admin API. This is the place to do this.

As an alternative you can do the same thing as you did for the SlowLog API where a filter can be supplied, and one thing the filter interface lets you do is specify a limit.

I won't approve this without a limit option of some kind in the public user facing admin API.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the future please do not mark conversations that are not resolved as resolved.

@@ -40,7 +40,7 @@

public RpcLogDetails(RpcCall rpcCall, Message param, String clientAddress, long responseSize,
String className, boolean isSlowLog, boolean isLargeLog) {
super(NamedQueueEvent.SLOW_LOG);
super(0);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The idea of using ordinals is we can encode them into message types and be more gentle at runtime if missing one (as opposed to what Java would do by default, a linkage error I think). However for code I think enums provide documentation of intent and magic constants should be avoided. Or, at least, use a static integer constant named "SLOW_LOG"...

@@ -201,7 +201,7 @@ public NamedQueueGetResponse getNamedQueueRecords(NamedQueueGetRequest request)
slowLogPayloads = getSlowLogPayloads(slowLogResponseRequest);
}
NamedQueueGetResponse response = new NamedQueueGetResponse();
response.setNamedQueueEvent(NamedQueuePayload.NamedQueueEvent.SLOW_LOG);
response.setNamedQueueEvent(0);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See related comment.

}

@Override
public String toString() {
return new ToStringBuilder(this)
.append("slowLogResponseRequest", slowLogResponseRequest)
.append("namedQueueEvent", namedQueueEvent)
.append("balancerDecisionRequest", balancerDecisionRequest)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"balancerDecisionsRequest"

@@ -3966,7 +3966,7 @@ public SlowLogResponses getSlowLogResponses(final RpcController controller,
}
List<SlowLogPayload> slowLogPayloads;
NamedQueueGetRequest namedQueueGetRequest = new NamedQueueGetRequest();
namedQueueGetRequest.setNamedQueueEvent(NamedQueuePayload.NamedQueueEvent.SLOW_LOG);
namedQueueGetRequest.setNamedQueueEvent(0);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See related comment.

Copy link
Contributor Author

@virajjasani virajjasani left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the review @apurtell
Reg limit param, we can treat it as one of the request payload params for Admin API and hence, both slowLog and balancerDecision Admin request params contain limit.


private int limit = 250;

public int getLimit() {
Copy link
Contributor Author

@virajjasani virajjasani Aug 20, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the request param coming to Admin API.

Detailed comment on Admin API: limit is included in request payload by individual use-cases.

* @return Log entries representing online records from servers
* @throws IOException if a remote or network exception occurs
*/
List<LogEntry> getLogEntries(LogRequest logRequest) throws IOException;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can consider limit as also one of the request attribute right? Hence, we are leaving upto individual use-cases to provide limit and both slowLog and balancerDecision do have limit provided as part of their request payload and both use-cases consider limit:
(All references are for 1. BalancerDecision followed by 2. SlowLog)

1.
public class BalancerDecisionRequest extends LogRequest {

  private int limit = 250;

  public int getLimit() {
...
...

2.
public class LogQueryFilter extends LogRequest {
...
...
  private int limit = 10;
...
...

And, here on the server side, we consider limiting records for both use-cases:

1.
public class BalancerDecisionQueueService implements NamedQueueService {
...
...
  @Override
  public NamedQueueGetResponse getNamedQueueRecords(NamedQueueGetRequest request) {
...
...
    int limit = Math.min(request.getBalancerDecisionRequest().getLimit(), balancerDecisions.size());
    // filter limit if provided
    balancerDecisions = balancerDecisions.subList(0, limit);
...
...

2.
public class LogHandlerUtils {
...
...
  public static List<TooSlowLog.SlowLogPayload> getFilteredLogs(
      AdminProtos.SlowLogResponseRequest request, List<TooSlowLog.SlowLogPayload> logPayloadList) {
...
...
    int limit = Math.min(request.getLimit(), logPayloadList.size());
    return logPayloadList.subList(0, limit);
  }

Also, this is where we convert Admin request payload to Protobuf request payload before passing on to servers over network call:

1.
  private CompletableFuture<List<LogEntry>> getBalancerDecisions(
      BalancerDecisionRequest balancerDecisionRequest) {
    return this.<List<LogEntry>>newMasterCaller()
      .action((controller, stub) ->
        this.call(controller, stub,
          MasterProtos.BalancerDecisionRequest.newBuilder()
/** => **/            .setLimit(balancerDecisionRequest.getLimit()).build(),
          MasterService.Interface::getBalancerDecisions, ProtobufUtil::toBalancerDecisionResponse))
      .call();
  }

2.
public final class RequestConverter {
...
...
  public static SlowLogResponseRequest buildSlowLogResponseRequest(
      final LogQueryFilter logQueryFilter) {
...
...
    return builder.setLimit(logQueryFilter.getLimit()).build();
  }

@@ -1057,4 +1051,9 @@ public void updateRSGroupConfig(String groupName, Map<String, String> configurat
throws IOException {
get(admin.updateRSGroupConfig(groupName, configuration));
}

@Override
public List<LogEntry> getLogEntries(LogRequest logRequest) throws IOException {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above, limit is included in request payload by individual use-cases.

* @param logRequest request payload with possible filters
* @return Log entries representing online records from servers
*/
CompletableFuture<List<LogEntry>> getLogEntries(LogRequest logRequest);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above, limit is included in request payload by individual use-cases.

@@ -693,6 +694,14 @@ message SwitchExceedThrottleQuotaResponse {
required bool previous_exceed_throttle_quota_enabled = 1;
}

message BalancerDecisionRequest {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh I see, this is just request payload we are sending over to RPC endpoint. But yeah, it could be Decisions

@@ -222,6 +226,14 @@ public synchronized void setConf(Configuration conf) {

curFunctionCosts= new Double[costFunctions.size()];
tempFunctionCosts= new Double[costFunctions.size()];

boolean isBalancerDecisionEnabled = getConf()
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Definitely, with verb it looks good.

@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 1m 29s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 1s No case conflicting files found.
+0 🆗 prototool 0m 1s prototool was not available.
+1 💚 hbaseanti 0m 0s Patch does not have any anti-patterns.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
_ master Compile Tests _
+0 🆗 mvndep 1m 29s Maven dependency ordering for branch
+1 💚 mvninstall 4m 11s master passed
+1 💚 checkstyle 2m 25s master passed
+0 🆗 refguide 5m 32s branch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect.
+1 💚 spotbugs 20m 58s master passed
-0 ⚠️ patch 2m 22s Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 13s Maven dependency ordering for patch
+1 💚 mvninstall 4m 4s the patch passed
-0 ⚠️ checkstyle 2m 38s root: The patch generated 4 new + 512 unchanged - 0 fixed = 516 total (was 512)
-0 ⚠️ rubocop 0m 27s The patch generated 16 new + 587 unchanged - 9 fixed = 603 total (was 596)
+1 💚 whitespace 0m 0s The patch has no whitespace issues.
+1 💚 xml 0m 1s The patch has no ill-formed XML file.
+0 🆗 refguide 6m 32s patch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect.
+1 💚 hadoopcheck 13m 42s Patch does not cause any errors with Hadoop 3.1.2 3.2.1.
+1 💚 hbaseprotoc 8m 20s the patch passed
+1 💚 spotbugs 25m 7s the patch passed
_ Other Tests _
+1 💚 asflicense 1m 30s The patch does not generate ASF License warnings.
108m 58s
Subsystem Report/Notes
Docker Client=19.03.12 Server=19.03.12 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/13/artifact/yetus-general-check/output/Dockerfile
GITHUB PR #2261
Optional Tests dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle refguide xml cc hbaseprotoc prototool rubocop
uname Linux 2cdacd09d851 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / 1e8db48
refguide https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/13/artifact/yetus-general-check/output/branch-site/book.html
checkstyle https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/13/artifact/yetus-general-check/output/diff-checkstyle-root.txt
rubocop https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/13/artifact/yetus-general-check/output/diff-patch-rubocop.txt
refguide https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/13/artifact/yetus-general-check/output/patch-site/book.html
Max. process+thread count 122 (vs. ulimit of 12500)
modules C: hbase-protocol-shaded hbase-common hbase-hadoop-compat hbase-client hbase-server hbase-thrift hbase-shell . U: .
Console output https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/13/console
versions git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 rubocop=0.80.0
Powered by Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 0m 33s Docker mode activated.
-0 ⚠️ yetus 0m 4s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+0 🆗 mvndep 6m 2s Maven dependency ordering for branch
+1 💚 mvninstall 5m 10s master passed
+1 💚 compile 3m 44s master passed
+1 💚 shadedjars 9m 1s branch has no errors when building our shaded downstream artifacts.
-0 ⚠️ javadoc 0m 18s hbase-common in master failed.
-0 ⚠️ javadoc 0m 21s hbase-hadoop-compat in master failed.
-0 ⚠️ javadoc 0m 29s hbase-client in master failed.
-0 ⚠️ javadoc 0m 52s hbase-server in master failed.
-0 ⚠️ javadoc 1m 5s hbase-thrift in master failed.
-0 ⚠️ javadoc 0m 17s root in master failed.
-0 ⚠️ patch 14m 10s Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 16s Maven dependency ordering for patch
+1 💚 mvninstall 5m 48s the patch passed
+1 💚 compile 3m 51s the patch passed
+1 💚 javac 3m 51s the patch passed
+1 💚 shadedjars 9m 4s patch has no errors when building our shaded downstream artifacts.
-0 ⚠️ javadoc 0m 18s hbase-common in the patch failed.
-0 ⚠️ javadoc 0m 22s hbase-hadoop-compat in the patch failed.
-0 ⚠️ javadoc 0m 29s hbase-client in the patch failed.
-0 ⚠️ javadoc 0m 50s hbase-server in the patch failed.
-0 ⚠️ javadoc 1m 12s hbase-thrift in the patch failed.
-0 ⚠️ javadoc 0m 17s root in the patch failed.
_ Other Tests _
+1 💚 unit 165m 59s root in the patch passed.
221m 52s
Subsystem Report/Notes
Docker Client=19.03.12 Server=19.03.12 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/13/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR #2261
Optional Tests javac javadoc unit shadedjars compile
uname Linux a050ca43fea6 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / 1e8db48
Default Java 2020-01-14
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/13/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-common.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/13/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-hadoop-compat.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/13/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-client.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/13/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/13/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-thrift.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/13/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-root.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/13/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-common.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/13/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-hadoop-compat.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/13/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-client.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/13/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/13/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-thrift.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/13/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-root.txt
Test Results https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/13/testReport/
Max. process+thread count 6513 (vs. ulimit of 12500)
modules C: hbase-protocol-shaded hbase-common hbase-hadoop-compat hbase-client hbase-server hbase-thrift hbase-shell . U: .
Console output https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/13/console
versions git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered by Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 0m 0s Docker mode activated.
-1 ❌ patch 0m 4s #2261 does not apply to master. Rebase required? Wrong Branch? See https://yetus.apache.org/documentation/in-progress/precommit-patchnames for help.
Subsystem Report/Notes
GITHUB PR #2261
Console output https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/14/console
versions git=2.17.1
Powered by Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 0m 1s Docker mode activated.
-1 ❌ patch 0m 3s #2261 does not apply to master. Rebase required? Wrong Branch? See https://yetus.apache.org/documentation/in-progress/precommit-patchnames for help.
Subsystem Report/Notes
GITHUB PR #2261
Console output https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/14/console
versions git=2.17.1
Powered by Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 0m 0s Docker mode activated.
-1 ❌ patch 0m 4s #2261 does not apply to master. Rebase required? Wrong Branch? See https://yetus.apache.org/documentation/in-progress/precommit-patchnames for help.
Subsystem Report/Notes
GITHUB PR #2261
Console output https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/14/console
versions git=2.17.1
Powered by Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 1m 16s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 1s No case conflicting files found.
+0 🆗 prototool 0m 1s prototool was not available.
+1 💚 hbaseanti 0m 0s Patch does not have any anti-patterns.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
_ master Compile Tests _
+0 🆗 mvndep 0m 29s Maven dependency ordering for branch
+1 💚 mvninstall 3m 50s master passed
+1 💚 checkstyle 2m 21s master passed
+0 🆗 refguide 5m 47s branch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect.
+1 💚 spotbugs 21m 55s master passed
-0 ⚠️ patch 12m 3s Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 14s Maven dependency ordering for patch
+1 💚 mvninstall 4m 29s the patch passed
-0 ⚠️ checkstyle 2m 35s root: The patch generated 8 new + 511 unchanged - 0 fixed = 519 total (was 511)
-0 ⚠️ rubocop 0m 25s The patch generated 15 new + 587 unchanged - 9 fixed = 602 total (was 596)
+1 💚 whitespace 0m 0s The patch has no whitespace issues.
+1 💚 xml 0m 2s The patch has no ill-formed XML file.
+0 🆗 refguide 5m 48s patch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect.
+1 💚 hadoopcheck 13m 2s Patch does not cause any errors with Hadoop 3.1.2 3.2.1.
+1 💚 hbaseprotoc 7m 50s the patch passed
+1 💚 spotbugs 24m 27s the patch passed
_ Other Tests _
+1 💚 asflicense 1m 32s The patch does not generate ASF License warnings.
106m 3s
Subsystem Report/Notes
Docker Client=19.03.12 Server=19.03.12 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/15/artifact/yetus-general-check/output/Dockerfile
GITHUB PR #2261
Optional Tests dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle refguide xml cc hbaseprotoc prototool rubocop
uname Linux 64afbef1e046 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / 2e96a5b
refguide https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/15/artifact/yetus-general-check/output/branch-site/book.html
checkstyle https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/15/artifact/yetus-general-check/output/diff-checkstyle-root.txt
rubocop https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/15/artifact/yetus-general-check/output/diff-patch-rubocop.txt
refguide https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/15/artifact/yetus-general-check/output/patch-site/book.html
Max. process+thread count 123 (vs. ulimit of 12500)
modules C: hbase-protocol-shaded hbase-common hbase-hadoop-compat hbase-client hbase-server hbase-thrift hbase-shell . U: .
Console output https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/15/console
versions git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 rubocop=0.80.0
Powered by Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 1m 11s Docker mode activated.
-0 ⚠️ yetus 0m 3s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+0 🆗 mvndep 0m 34s Maven dependency ordering for branch
+1 💚 mvninstall 4m 56s master passed
+1 💚 compile 3m 25s master passed
+1 💚 shadedjars 7m 49s branch has no errors when building our shaded downstream artifacts.
-0 ⚠️ javadoc 0m 18s hbase-common in master failed.
-0 ⚠️ javadoc 0m 20s hbase-hadoop-compat in master failed.
-0 ⚠️ javadoc 0m 28s hbase-client in master failed.
-0 ⚠️ javadoc 0m 45s hbase-server in master failed.
-0 ⚠️ javadoc 1m 4s hbase-thrift in master failed.
-0 ⚠️ javadoc 0m 15s root in master failed.
-0 ⚠️ patch 12m 36s Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 14s Maven dependency ordering for patch
+1 💚 mvninstall 4m 52s the patch passed
+1 💚 compile 3m 23s the patch passed
+1 💚 javac 3m 23s the patch passed
+1 💚 shadedjars 7m 49s patch has no errors when building our shaded downstream artifacts.
-0 ⚠️ javadoc 0m 17s hbase-common in the patch failed.
-0 ⚠️ javadoc 0m 20s hbase-hadoop-compat in the patch failed.
-0 ⚠️ javadoc 0m 28s hbase-client in the patch failed.
-0 ⚠️ javadoc 0m 45s hbase-server in the patch failed.
-0 ⚠️ javadoc 1m 2s hbase-thrift in the patch failed.
-0 ⚠️ javadoc 0m 15s root in the patch failed.
_ Other Tests _
+1 💚 unit 259m 49s root in the patch passed.
304m 19s
Subsystem Report/Notes
Docker Client=19.03.12 Server=19.03.12 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/15/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR #2261
Optional Tests javac javadoc unit shadedjars compile
uname Linux c0b90445b623 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / 2e96a5b
Default Java 2020-01-14
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/15/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-common.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/15/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-hadoop-compat.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/15/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-client.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/15/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/15/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-thrift.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/15/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-root.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/15/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-common.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/15/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-hadoop-compat.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/15/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-client.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/15/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/15/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-thrift.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/15/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-root.txt
Test Results https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/15/testReport/
Max. process+thread count 4484 (vs. ulimit of 12500)
modules C: hbase-protocol-shaded hbase-common hbase-hadoop-compat hbase-client hbase-server hbase-thrift hbase-shell . U: .
Console output https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/15/console
versions git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered by Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 0m 28s Docker mode activated.
-0 ⚠️ yetus 0m 3s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+0 🆗 mvndep 0m 35s Maven dependency ordering for branch
+1 💚 mvninstall 3m 27s master passed
+1 💚 compile 2m 23s master passed
+1 💚 shadedjars 6m 30s branch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 4m 37s master passed
-0 ⚠️ patch 12m 19s Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 13s Maven dependency ordering for patch
+1 💚 mvninstall 3m 24s the patch passed
+1 💚 compile 2m 18s the patch passed
+1 💚 javac 2m 18s the patch passed
+1 💚 shadedjars 6m 36s patch has no errors when building our shaded downstream artifacts.
-0 ⚠️ javadoc 0m 21s hbase-client generated 1 new + 2 unchanged - 0 fixed = 3 total (was 2)
-0 ⚠️ javadoc 2m 8s root generated 1 new + 45 unchanged - 0 fixed = 46 total (was 45)
_ Other Tests _
-1 ❌ unit 340m 11s root in the patch failed.
379m 11s
Subsystem Report/Notes
Docker Client=19.03.12 Server=19.03.12 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/15/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR #2261
Optional Tests javac javadoc unit shadedjars compile
uname Linux 9fb4abfc81cc 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / 2e96a5b
Default Java 1.8.0_232
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/15/artifact/yetus-jdk8-hadoop3-check/output/diff-javadoc-javadoc-hbase-client.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/15/artifact/yetus-jdk8-hadoop3-check/output/diff-javadoc-javadoc-root.txt
unit https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/15/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-root.txt
Test Results https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/15/testReport/
Max. process+thread count 6333 (vs. ulimit of 12500)
modules C: hbase-protocol-shaded hbase-common hbase-hadoop-compat hbase-client hbase-server hbase-thrift hbase-shell . U: .
Console output https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/15/console
versions git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered by Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

Copy link
Contributor

@apurtell apurtell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some cleanup needed for stuff left over from earlier revisions, then I will approve.

* Retrieve recent online records from HMaster / RegionServers.
* Examples include slow/large RPC logs, balancer decisions by master.
*
* @param serverNames servers to retrieve records from, useful in case of records maintained by
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happens when we have multiple masters? I think just a doc update is needed here to indicate the log for servertype=MASTER will only come from the currently active master. Can be done at commit time.

*/
@InterfaceAudience.Public
public enum ServerType {
HMASTER,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We try to not use 'H' prefixes in new code. Please just call these MASTER and REGION_SERVER. Can be fixed at commit time.

message LogRequest {
required string log_class_name = 1;
required bytes log_initializer_message = 2;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why "initializer"? What does that mean?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dang, it doesn't look good now. Context was different earlier. Let me keep it log_message, it's payload now.

optional uint32 limit = 1;
}

message BalancerDecisionsResponse {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we still need these?

We have generic LogRequest now, and the log request gives the name of the log we want to return (encapsulated) results from.

Protobuf for BalancerDecisions is fine!

But the BalancerDecisionsRequest and BalancerDecisionsResponse proto types are for an RPC interface from an earlier revision of this patch, and that (specialized) RPC no longer exists.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Although specialized RPC does not exist anymore, we would want client to encode use-case specific request (BalancerDecisionsRequest) in bytes to generic RPC and also retrieve response and then decode bytes into use-case specific response (BalancerDecisionsResponse, which wraps BalancerDecision proto).

Again, this decision comes for relatively easy-to-understand code. What do we need to do for a new ring buffer use-case?

  1. Define request and response message. Which will be encoded in bytes and sent to generic RPC API getLogEntries (BalancerDecisionsRequest and BalancerDecisionsResponse in this case)
  2. Define message for use-case specific payload that we want to return to end user. (message BalancerDecision in this case)
  3. Add parsing logic in ProtobufUtil.
    For our use-cases:
if (logClassName.contains("SlowLogResponses")) {

and

if (logClassName.contains("BalancerDecisionsResponse")) {

We don't need a new RPC or Admin API, but good to have new request/response message which can be encoded within generic LogRequest and LogResponse and the relevant parsing logic becomes easy to grasp.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok.
Please add a comment that explains this in the proto file where these definitions are made. Can be done at commit time, np.

Copy link
Contributor Author

@virajjasani virajjasani left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressing concerns, an addendum commit will follow.

message LogRequest {
required string log_class_name = 1;
required bytes log_initializer_message = 2;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dang, it doesn't look good now. Context was different earlier. Let me keep it log_message, it's payload now.

optional uint32 limit = 1;
}

message BalancerDecisionsResponse {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Although specialized RPC does not exist anymore, we would want client to encode use-case specific request (BalancerDecisionsRequest) in bytes to generic RPC and also retrieve response and then decode bytes into use-case specific response (BalancerDecisionsResponse, which wraps BalancerDecision proto).

Again, this decision comes for relatively easy-to-understand code. What do we need to do for a new ring buffer use-case?

  1. Define request and response message. Which will be encoded in bytes and sent to generic RPC API getLogEntries (BalancerDecisionsRequest and BalancerDecisionsResponse in this case)
  2. Define message for use-case specific payload that we want to return to end user. (message BalancerDecision in this case)
  3. Add parsing logic in ProtobufUtil.
    For our use-cases:
if (logClassName.contains("SlowLogResponses")) {

and

if (logClassName.contains("BalancerDecisionsResponse")) {

We don't need a new RPC or Admin API, but good to have new request/response message which can be encoded within generic LogRequest and LogResponse and the relevant parsing logic becomes easy to grasp.

@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 1m 33s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 prototool 0m 0s prototool was not available.
+1 💚 hbaseanti 0m 0s Patch does not have any anti-patterns.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
_ master Compile Tests _
+0 🆗 mvndep 0m 47s Maven dependency ordering for branch
+1 💚 mvninstall 4m 15s master passed
+1 💚 checkstyle 2m 33s master passed
+0 🆗 refguide 5m 39s branch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect.
+1 💚 spotbugs 20m 54s master passed
-0 ⚠️ patch 11m 30s Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 12s Maven dependency ordering for patch
+1 💚 mvninstall 4m 11s the patch passed
-0 ⚠️ checkstyle 2m 46s root: The patch generated 8 new + 511 unchanged - 0 fixed = 519 total (was 511)
-0 ⚠️ rubocop 0m 27s The patch generated 15 new + 587 unchanged - 9 fixed = 602 total (was 596)
+1 💚 whitespace 0m 0s The patch has no whitespace issues.
+1 💚 xml 0m 1s The patch has no ill-formed XML file.
+0 🆗 refguide 6m 24s patch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect.
+1 💚 hadoopcheck 13m 36s Patch does not cause any errors with Hadoop 3.1.2 3.2.1.
+1 💚 hbaseprotoc 8m 17s the patch passed
+1 💚 spotbugs 24m 46s the patch passed
_ Other Tests _
+1 💚 asflicense 1m 39s The patch does not generate ASF License warnings.
108m 14s
Subsystem Report/Notes
Docker Client=19.03.12 Server=19.03.12 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/16/artifact/yetus-general-check/output/Dockerfile
GITHUB PR #2261
Optional Tests dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle refguide xml cc hbaseprotoc prototool rubocop
uname Linux 9ee01311a561 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / e5ca9ad
refguide https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/16/artifact/yetus-general-check/output/branch-site/book.html
checkstyle https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/16/artifact/yetus-general-check/output/diff-checkstyle-root.txt
rubocop https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/16/artifact/yetus-general-check/output/diff-patch-rubocop.txt
refguide https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/16/artifact/yetus-general-check/output/patch-site/book.html
Max. process+thread count 122 (vs. ulimit of 12500)
modules C: hbase-protocol-shaded hbase-common hbase-hadoop-compat hbase-client hbase-server hbase-thrift hbase-shell . U: .
Console output https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/16/console
versions git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 rubocop=0.80.0
Powered by Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 0m 27s Docker mode activated.
-0 ⚠️ yetus 0m 4s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+0 🆗 mvndep 0m 23s Maven dependency ordering for branch
+1 💚 mvninstall 4m 55s master passed
+1 💚 compile 3m 20s master passed
+1 💚 shadedjars 7m 48s branch has no errors when building our shaded downstream artifacts.
-0 ⚠️ javadoc 0m 15s root in master failed.
-0 ⚠️ javadoc 0m 29s hbase-client in master failed.
-0 ⚠️ javadoc 0m 17s hbase-common in master failed.
-0 ⚠️ javadoc 0m 20s hbase-hadoop-compat in master failed.
-0 ⚠️ javadoc 0m 46s hbase-server in master failed.
-0 ⚠️ javadoc 1m 3s hbase-thrift in master failed.
-0 ⚠️ patch 12m 31s Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 14s Maven dependency ordering for patch
+1 💚 mvninstall 4m 55s the patch passed
+1 💚 compile 3m 22s the patch passed
+1 💚 javac 3m 22s the patch passed
+1 💚 shadedjars 7m 47s patch has no errors when building our shaded downstream artifacts.
-0 ⚠️ javadoc 0m 18s hbase-common in the patch failed.
-0 ⚠️ javadoc 0m 20s hbase-hadoop-compat in the patch failed.
-0 ⚠️ javadoc 0m 28s hbase-client in the patch failed.
-0 ⚠️ javadoc 0m 46s hbase-server in the patch failed.
-0 ⚠️ javadoc 1m 3s hbase-thrift in the patch failed.
-0 ⚠️ javadoc 0m 15s root in the patch failed.
_ Other Tests _
+1 💚 unit 296m 22s root in the patch passed.
340m 19s
Subsystem Report/Notes
Docker Client=19.03.12 Server=19.03.12 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/16/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR #2261
Optional Tests javac javadoc unit shadedjars compile
uname Linux 45bcc05a1218 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / e5ca9ad
Default Java 2020-01-14
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/16/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-root.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/16/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-client.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/16/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-common.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/16/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-hadoop-compat.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/16/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/16/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-thrift.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/16/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-common.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/16/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-hadoop-compat.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/16/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-client.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/16/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/16/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-thrift.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/16/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-root.txt
Test Results https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/16/testReport/
Max. process+thread count 4604 (vs. ulimit of 12500)
modules C: hbase-protocol-shaded hbase-common hbase-hadoop-compat hbase-client hbase-server hbase-thrift hbase-shell . U: .
Console output https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/16/console
versions git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered by Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 1m 56s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 1s No case conflicting files found.
+0 🆗 prototool 0m 1s prototool was not available.
+1 💚 hbaseanti 0m 0s Patch does not have any anti-patterns.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
_ master Compile Tests _
+0 🆗 mvndep 0m 14s Maven dependency ordering for branch
+1 💚 mvninstall 4m 29s master passed
+1 💚 checkstyle 2m 25s master passed
+0 🆗 refguide 5m 40s branch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect.
+1 💚 spotbugs 18m 50s master passed
-0 ⚠️ patch 10m 35s Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 12s Maven dependency ordering for patch
+1 💚 mvninstall 3m 48s the patch passed
-0 ⚠️ checkstyle 2m 23s root: The patch generated 8 new + 511 unchanged - 0 fixed = 519 total (was 511)
-0 ⚠️ rubocop 0m 24s The patch generated 15 new + 587 unchanged - 9 fixed = 602 total (was 596)
+1 💚 whitespace 0m 0s The patch has no whitespace issues.
+1 💚 xml 0m 1s The patch has no ill-formed XML file.
+0 🆗 refguide 5m 37s patch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect.
+1 💚 hadoopcheck 12m 22s Patch does not cause any errors with Hadoop 3.1.2 3.2.1.
+1 💚 hbaseprotoc 7m 35s the patch passed
+1 💚 spotbugs 20m 40s the patch passed
_ Other Tests _
+1 💚 asflicense 1m 27s The patch does not generate ASF License warnings.
97m 46s
Subsystem Report/Notes
Docker Client=19.03.12 Server=19.03.12 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/18/artifact/yetus-general-check/output/Dockerfile
GITHUB PR #2261
Optional Tests dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle refguide xml cc hbaseprotoc prototool rubocop
uname Linux 2578ca0f47cf 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / fe77630
refguide https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/18/artifact/yetus-general-check/output/branch-site/book.html
checkstyle https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/18/artifact/yetus-general-check/output/diff-checkstyle-root.txt
rubocop https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/18/artifact/yetus-general-check/output/diff-patch-rubocop.txt
refguide https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/18/artifact/yetus-general-check/output/patch-site/book.html
Max. process+thread count 122 (vs. ulimit of 12500)
modules C: hbase-protocol-shaded hbase-common hbase-hadoop-compat hbase-client hbase-server hbase-thrift hbase-shell . U: .
Console output https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/18/console
versions git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 rubocop=0.80.0
Powered by Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 0m 26s Docker mode activated.
-0 ⚠️ yetus 0m 3s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+0 🆗 mvndep 0m 14s Maven dependency ordering for branch
+1 💚 mvninstall 4m 47s master passed
+1 💚 compile 3m 20s master passed
+1 💚 shadedjars 7m 44s branch has no errors when building our shaded downstream artifacts.
-0 ⚠️ javadoc 0m 17s hbase-common in master failed.
-0 ⚠️ javadoc 0m 19s hbase-hadoop-compat in master failed.
-0 ⚠️ javadoc 0m 29s hbase-client in master failed.
-0 ⚠️ javadoc 0m 46s hbase-server in master failed.
-0 ⚠️ javadoc 1m 1s hbase-thrift in master failed.
-0 ⚠️ javadoc 0m 14s root in master failed.
-0 ⚠️ patch 12m 29s Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 13s Maven dependency ordering for patch
+1 💚 mvninstall 4m 48s the patch passed
+1 💚 compile 3m 20s the patch passed
+1 💚 javac 3m 20s the patch passed
+1 💚 shadedjars 7m 45s patch has no errors when building our shaded downstream artifacts.
-0 ⚠️ javadoc 0m 17s hbase-common in the patch failed.
-0 ⚠️ javadoc 0m 19s hbase-hadoop-compat in the patch failed.
-0 ⚠️ javadoc 0m 28s hbase-client in the patch failed.
-0 ⚠️ javadoc 0m 45s hbase-server in the patch failed.
-0 ⚠️ javadoc 1m 2s hbase-thrift in the patch failed.
-0 ⚠️ javadoc 0m 15s root in the patch failed.
_ Other Tests _
+1 💚 unit 255m 26s root in the patch passed.
298m 16s
Subsystem Report/Notes
Docker Client=19.03.12 Server=19.03.12 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/18/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR #2261
Optional Tests javac javadoc unit shadedjars compile
uname Linux d1256e8b6594 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / fe77630
Default Java 2020-01-14
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/18/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-common.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/18/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-hadoop-compat.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/18/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-client.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/18/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/18/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-thrift.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/18/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-root.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/18/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-common.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/18/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-hadoop-compat.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/18/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-client.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/18/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/18/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-thrift.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/18/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-root.txt
Test Results https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/18/testReport/
Max. process+thread count 4440 (vs. ulimit of 12500)
modules C: hbase-protocol-shaded hbase-common hbase-hadoop-compat hbase-client hbase-server hbase-thrift hbase-shell . U: .
Console output https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/18/console
versions git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered by Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 0m 29s Docker mode activated.
-0 ⚠️ yetus 0m 4s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+0 🆗 mvndep 0m 14s Maven dependency ordering for branch
+1 💚 mvninstall 3m 27s master passed
+1 💚 compile 2m 18s master passed
+1 💚 shadedjars 6m 30s branch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 4m 32s master passed
-0 ⚠️ patch 12m 13s Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 13s Maven dependency ordering for patch
+1 💚 mvninstall 3m 26s the patch passed
+1 💚 compile 2m 19s the patch passed
+1 💚 javac 2m 19s the patch passed
+1 💚 shadedjars 6m 30s patch has no errors when building our shaded downstream artifacts.
-0 ⚠️ javadoc 0m 22s hbase-client generated 1 new + 2 unchanged - 0 fixed = 3 total (was 2)
-0 ⚠️ javadoc 2m 1s root generated 1 new + 46 unchanged - 0 fixed = 47 total (was 46)
_ Other Tests _
-1 ❌ unit 325m 17s root in the patch failed.
363m 38s
Subsystem Report/Notes
Docker Client=19.03.12 Server=19.03.12 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/18/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR #2261
Optional Tests javac javadoc unit shadedjars compile
uname Linux 0504a5f9f0d2 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / fe77630
Default Java 1.8.0_232
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/18/artifact/yetus-jdk8-hadoop3-check/output/diff-javadoc-javadoc-hbase-client.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/18/artifact/yetus-jdk8-hadoop3-check/output/diff-javadoc-javadoc-root.txt
unit https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/18/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-root.txt
Test Results https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/18/testReport/
Max. process+thread count 5230 (vs. ulimit of 12500)
modules C: hbase-protocol-shaded hbase-common hbase-hadoop-compat hbase-client hbase-server hbase-thrift hbase-shell . U: .
Console output https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/18/console
versions git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered by Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 0m 30s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 1s No case conflicting files found.
+0 🆗 prototool 0m 1s prototool was not available.
+1 💚 hbaseanti 0m 0s Patch does not have any anti-patterns.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
_ master Compile Tests _
+0 🆗 mvndep 0m 34s Maven dependency ordering for branch
+1 💚 mvninstall 5m 11s master passed
+1 💚 checkstyle 2m 55s master passed
+0 🆗 refguide 8m 25s branch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect.
+1 💚 spotbugs 26m 21s master passed
-0 ⚠️ patch 14m 56s Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 16s Maven dependency ordering for patch
+1 💚 mvninstall 4m 48s the patch passed
-0 ⚠️ checkstyle 2m 54s root: The patch generated 9 new + 510 unchanged - 0 fixed = 519 total (was 510)
-0 ⚠️ rubocop 0m 34s The patch generated 15 new + 587 unchanged - 9 fixed = 602 total (was 596)
+1 💚 whitespace 0m 0s The patch has no whitespace issues.
+1 💚 xml 0m 1s The patch has no ill-formed XML file.
+0 🆗 refguide 7m 22s patch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect.
+1 💚 hadoopcheck 15m 58s Patch does not cause any errors with Hadoop 3.1.2 3.2.1.
+1 💚 hbaseprotoc 10m 10s the patch passed
+1 💚 spotbugs 28m 40s the patch passed
_ Other Tests _
+1 💚 asflicense 1m 48s The patch does not generate ASF License warnings.
128m 18s
Subsystem Report/Notes
Docker Client=19.03.12 Server=19.03.12 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/19/artifact/yetus-general-check/output/Dockerfile
GITHUB PR #2261
Optional Tests dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle refguide xml cc hbaseprotoc prototool rubocop
uname Linux ee1b039253cf 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / 325317f
refguide https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/19/artifact/yetus-general-check/output/branch-site/book.html
checkstyle https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/19/artifact/yetus-general-check/output/diff-checkstyle-root.txt
rubocop https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/19/artifact/yetus-general-check/output/diff-patch-rubocop.txt
refguide https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/19/artifact/yetus-general-check/output/patch-site/book.html
Max. process+thread count 123 (vs. ulimit of 12500)
modules C: hbase-protocol-shaded hbase-common hbase-hadoop-compat hbase-client hbase-server hbase-thrift hbase-shell . U: .
Console output https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/19/console
versions git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 rubocop=0.80.0
Powered by Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

Copy link
Contributor

@apurtell apurtell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ship it!

optional uint32 limit = 1;
}

message BalancerDecisionsResponse {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok.
Please add a comment that explains this in the proto file where these definitions are made. Can be done at commit time, np.

@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 1m 7s Docker mode activated.
-0 ⚠️ yetus 0m 3s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+0 🆗 mvndep 0m 28s Maven dependency ordering for branch
+1 💚 mvninstall 4m 39s master passed
+1 💚 compile 3m 4s master passed
+1 💚 shadedjars 7m 20s branch has no errors when building our shaded downstream artifacts.
-0 ⚠️ javadoc 0m 15s hbase-common in master failed.
-0 ⚠️ javadoc 0m 17s hbase-hadoop-compat in master failed.
-0 ⚠️ javadoc 0m 25s hbase-client in master failed.
-0 ⚠️ javadoc 0m 40s hbase-server in master failed.
-0 ⚠️ javadoc 0m 58s hbase-thrift in master failed.
-0 ⚠️ javadoc 0m 14s root in master failed.
-0 ⚠️ patch 11m 38s Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 11s Maven dependency ordering for patch
+1 💚 mvninstall 4m 34s the patch passed
+1 💚 compile 3m 5s the patch passed
+1 💚 javac 3m 5s the patch passed
+1 💚 shadedjars 7m 21s patch has no errors when building our shaded downstream artifacts.
-0 ⚠️ javadoc 0m 15s hbase-common in the patch failed.
-0 ⚠️ javadoc 0m 17s hbase-hadoop-compat in the patch failed.
-0 ⚠️ javadoc 0m 25s hbase-client in the patch failed.
-0 ⚠️ javadoc 0m 41s hbase-server in the patch failed.
-0 ⚠️ javadoc 0m 57s hbase-thrift in the patch failed.
-0 ⚠️ javadoc 0m 14s root in the patch failed.
_ Other Tests _
+1 💚 unit 238m 42s root in the patch passed.
280m 16s
Subsystem Report/Notes
Docker Client=19.03.12 Server=19.03.12 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/19/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR #2261
Optional Tests javac javadoc unit shadedjars compile
uname Linux 2c4daf389985 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / 325317f
Default Java 2020-01-14
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/19/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-common.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/19/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-hadoop-compat.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/19/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-client.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/19/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/19/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-thrift.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/19/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-root.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/19/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-common.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/19/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-hadoop-compat.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/19/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-client.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/19/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/19/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-thrift.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/19/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-root.txt
Test Results https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/19/testReport/
Max. process+thread count 4421 (vs. ulimit of 12500)
modules C: hbase-protocol-shaded hbase-common hbase-hadoop-compat hbase-client hbase-server hbase-thrift hbase-shell . U: .
Console output https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/19/console
versions git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered by Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 0m 28s Docker mode activated.
-0 ⚠️ yetus 0m 4s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+0 🆗 mvndep 0m 31s Maven dependency ordering for branch
+1 💚 mvninstall 3m 25s master passed
+1 💚 compile 2m 22s master passed
+1 💚 shadedjars 7m 8s branch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 4m 47s master passed
-0 ⚠️ patch 13m 1s Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 13s Maven dependency ordering for patch
+1 💚 mvninstall 3m 42s the patch passed
+1 💚 compile 2m 25s the patch passed
+1 💚 javac 2m 25s the patch passed
+1 💚 shadedjars 6m 37s patch has no errors when building our shaded downstream artifacts.
-0 ⚠️ javadoc 0m 20s hbase-client generated 1 new + 2 unchanged - 0 fixed = 3 total (was 2)
-0 ⚠️ javadoc 2m 18s root generated 1 new + 46 unchanged - 0 fixed = 47 total (was 46)
_ Other Tests _
-1 ❌ unit 262m 34s root in the patch failed.
302m 35s
Subsystem Report/Notes
Docker Client=19.03.12 Server=19.03.12 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/19/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR #2261
Optional Tests javac javadoc unit shadedjars compile
uname Linux 811e3a00c47b 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / 325317f
Default Java 1.8.0_232
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/19/artifact/yetus-jdk8-hadoop3-check/output/diff-javadoc-javadoc-hbase-client.txt
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/19/artifact/yetus-jdk8-hadoop3-check/output/diff-javadoc-javadoc-root.txt
unit https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/19/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-root.txt
Test Results https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/19/testReport/
Max. process+thread count 3761 (vs. ulimit of 12500)
modules C: hbase-protocol-shaded hbase-common hbase-hadoop-compat hbase-client hbase-server hbase-thrift hbase-shell . U: .
Console output https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2261/19/console
versions git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered by Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

@virajjasani virajjasani deleted the HBASE-24528-trunk branch September 16, 2020 15:28
virajjasani added a commit to virajjasani/hbase that referenced this pull request Sep 16, 2020
…dmin API

* Admin API getLogEntries() for ring buffer use-cases: so far, provides balancerDecision and slowLogResponse
* Refactor RPC call for similar use-cases
* Single RPC API getLogEntries() for both Master.proto and Admin.proto

Closes apache#2261

Signed-off-by: Andrew Purtell <apurtell@apache.org>
virajjasani added a commit that referenced this pull request Sep 17, 2020
…dmin API (#2411)

* Admin API getLogEntries() for ring buffer use-cases: so far, provides balancerDecision and slowLogResponse
* Refactor RPC call for similar use-cases
* Single RPC API getLogEntries() for both Master.proto and Admin.proto

Closes #2261

Signed-off-by: Andrew Purtell <apurtell@apache.org>
clarax pushed a commit to clarax/hbase that referenced this pull request Nov 15, 2020
…dmin API (apache#2411)

* Admin API getLogEntries() for ring buffer use-cases: so far, provides balancerDecision and slowLogResponse
* Refactor RPC call for similar use-cases
* Single RPC API getLogEntries() for both Master.proto and Admin.proto

Closes apache#2261

Signed-off-by: Andrew Purtell <apurtell@apache.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants