HBASE-26443 Some BaseLoadBalancer log lines should be at DEBUG level

[apurtell]

No description provided.

[Apache-HBase]

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	3m 33s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 1s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+0 🆗	mvndep	0m 19s	Maven dependency ordering for branch
+1 💚	mvninstall	4m 50s	master passed
+1 💚	compile	4m 9s	master passed
+1 💚	checkstyle	1m 28s	master passed
+1 💚	spotbugs	2m 56s	master passed
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 14s	Maven dependency ordering for patch
+1 💚	mvninstall	4m 26s	the patch passed
+1 💚	compile	4m 5s	the patch passed
+1 💚	javac	4m 5s	the patch passed
+1 💚	checkstyle	1m 28s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	21m 57s	Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1.
+1 💚	spotbugs	3m 53s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 29s	The patch does not generate ASF License warnings.
		64m 11s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3838/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#3838
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile
uname	Linux 261d2f8bf9d0 4.15.0-147-generic #151-Ubuntu SMP Fri Jun 18 19:21:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / cb3ecf6
Default Java	AdoptOpenJDK-1.8.0_282-b08
Max. process+thread count	86 (vs. ulimit of 30000)
modules	C: hbase-balancer hbase-server U: .
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3838/1/console
versions	git=2.17.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

[Apache-HBase]

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 2s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+0 🆗	mvndep	0m 17s	Maven dependency ordering for branch
+1 💚	mvninstall	4m 44s	master passed
+1 💚	compile	1m 39s	master passed
+1 💚	shadedjars	8m 24s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	1m 3s	master passed
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 18s	Maven dependency ordering for patch
+1 💚	mvninstall	4m 28s	the patch passed
+1 💚	compile	1m 35s	the patch passed
+1 💚	javac	1m 35s	the patch passed
+1 💚	shadedjars	8m 16s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 59s	the patch passed
		_ Other Tests _
+1 💚	unit	8m 58s	hbase-balancer in the patch passed.
+1 💚	unit	151m 43s	hbase-server in the patch passed.
		195m 40s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3838/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#3838
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux b9f65af06a78 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / cb3ecf6
Default Java	AdoptOpenJDK-11.0.10+9
Test Results	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3838/1/testReport/
Max. process+thread count	3940 (vs. ulimit of 30000)
modules	C: hbase-balancer hbase-server U: .
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3838/1/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

[Apache-HBase]

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	3m 22s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+0 🆗	mvndep	0m 20s	Maven dependency ordering for branch
+1 💚	mvninstall	4m 47s	master passed
+1 💚	compile	1m 32s	master passed
+1 💚	shadedjars	9m 28s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 57s	master passed
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 16s	Maven dependency ordering for patch
+1 💚	mvninstall	4m 25s	the patch passed
+1 💚	compile	1m 31s	the patch passed
+1 💚	javac	1m 31s	the patch passed
+1 💚	shadedjars	9m 14s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 56s	the patch passed
		_ Other Tests _
+1 💚	unit	10m 23s	hbase-balancer in the patch passed.
+1 💚	unit	216m 39s	hbase-server in the patch passed.
		266m 11s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3838/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#3838
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux f318f45c40ad 4.15.0-147-generic #151-Ubuntu SMP Fri Jun 18 19:21:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / cb3ecf6
Default Java	AdoptOpenJDK-1.8.0_282-b08
Test Results	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3838/1/testReport/
Max. process+thread count	2341 (vs. ulimit of 30000)
modules	C: hbase-balancer hbase-server U: .
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3838/1/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

[virajjasani]

+1

[ndimiduk]

I have no objections, only a question -- do we currently rely on the presence of these log statements to communicate to the operator that the balancer is running and doing work? Is there instead, maybe a metric or something on the UI that says the balancer is running, when it last ran, how long it's taking in a top style view, anything at all?

[busbey]

I don't think balancer activity makes it into the cluster status api.

There's a metric for how long the last balancer run took, but that only gets updated when a run finishes. it's up to each balancer implementation to properly set it; the SimpleLoadBalancer doesn't set it.

there are other more specific log messages emitted at INFO by the StochasticLoadBalancer about the start of, end of, and performing work. the SimpleLoadBalancer mostly logs at DEBUG and TRACE, with an INFO message only printed at the end of each run.

[virajjasani]

While not exact top command style view, but we do have balancer observability using the NamedQueue framework (opt-in feature): HBASE-24528 that provides recent balancer run's cost functions. On other other hand, using the same framework, we can also avail recent balancer rejection factors (opt-in feature) HBASE-25790.
With HBASE-25791, we also have UI of recent balancer history (using the same underlying framework). If we really want to combine both read recent balancer decision/rejection and it's associated cost functions as well as when was the last time it was executed, how much time it took, I think we might want to attach timestamp as well as time duration with these BalancerDecision/BalancerRejection objects?

[apurtell]

I left the balancer INFO logging as INFO where the message varies from log line to log line and is operationally useful. Only repetitive and uninteresting lines were considered.