HBASE-24138 log more details about balancer decisions for StochasticLoadBalancer #1455

busbey · 2020-04-08T15:55:32Z

Based on logs needed when troubleshooting some balancer decisions.

when the log level on StochasticLoadBalancer is set to DEBUG messages look like this:

2020-04-08 09:57:39,515 DEBUG [Time-limited test] balancer.StochasticLoadBalancer$RegionCountSkewCostFunction(1196): RegionCountSkewCostFunction sees a total of 3 servers and 9 regions.
2020-04-08 09:57:39,527 DEBUG [Time-limited test] balancer.StochasticLoadBalancer(348): We need to load balance cluster; total cost=10005.833333333334, sum multiplier=10602.0; min cost/multiplier which needs balance is 0.05

when it is TRACE it looks like:

2020-04-08 10:49:48,416 DEBUG [Time-limited test] balancer.StochasticLoadBalancer$RegionCountSkewCostFunction(1196): RegionCountSkewCostFunction sees a total of 3 servers and 9 regions.
2020-04-08 10:49:48,417 TRACE [Time-limited test] balancer.StochasticLoadBalancer$RegionCountSkewCostFunction(1200): RegionCountSkewCostFunction sees server 'server11855,19031,-1' has 3 regions
2020-04-08 10:49:48,418 TRACE [Time-limited test] balancer.StochasticLoadBalancer$RegionCountSkewCostFunction(1200): RegionCountSkewCostFunction sees server 'server15849,14000,-1' has 3 regions
2020-04-08 10:49:48,418 TRACE [Time-limited test] balancer.StochasticLoadBalancer$RegionCountSkewCostFunction(1200): RegionCountSkewCostFunction sees server 'server87321,54683,-1' has 3 regions
2020-04-08 10:49:48,446 TRACE [Time-limited test] balancer.StochasticLoadBalancer(338): PrimaryRegionCountSkewCostFunction not needed
2020-04-08 10:49:48,447 TRACE [Time-limited test] balancer.StochasticLoadBalancer(338): RegionReplicaHostCostFunction not needed
2020-04-08 10:49:48,450 TRACE [Time-limited test] balancer.StochasticLoadBalancer(338): RegionReplicaRackCostFunction not needed
2020-04-08 10:49:48,451 DEBUG [Time-limited test] balancer.StochasticLoadBalancer(348): We need to load balance cluster; total cost=10005.833333333334, sum multiplier=10602.0; min cost/multiplier which needs balance is 0.05
2020-04-08 10:49:48,452 TRACE [Time-limited test] balancer.StochasticLoadBalancer(353): Balance decision detailed function costs=RegionCountSkewCostFunction : (500.0, 0.0); PrimaryRegionCountSkewCostFunction : (500.0, 0.0); MoveCostFunction : (7.0, 0.0); ServerLocalityCostFunction : (25.0, 0.0); RackLocalityCostFunction : (15.0, 0.0); TableSkewCostFunction : (35.0, 0.16666666666666666); RegionReplicaHostCostFunction : (100000.0, 0.0); RegionReplicaRackCostFunction : (10000.0, 0.0); ReadRequestCostFunction : (10000.0, 1.0); CPRequestCostFunction : (5.0, 0.0); WriteRequestCostFunction : (5.0, 0.0); MemStoreSizeCostFunction : (5.0, 0.0); StoreFileCostFunction : (5.0, 0.0);

…oadBalancer Change-Id: I35a182453525973c6006069176fea3bac1c8513e

Apache-HBase · 2020-04-08T16:42:41Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 46s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+1 💚	mvninstall	6m 8s	master passed
+1 💚	checkstyle	1m 18s	master passed
+1 💚	spotbugs	2m 17s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	5m 40s	the patch passed
-0 ⚠️	checkstyle	1m 14s	hbase-server: The patch generated 1 new + 104 unchanged - 0 fixed = 105 total (was 104)
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	12m 9s	Patch does not cause any errors with Hadoop 2.10.0 or 3.1.2.
+1 💚	spotbugs	2m 58s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 15s	The patch does not generate ASF License warnings.
		42m 21s

Subsystem	Report/Notes
Docker	Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1455/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#1455
Optional Tests	dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle
uname	Linux 6cf8ff945d33 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `c383bd1`
checkstyle	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1455/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt
Max. process+thread count	83 (vs. ulimit of 10000)
modules	C: hbase-server U: hbase-server
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1455/1/console
versions	git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

Change-Id: Id123299b1aab046f74f3d59bc7a7677e18fa5c6a

hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/StochasticLoadBalancer.java

virajjasani

+1

busbey · 2020-04-08T19:03:48Z

thanks for the review!

Apache-HBase · 2020-04-08T19:21:59Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 10s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	7m 3s	master passed
+1 💚	compile	1m 9s	master passed
+1 💚	shadedjars	5m 47s	branch has no errors when building our shaded downstream artifacts.
-0 ⚠️	javadoc	0m 43s	hbase-server in master failed.
		_ Patch Compile Tests _
+1 💚	mvninstall	6m 47s	the patch passed
+1 💚	compile	1m 9s	the patch passed
+1 💚	javac	1m 9s	the patch passed
+1 💚	shadedjars	5m 47s	patch has no errors when building our shaded downstream artifacts.
-0 ⚠️	javadoc	0m 40s	hbase-server in the patch failed.
		_ Other Tests _
+1 💚	unit	169m 37s	hbase-server in the patch passed.
		201m 42s

Subsystem	Report/Notes
Docker	Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1455/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#1455
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux d3f5be4f958e 4.15.0-91-generic #92-Ubuntu SMP Fri Feb 28 11:09:48 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `c383bd1`
Default Java	2020-01-14
javadoc	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1455/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt
javadoc	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1455/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt
Test Results	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1455/1/testReport/
Max. process+thread count	3187 (vs. ulimit of 10000)
modules	C: hbase-server U: hbase-server
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1455/1/console
versions	git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2020-04-08T19:35:47Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 46s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	6m 10s	master passed
+1 💚	compile	1m 1s	master passed
+1 💚	shadedjars	5m 5s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 38s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	5m 43s	the patch passed
+1 💚	compile	0m 59s	the patch passed
+1 💚	javac	0m 59s	the patch passed
+1 💚	shadedjars	4m 57s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 38s	the patch passed
		_ Other Tests _
+1 💚	unit	186m 43s	hbase-server in the patch passed.
		215m 24s

Subsystem	Report/Notes
Docker	Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1455/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile
GITHUB PR	#1455
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux ddc2cddfee3a 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `c383bd1`
Default Java	1.8.0_232
Test Results	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1455/1/testReport/
Max. process+thread count	2397 (vs. ulimit of 10000)
modules	C: hbase-server U: hbase-server
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1455/1/console
versions	git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

…oadBalancer (#1455) - at DEBUG log messages about RegionCountSkewCostFunction region/server totals - at DEBUG log messages about the decision to balance or not with total costs - at TRACE log messages about region count on each server RegionCountSkewCostFunction sees - at TRACE log message with the individual cost functions used in the decision to balance or not Signed-off-by: Viraj Jasani <vjasani@apache.org> (cherry picked from commit 2d78a28)

…oadBalancer (apache#1455) - at DEBUG log messages about RegionCountSkewCostFunction region/server totals - at DEBUG log messages about the decision to balance or not with total costs - at TRACE log messages about region count on each server RegionCountSkewCostFunction sees - at TRACE log message with the individual cost functions used in the decision to balance or not Signed-off-by: Viraj Jasani <vjasani@apache.org>

…oadBalancer (apache#1455) - at DEBUG log messages about RegionCountSkewCostFunction region/server totals - at DEBUG log messages about the decision to balance or not with total costs - at TRACE log messages about region count on each server RegionCountSkewCostFunction sees - at TRACE log message with the individual cost functions used in the decision to balance or not Signed-off-by: Viraj Jasani <vjasani@apache.org> (cherry picked from commit 2d78a28)

…oadBalancer (apache#1455) - at DEBUG log messages about RegionCountSkewCostFunction region/server totals - at DEBUG log messages about the decision to balance or not with total costs - at TRACE log messages about region count on each server RegionCountSkewCostFunction sees - at TRACE log message with the individual cost functions used in the decision to balance or not Signed-off-by: Viraj Jasani <vjasani@apache.org> (cherry picked from commit 2d78a28) (cherry picked from commit ce53382) Change-Id: I423ba070e9dffd3cc9e6a652574f5dae51912f5a

HBASE-24138 log more details about balancer decisions for StochasticL…

0786490

…oadBalancer Change-Id: I35a182453525973c6006069176fea3bac1c8513e

HBASE-24138 checkstyle fix per qabot.

e82b288

Change-Id: Id123299b1aab046f74f3d59bc7a7677e18fa5c6a

virajjasani reviewed Apr 8, 2020

View reviewed changes

hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/StochasticLoadBalancer.java Show resolved Hide resolved

virajjasani approved these changes Apr 8, 2020

View reviewed changes

busbey merged commit 2d78a28 into apache:master Apr 8, 2020

busbey deleted the HBASE-24138 branch April 8, 2020 19:03

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HBASE-24138 log more details about balancer decisions for StochasticLoadBalancer #1455

HBASE-24138 log more details about balancer decisions for StochasticLoadBalancer #1455

busbey commented Apr 8, 2020

Apache-HBase commented Apr 8, 2020

virajjasani left a comment

busbey commented Apr 8, 2020

Apache-HBase commented Apr 8, 2020

Apache-HBase commented Apr 8, 2020

HBASE-24138 log more details about balancer decisions for StochasticLoadBalancer #1455

HBASE-24138 log more details about balancer decisions for StochasticLoadBalancer #1455

Conversation

busbey commented Apr 8, 2020

Apache-HBase commented Apr 8, 2020

virajjasani left a comment

Choose a reason for hiding this comment

busbey commented Apr 8, 2020

Apache-HBase commented Apr 8, 2020

Apache-HBase commented Apr 8, 2020