HBASE-26308 Sum of multiplier of cost functions is not populated properly when we have a shortcut for trigger #3710

clarax · 2021-09-30T05:49:23Z

No description provided.

…erly when we have a shortcut for trigger

Apache-HBase · 2021-09-30T06:25:59Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 28s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	4m 6s	master passed
+1 💚	compile	0m 20s	master passed
+1 💚	shadedjars	8m 20s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 19s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 42s	the patch passed
+1 💚	compile	0m 20s	the patch passed
+1 💚	javac	0m 20s	the patch passed
+1 💚	shadedjars	8m 16s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 17s	the patch passed
		_ Other Tests _
+1 💚	unit	8m 26s	hbase-balancer in the patch passed.
		35m 48s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3710/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#3710
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux c2a31773192c 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `121bdea`
Default Java	AdoptOpenJDK-1.8.0_282-b08
Test Results	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3710/1/testReport/
Max. process+thread count	346 (vs. ulimit of 30000)
modules	C: hbase-balancer U: hbase-balancer
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3710/1/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2021-09-30T06:28:00Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 27s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+1 💚	mvninstall	4m 4s	master passed
+1 💚	compile	0m 34s	master passed
+1 💚	checkstyle	0m 17s	master passed
+1 💚	spotbugs	0m 38s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 39s	the patch passed
+1 💚	compile	0m 32s	the patch passed
+1 💚	javac	0m 32s	the patch passed
+1 💚	checkstyle	0m 15s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	18m 31s	Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0.
+1 💚	spotbugs	0m 47s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 15s	The patch does not generate ASF License warnings.
		37m 47s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3710/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#3710
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile
uname	Linux 69011cd5c07b 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `121bdea`
Default Java	AdoptOpenJDK-1.8.0_282-b08
Max. process+thread count	95 (vs. ulimit of 30000)
modules	C: hbase-balancer U: hbase-balancer
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3710/1/console
versions	git=2.17.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2021-09-30T06:32:17Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 31s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	5m 5s	master passed
+1 💚	compile	0m 22s	master passed
+1 💚	shadedjars	9m 11s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 19s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	4m 53s	the patch passed
+1 💚	compile	0m 21s	the patch passed
+1 💚	javac	0m 21s	the patch passed
+1 💚	shadedjars	9m 7s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 17s	the patch passed
		_ Other Tests _
+1 💚	unit	9m 45s	hbase-balancer in the patch passed.
		42m 4s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3710/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#3710
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 984d57abb8cc 4.15.0-143-generic #147-Ubuntu SMP Wed Apr 14 16:10:11 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `121bdea`
Default Java	AdoptOpenJDK-11.0.10+9
Test Results	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3710/1/testReport/
Max. process+thread count	211 (vs. ulimit of 30000)
modules	C: hbase-balancer U: hbase-balancer
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3710/1/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

...e-balancer/src/main/java/org/apache/hadoop/hbase/master/balancer/StochasticLoadBalancer.java

Apache9 · 2021-10-01T15:28:13Z

...e-balancer/src/main/java/org/apache/hadoop/hbase/master/balancer/StochasticLoadBalancer.java

-        + "hbase.master.balancer.stochastic.regionCountCost to a positive value or default");
-      return false;
-    }
-
    boolean balanced = (total / sumMultiplier < minCostNeedBalance);


We do not update sumMultiplier but use it directly? It could be zero here?

It is moved up to the earlier stage before needsBalance is called so the short cut for idle server or cohosted replicas wouldn't get the wrong sum.

...e-balancer/src/main/java/org/apache/hadoop/hbase/master/balancer/StochasticLoadBalancer.java

clarax · 2021-10-14T05:18:09Z

@Apache9 Any more concerned? I pushed the pre-commit check fixes.

huaxiangsun

Looks good to me.

clarax added 2 commits September 29, 2021 16:04

HBASE-26308 Sum of multiplier of cost functions is not populated prop…

908597f

…erly when we have a shortcut for trigger

Move to after init the cost functions.

470ee6f

clarax mentioned this pull request Sep 30, 2021

HBASE-26308 Sum of multiplier of cost functions is not populated prop… #3709

Closed

clarax changed the title ~~Hbase 26308 Sum of multiplier of cost functions is not populated properly when we have a shortcut for trigger~~ HNASE-26308 Sum of multiplier of cost functions is not populated properly when we have a shortcut for trigger Sep 30, 2021

clarax changed the title ~~HNASE-26308 Sum of multiplier of cost functions is not populated properly when we have a shortcut for trigger~~ HBASE-26308 Sum of multiplier of cost functions is not populated properly when we have a shortcut for trigger Sep 30, 2021

Apache9 reviewed Oct 1, 2021

View reviewed changes

clarax mentioned this pull request Oct 8, 2021

HBASE-26337 Optimization for weighted random generators #3732

Merged

huaxiangsun approved these changes Oct 20, 2021

View reviewed changes

Apache9 approved these changes Oct 21, 2021

View reviewed changes

huaxiangsun merged commit 0d49824 into apache:master Oct 21, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HBASE-26308 Sum of multiplier of cost functions is not populated properly when we have a shortcut for trigger #3710

HBASE-26308 Sum of multiplier of cost functions is not populated properly when we have a shortcut for trigger #3710

clarax commented Sep 30, 2021

Apache-HBase commented Sep 30, 2021

Apache-HBase commented Sep 30, 2021

Apache-HBase commented Sep 30, 2021

Apache9 Oct 1, 2021

clarax Oct 6, 2021

clarax commented Oct 14, 2021

huaxiangsun left a comment

HBASE-26308 Sum of multiplier of cost functions is not populated properly when we have a shortcut for trigger #3710

HBASE-26308 Sum of multiplier of cost functions is not populated properly when we have a shortcut for trigger #3710

Conversation

clarax commented Sep 30, 2021

Apache-HBase commented Sep 30, 2021

Apache-HBase commented Sep 30, 2021

Apache-HBase commented Sep 30, 2021

Apache9 Oct 1, 2021

Choose a reason for hiding this comment

clarax Oct 6, 2021

Choose a reason for hiding this comment

clarax commented Oct 14, 2021

huaxiangsun left a comment

Choose a reason for hiding this comment