Adaptive loadbalance #10745

ningboliu · 2022-10-13T07:32:43Z

What is the purpose of the change

issue #10571
adaptive loadbalance

Brief changelog

Verifying this change

Checklist

Make sure there is a GitHub_issue field for the change (usually before you start working on it). Trivial changes like typos do not require a GitHub issue. Your pull request should address just this issue, without pulling in other changes - one PR resolves one issue.
Each commit in the pull request should have a meaningful subject line and body.
Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
Check if is necessary to patch to Dubbo 3 if you are work on Dubbo 2.7
Write necessary unit-test to verify your logic correction, more mock a little better when cross module dependency exist. If the new feature or significant change is committed, please remember to add sample in dubbo samples project.
Add some description to dubbo-website project if you are requesting to add a feature.
GitHub Actions works fine on your own branch.
If this contribution is large, please follow the Software Donation Guide.

ningboliu · 2022-10-13T07:43:38Z

基准测试
环境
Linux version 3.10.0-1127.el7.x86_64
4C8G
java version "11.0.16.1"
网络：内网千兆网络

启动参数：-Xms2048m -Xmx2048m -XX:+UseG1GC

之前写的基准测试有问题，拿到的都是同一LoadBalance，以下数据为更新后的。
2服务器

Benchmark                      Mode  Cnt     Score      Error  Units
LoadBalance.adaptive          thrpt    4  7156.037 ± 9787.735  ops/s
LoadBalance.random            thrpt    4  6475.586 ± 2325.695  ops/s
LoadBalance.roundrobin        thrpt    4  6498.221 ± 1064.650  ops/s
LoadBalance.shortestresponse  thrpt    4  6407.356 ±  560.281  ops/s

3服务器,其中一台cpu 100%

Benchmark                      Mode  Cnt      Score      Error  Units
LoadBalance.adaptive          thrpt    4  15035.626 ± 2484.066  ops/s
LoadBalance.random            thrpt    4  10191.162 ± 2346.887  ops/s
LoadBalance.roundrobin        thrpt    4  10064.456 ± 4554.784  ops/s
LoadBalance.shortestresponse  thrpt    4  10487.329 ± 2498.746  ops/s

AlbumenJ · 2022-10-20T02:32:08Z

...ter/src/main/java/org/apache/dubbo/rpc/cluster/filter/support/AdaptiveLoadBalanceFilter.java

+    /**
+     * uses a single worker thread operating off an bounded queue
+     */
+    private static final ThreadPoolExecutor executor = new ThreadPoolExecutor(1, 1, 0L,TimeUnit.MILLISECONDS, new LinkedBlockingQueue<>(1024),


should be lazy init or use dubbo shared executor

ningboliu · 2022-10-21T14:25:03Z

三轮基准测试结果

3服务器均空闲

AlbumenJ · 2022-10-24T04:33:31Z

...ter/src/main/java/org/apache/dubbo/rpc/cluster/filter/support/AdaptiveLoadBalanceFilter.java

+        executor = new ThreadPoolExecutor(1, 1, 0L,TimeUnit.MILLISECONDS, new LinkedBlockingQueue<>(1024),
+            new NamedInternalThreadFactory("Dubbo-framework-loadbalance-adaptive", true), new ThreadPoolExecutor.DiscardOldestPolicy());


should not create executor by default when user do not use this

AlbumenJ · 2022-10-25T03:34:41Z

...ter/src/main/java/org/apache/dubbo/rpc/cluster/filter/support/AdaptiveLoadBalanceFilter.java

+                    executor = new ThreadPoolExecutor(1, 1, 0L,TimeUnit.MILLISECONDS, new LinkedBlockingQueue<>(1024),
+                        new NamedInternalThreadFactory("Dubbo-framework-loadbalance-adaptive", true), new ThreadPoolExecutor.DiscardOldestPolicy());


Register to org.apache.dubbo.common.resource.GlobalResourcesRepository#registerDisposable

AlbumenJ · 2022-10-25T03:35:33Z

dubbo-rpc/dubbo-rpc-api/src/main/java/org/apache/dubbo/rpc/AdaptiveMetrics.java

+ */
+public class AdaptiveMetrics {
+
+    private static final ConcurrentMap<String, AdaptiveMetrics> METRICS_STATISTICS = new ConcurrentHashMap<String,


Should not use static field to store status in Dubbo. Use scope bean instead

AlbumenJ · 2022-11-01T00:10:47Z

...ter/src/main/java/org/apache/dubbo/rpc/cluster/filter/support/AdaptiveLoadBalanceFilter.java

+ * @see org.apache.dubbo.rpc.Filter
+ * @see org.apache.dubbo.rpc.RpcContext
+ */
+@Activate(group = CONSUMER, order = -200000)


Suggested change

@Activate(group = CONSUMER, order = -200000)

@Activate(group = CONSUMER, order = -200000, value = {"loadbalance:adaptive"})

AlbumenJ · 2022-11-01T00:11:04Z

...ter/src/main/java/org/apache/dubbo/rpc/cluster/filter/support/AdaptiveLoadBalanceFilter.java

+    private AdaptiveMetrics adaptiveMetrics;
+
+    @Override
+    public void setApplicationModel(ApplicationModel scopeModel) {


replace to constructor

AlbumenJ · 2022-11-01T00:11:26Z

...ter/src/main/java/org/apache/dubbo/rpc/cluster/filter/support/AdaptiveLoadBalanceFilter.java

+    public void setApplicationModel(ApplicationModel scopeModel) {
+        AdaptiveMetrics bean = scopeModel.getBeanFactory().getBean(AdaptiveMetrics.class);
+        if (bean == null) {
+            scopeModel.getBeanFactory().registerBean(new AdaptiveMetrics());


Add into ScopeBeanInitializer

AlbumenJ · 2022-11-01T00:12:11Z

...ter/src/main/java/org/apache/dubbo/rpc/cluster/filter/support/AdaptiveLoadBalanceFilter.java

+    private AdaptiveMetrics getAdaptiveMetricsInstance(){
+        if (adaptiveMetrics == null) {
+            adaptiveMetrics = scopeModel.getBeanFactory().getBean(AdaptiveMetrics.class);
+        }
+        return adaptiveMetrics;
+    }


move to constructor

AlbumenJ · 2022-11-01T00:12:23Z

...ter/src/main/java/org/apache/dubbo/rpc/cluster/filter/support/AdaptiveLoadBalanceFilter.java

+    @Override
+    public void setApplicationModel(ApplicationModel scopeModel) {
+        AdaptiveMetrics bean = scopeModel.getBeanFactory().getBean(AdaptiveMetrics.class);
+        if (bean == null) {
+            scopeModel.getBeanFactory().registerBean(new AdaptiveMetrics());
+        }
+        this.scopeModel = scopeModel;
+    }


move to constructor

AlbumenJ · 2022-11-01T00:15:31Z

dubbo-cluster/src/main/java/org/apache/dubbo/rpc/cluster/loadbalance/AdaptiveLoadBalance.java

+        if (bean == null) {
+            scopeModel.getBeanFactory().registerBean(new AdaptiveMetrics());
+        }


use ScopeBeanInitializer

AlbumenJ · 2022-11-01T00:15:58Z

dubbo-cluster/src/main/java/org/apache/dubbo/rpc/cluster/loadbalance/AdaptiveLoadBalance.java

+    @Override
+    protected <T> Invoker<T> doSelect(List<Invoker<T>> invokers, URL url, Invocation invocation) {
+        Invoker invoker = selectByP2C(invokers,url,invocation);
+        invocation.setAttachment(Constants.ADAPTIVE_LOADBALANCE_ATTACHMENT_KEY,attachmentKey);


use attribute would better?

Need to pass parameters to provider

AlbumenJ · 2022-11-01T00:17:22Z

dubbo-cluster/src/main/java/org/apache/dubbo/rpc/cluster/loadbalance/AdaptiveLoadBalance.java

+    private String buildServiceKey(Invoker<?> invoker,Invocation invocation){
+        URL url = invoker.getUrl();
+        return url.getAddress() + ":" + invocation.getProtocolServiceKey();
+    }


cache this in invocation attribute

AlbumenJ · 2022-11-01T00:18:29Z

dubbo-cluster/src/main/java/org/apache/dubbo/rpc/cluster/loadbalance/AdaptiveLoadBalance.java

+        URL url = invoker.getUrl();
+        Object countdown = RpcContext.getClientAttachment().getObjectAttachment(TIME_COUNTDOWN_KEY);
+        int timeout;
+        if (countdown == null) {
+            timeout = (int) RpcUtils.getTimeout(url, invocation.getMethodName(), RpcContext.getClientAttachment(), DEFAULT_TIMEOUT);
+        } else {
+            TimeoutCountDown timeoutCountDown = (TimeoutCountDown) countdown;
+            timeout = (int) timeoutCountDown.timeRemaining(TimeUnit.MILLISECONDS);
+        }
+        return timeout;


why read current invocation's timeout?

AlbumenJ · 2022-11-01T00:18:53Z

dubbo-cluster/src/main/java/org/apache/dubbo/rpc/cluster/loadbalance/AdaptiveLoadBalance.java

+    private <T> Invoker<T> chooseLowLoadInvoker(Invoker<T> invoker1,Invoker<T> invoker2,Invocation invocation){
+        int weight1 = getWeight(invoker1, invocation);
+        int weight2 = getWeight(invoker2, invocation);
+        int timeout1 = getTimeout(invoker2, invocation);


Are there some spell mistakes?

AlbumenJ · 2022-11-25T08:43:52Z

...ter/src/main/java/org/apache/dubbo/rpc/cluster/filter/support/AdaptiveLoadBalanceFilter.java

+ * @see org.apache.dubbo.rpc.RpcContext
+ */
+@Activate(group = CONSUMER, order = -200000, value = {"loadbalance:adaptive"})
+public class AdaptiveLoadBalanceFilter implements ClusterFilter, ClusterFilter.Listener {


Move this filter to implement Filter would be better. The implementation are related with the actual provider address.

AlbumenJ · 2022-11-25T08:44:18Z

...ter/src/main/java/org/apache/dubbo/rpc/cluster/filter/support/AdaptiveLoadBalanceFilter.java

+    private String buildServiceKey(Invocation invocation){
+        StringBuilder sb = new StringBuilder(128);
+        sb.append(invocation.getInvoker().getUrl().getAddress()).append(":").append(invocation.getProtocolServiceKey());
+        return sb.toString();
+        //return url.getAddress() + ProtocolUtils.serviceKey(url.getPort(), url.getPath(), url.getVersion(), url.getGroup());
+    }
+
+    private String getServiceKey(Invocation invocation){
+
+        String key = (String) invocation.getAttributes().get(invocation.getInvoker());
+        if (StringUtils.isNotEmpty(key)){
+            return key;
+        }
+
+        key = buildServiceKey(invocation);
+        invocation.getAttributes().put(invocation.getInvoker(),key);
+        return key;
+    }


Will this key be reused?

used 2 times

codecov-commenter · 2022-11-25T09:25:28Z

Codecov Report

Merging #10745 (a65dbde) into 3.2 (9508acd) will decrease coverage by 0.01%.
The diff coverage is 57.14%.

@@             Coverage Diff              @@
##                3.2   #10745      +/-   ##
============================================
- Coverage     64.90%   64.88%   -0.02%     
  Complexity       14       14              
============================================
  Files          1468     1471       +3     
  Lines         61031    61245     +214     
  Branches       8944     8983      +39     
============================================
+ Hits          39614    39741     +127     
- Misses        17233    17296      +63     
- Partials       4184     4208      +24

Impacted Files	Coverage Δ
...he/dubbo/rpc/filter/AdaptiveLoadBalanceFilter.java	`0.00% <0.00%> (ø)`
.../apache/dubbo/rpc/filter/ProfilerServerFilter.java	`55.55% <33.33%> (-5.56%)`	⬇️
...o/rpc/cluster/loadbalance/AdaptiveLoadBalance.java	`88.00% <88.00%> (ø)`
...ain/java/org/apache/dubbo/rpc/AdaptiveMetrics.java	`89.79% <89.79%> (ø)`
...ubbo/rpc/cluster/ClusterScopeModelInitializer.java	`100.00% <100.00%> (ø)`
...che/dubbo/config/utils/DefaultConfigValidator.java	`86.36% <0.00%> (-9.10%)`	⬇️
...e/dubbo/remoting/transport/netty/NettyChannel.java	`52.27% <0.00%> (-7.96%)`	⬇️
.../rpc/cluster/router/mock/MockInvokersSelector.java	`58.97% <0.00%> (-5.13%)`	⬇️
.../apache/dubbo/remoting/transport/AbstractPeer.java	`67.39% <0.00%> (-4.35%)`	⬇️
.../cluster/filter/support/ConsumerContextFilter.java	`61.22% <0.00%> (-4.00%)`	⬇️
... and 33 more

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

AlbumenJ · 2022-11-28T01:46:01Z

...-cluster/src/test/java/org/apache/dubbo/rpc/cluster/loadbalance/AdaptiveLoadBalanceTest.java

+        System.out.println(sumInvoker1);
+        System.out.println(sumInvoker2);
+        System.out.println(sumInvoker5);


Print here is useless

AlbumenJ · 2022-11-28T01:46:27Z

...-cluster/src/test/java/org/apache/dubbo/rpc/cluster/loadbalance/AdaptiveLoadBalanceTest.java

+        int expectWeightValue = loop / totalWeight;
+        int maxDeviation = expectWeightValue * 2;
+        double beta = 0.5;
+        //这个估算值并不准确


comment in English

AlbumenJ · 2022-11-28T01:46:47Z

...-cluster/src/test/java/org/apache/dubbo/rpc/cluster/loadbalance/AdaptiveLoadBalanceTest.java

+        System.out.println(sumInvoker1);
+        System.out.println(sumInvoker2);
+        System.out.println(sumInvoker3);


Print is useless in unit test

AlbumenJ

LGTM

sonarcloud · 2022-12-13T08:17:54Z

Kudos, SonarCloud Quality Gate passed!

0 Bugs
0 Vulnerabilities
0 Security Hotspots
3 Code Smells

28.4% Coverage
0.0% Duplication

liuningbo and others added 2 commits October 13, 2022 11:06

issue 10571

6e84c8c

Merge branch '3.1' into adaptiveLoadbalance

90da4de

AlbumenJ changed the base branch from 3.1 to 3.2 October 14, 2022 02:39

ningboliu added 2 commits October 20, 2022 00:26

test case

f7b1150

rollback

897bf0b

AlbumenJ requested changes Oct 20, 2022

View reviewed changes

ningboliu added 2 commits October 20, 2022 13:18

lazy init & code style

c355e39

code style

affa7eb

ningboliu requested a review from AlbumenJ October 21, 2022 14:11

ningboliu and others added 3 commits October 23, 2022 10:25

Merge branch '3.2' into adaptiveLoadbalance

b27cf64

code style

3effb4e

code style

9a4e917

AlbumenJ requested changes Oct 24, 2022

View reviewed changes

ningboliu added 2 commits October 24, 2022 19:02

Lazy loading the thread pool

9bd85ee

when user use adaptive loadbalance

8c43e89

ningboliu requested a review from AlbumenJ October 25, 2022 02:37

AlbumenJ requested changes Oct 25, 2022

View reviewed changes

ningboliu added 2 commits October 28, 2022 14:05

Use scope bean

c4a8fd1

revert demo

4705938

ningboliu requested a review from AlbumenJ October 29, 2022 05:43

AlbumenJ requested changes Nov 1, 2022

View reviewed changes

ningboliu and others added 5 commits November 24, 2022 16:05

code review fixed

6431161

attributes to attachment

bb558b3

cache key

a6fcdcd

cache key

260b449

Merge branch '3.2' into adaptiveLoadbalance

5ea3d84

ningboliu requested a review from AlbumenJ November 25, 2022 04:42

AlbumenJ requested changes Nov 25, 2022

View reviewed changes

ningboliu added 3 commits November 25, 2022 20:23

fix Sonar quality

5dd82ea

fix Sonar quality

2d46bb8

fix bug

8547ad8

ningboliu requested a review from AlbumenJ November 25, 2022 15:33

AlbumenJ requested changes Nov 28, 2022

View reviewed changes

Remove useless code

bc0c753

ningboliu requested a review from AlbumenJ November 28, 2022 02:24

ningboliu and others added 2 commits December 12, 2022 15:40

Merge branch '3.2' into adaptiveLoadbalance

b0ffe9c

merge 3.2

a65dbde

AlbumenJ approved these changes Dec 13, 2022

View reviewed changes

AlbumenJ merged commit d171605 into apache:3.2 Dec 13, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adaptive loadbalance #10745

Adaptive loadbalance #10745

ningboliu commented Oct 13, 2022 •

edited by AlbumenJ

Loading

ningboliu commented Oct 13, 2022 •

edited

Loading

AlbumenJ Oct 20, 2022

ningboliu commented Oct 21, 2022 •

edited

Loading

AlbumenJ Oct 24, 2022

AlbumenJ Oct 25, 2022

AlbumenJ Oct 25, 2022

AlbumenJ Nov 1, 2022

AlbumenJ Nov 1, 2022

AlbumenJ Nov 1, 2022

AlbumenJ Nov 1, 2022

AlbumenJ Nov 1, 2022

AlbumenJ Nov 1, 2022

AlbumenJ Nov 1, 2022

ningboliu Nov 25, 2022

AlbumenJ Nov 1, 2022

AlbumenJ Nov 1, 2022

AlbumenJ Nov 1, 2022

AlbumenJ Nov 25, 2022

AlbumenJ Nov 25, 2022

ningboliu Nov 25, 2022

codecov-commenter commented Nov 25, 2022 •

edited

Loading

AlbumenJ Nov 28, 2022

AlbumenJ Nov 28, 2022

AlbumenJ Nov 28, 2022

AlbumenJ left a comment

sonarcloud bot commented Dec 13, 2022 •

edited

Loading

		executor = new ThreadPoolExecutor(1, 1, 0L,TimeUnit.MILLISECONDS, new LinkedBlockingQueue<>(1024),
		new NamedInternalThreadFactory("Dubbo-framework-loadbalance-adaptive", true), new ThreadPoolExecutor.DiscardOldestPolicy());

	@Activate(group = CONSUMER, order = -200000)
	@Activate(group = CONSUMER, order = -200000, value = {"loadbalance:adaptive"})

Adaptive loadbalance #10745

Adaptive loadbalance #10745

Conversation

ningboliu commented Oct 13, 2022 • edited by AlbumenJ Loading

What is the purpose of the change

Brief changelog

Verifying this change

Checklist

ningboliu commented Oct 13, 2022 • edited Loading

Choose a reason for hiding this comment

ningboliu commented Oct 21, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov-commenter commented Nov 25, 2022 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

AlbumenJ left a comment

Choose a reason for hiding this comment

sonarcloud bot commented Dec 13, 2022 • edited Loading

ningboliu commented Oct 13, 2022 •

edited by AlbumenJ

Loading

ningboliu commented Oct 13, 2022 •

edited

Loading

ningboliu commented Oct 21, 2022 •

edited

Loading

codecov-commenter commented Nov 25, 2022 •

edited

Loading

sonarcloud bot commented Dec 13, 2022 •

edited

Loading