[#584] feat(netty): Add transport client pool for netty #771

xumanbu · 2023-03-27T10:04:16Z

What changes were proposed in this pull request?

add for netty rpc client TransportClient
TransportClientFactory for connection pool
TransportContext contains the context to create a TransportClientFactory, setup Netty Channel pipelines with a TransportResponseHandler
TransportConf for netty transport config create by RssConf

Why are the changes needed?

Does this PR introduce any user-facing change?

add client configurations and add the ability to reuse netty clients.
Todo: update the user documentation after the netty feature is completed @xumanbu

How was this patch tested?

local test

codecov-commenter · 2023-03-27T11:33:57Z

Codecov Report

Merging #771 (bd9c2a7) into master (3f9ba81) will increase coverage by 0.16%.
The diff coverage is 16.32%.

@@             Coverage Diff              @@
##             master     #771      +/-   ##
============================================
+ Coverage     60.95%   61.11%   +0.16%     
+ Complexity     1956     1851     -105     
============================================
  Files           244      226      -18     
  Lines         13308    10967    -2341     
  Branches       1119     1089      -30     
============================================
- Hits           8112     6703    -1409     
+ Misses         4740     3889     -851     
+ Partials        456      375      -81

Impacted Files	Coverage Δ
...e/uniffle/common/netty/client/TransportClient.java	`0.00% <0.00%> (ø)`
...le/common/netty/client/TransportClientFactory.java	`0.00% <0.00%> (ø)`
...che/uniffle/common/netty/client/TransportConf.java	`0.00% <0.00%> (ø)`
.../uniffle/common/netty/client/TransportContext.java	`0.00% <0.00%> (ø)`
.../common/netty/handle/TransportResponseHandler.java	`0.00% <0.00%> (ø)`
.../uniffle/common/netty/protocol/MessageDecoder.java	`0.00% <0.00%> (ø)`
.../uniffle/common/netty/protocol/MessageEncoder.java	`0.00% <0.00%> (ø)`
...rg/apache/uniffle/common/config/RssClientConf.java	`98.21% <100.00%> (+4.46%)`	⬆️

... and 26 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

common/src/main/java/org/apache/uniffle/common/netty/client/TransportContext.java

leixm · 2023-03-28T06:45:42Z

@xumanbu Encoder and Decoder already added, see #742 .

common/src/main/java/org/apache/uniffle/common/netty/client/TransportClientFactory.java

common/src/main/java/org/apache/uniffle/common/netty/client/TransportClient.java

common/src/main/java/org/apache/uniffle/common/netty/client/TransportClientFactory.java

…ansportClientFactory.java

jerqi

LGTM

advancedxy · 2023-03-28T09:01:11Z

@xumanbu would you mind modify the pr description to the current status?

xumanbu · 2023-03-28T09:06:53Z

@xumanbu Encoder and Decoder already added, see #742 .

I'll rebase with #742

xumanbu · 2023-03-28T09:45:27Z

@xumanbu would you mind modify the pr description to the current status?

done.

advancedxy · 2023-03-28T09:51:24Z

@xumanbu would you mind modify the pr description to the current status?

done.

I think the content of Does this PR introduce any user-facing change? should fall into What changes were proposed in this pull request??

And regarding the user changes, you could describe it as add client configurations and add the ability to reuse netty clients. And you may update the user documentation after the netty feature is completed.

xumanbu · 2023-03-28T10:20:06Z

@xumanbu would you mind modify the pr description to the current status?

done.

I think the content of Does this PR introduce any user-facing change? should fall into What changes were proposed in this pull request??

And regarding the user changes, you could describe it as add client configurations and add the ability to reuse netty clients. And you may update the user documentation after the netty feature is completed.

thanks for your guidance. edited.

smallzhongfeng · 2023-03-28T09:51:35Z

common/src/main/java/org/apache/uniffle/common/netty/client/TransportClientFactory.java

+    ClientPool clientPool = connectionPool.get(unresolvedAddress);
+    if (clientPool == null) {
+      connectionPool.putIfAbsent(unresolvedAddress, new ClientPool(numConnectionsPerPeer));
+      clientPool = connectionPool.get(unresolvedAddress);


Could we use computeIfAbsent to merge this two lines?

smallzhongfeng · 2023-03-28T11:08:24Z

common/src/main/java/org/apache/uniffle/common/netty/client/TransportClientFactory.java

+  public TransportClientFactory(TransportContext context) {
+    this.context = Preconditions.checkNotNull(context);
+    this.conf = context.getConf();
+    this.connectionPool = new ConcurrentHashMap<>();


Maybe we can use JavaUtils.newConcurrentMap()

advancedxy · 2023-03-28T11:09:09Z

common/src/main/java/org/apache/uniffle/common/config/RssClientConf.java

+  public static final ConfigOption<Integer> NETTY_CLIENT_NUM_CONNECTIONS_PER_PEER = ConfigOptions
+      .key("rss.client.netty.client.connections.per.peer")
+      .intType()
+      .defaultValue(2)


is this too small?

I think default value small is safty. In small number uniffle server cluster,the large value will case a lot of connections to server.

Make sense.

advancedxy · 2023-03-28T11:12:28Z

common/src/main/java/org/apache/uniffle/common/config/RssClientConf.java

+      .withDescription("netty connect to server time out mills");
+
+  public static final ConfigOption<IOMode> NETTY_IO_MODE = ConfigOptions
+      .key("rss.client.netty.io.mode")


@leixm added rss.server.netty.epoll.enable for netty sever,do you guys it's better to rename that configuration to rss.server.netty.io.mode instead? So they are more consistent.

It's ok for me, i will fix it later PRs.

advancedxy · 2023-03-28T11:19:16Z

common/src/main/java/org/apache/uniffle/common/netty/client/TransportClient.java

+
+  public TransportClient(Channel channel, TransportResponseHandler handler) {
+    this.channel = Preconditions.checkNotNull(channel);
+    this.handler = Preconditions.checkNotNull(handler);


Use java.util.Objects#requireNonNull instead?

Preconditions#checkNotNull will throw NPE when object is null,Preconditions may easyer to use than Object#requireNonNull.

Preconditions#checkNotNull will throw NPE when object is null,

Objects#requireNonNull does exactly the same. The point is that we should prefer std lib over guava as guava has its reputation to break things in various versions, therefore it will cause class conflict for other systems which also has different version of guava.

make sense.I'll instead.

advancedxy · 2023-03-28T11:22:02Z

common/src/main/java/org/apache/uniffle/common/netty/client/TransportClient.java

+    @Override
+    protected void handleFailure(String errorMsg, Throwable cause) {
+      handler.removeRpcRequest(rpcRequestId);
+      callback.onFailure(new IOException(errorMsg, cause));


should we add a logging here?

make sense.I'll add same log.

same log is already add in parent method,so I think it's enough at now.

advancedxy · 2023-03-28T11:22:49Z

common/src/main/java/org/apache/uniffle/common/netty/client/TransportClient.java

+    return counter.getAndIncrement();
+  }
+
+  public class StdChannelListener implements GenericFutureListener<Future<? super Void>> {


Seems like that it's better to declare it as public static class?

use non-static inner class for Listener is for the method to access the attributes of the out class,such as channel,handle.

advancedxy · 2023-03-28T11:25:05Z

common/src/main/java/org/apache/uniffle/common/netty/client/TransportClientFactory.java

+  public TransportClientFactory(TransportContext context) {
+    this.context = Preconditions.checkNotNull(context);
+    this.conf = context.getConf();
+    this.connectionPool = new ConcurrentHashMap<>();


Use JavaUtils.newConcurrentMap?

common/src/main/java/org/apache/uniffle/common/netty/client/TransportClientFactory.java

advancedxy · 2023-03-28T11:30:46Z

common/src/main/java/org/apache/uniffle/common/netty/client/TransportClientFactory.java

+    synchronized (clientPool.locks[clientIndex]) {
+      cachedClient = clientPool.clients[clientIndex];
+
+      if (cachedClient != null) {
+        if (cachedClient.isActive()) {
+          logger.trace("Returning cached connection to {}: {}", resolvedAddress, cachedClient);
+          return cachedClient;
+        } else {
+          logger.info("Found inactive connection to {}, creating a new one.", resolvedAddress);
+        }
+      }
+      clientPool.clients[clientIndex] = internalCreateClient(resolvedAddress, decoder);


Could you wrap these block of code into the ClientPool structure?

Such as clientPoo.createClientIfAbsent, which should be similar as ConcurrentMap's computeIfAbsent?

I'm trying to create createClientIfAbsent function in ClientPool. like this :

TransportClient createClientIfAbsent( int clientIndex , InetSocketAddress resolvedAddress , ChannelInboundHandlerAdapter decoder , BiFunction<InetSocketAddress,ChannelInboundHandlerAdapter,TransportClient> createClientFunction ){ clients[clientIndex] = createClientFunction.apply(resolvedAddress, decoder); return clients[clientIndex]; } }

but It seems not very graceful becuase of createClientFunction have two unrelated args.

common/src/main/java/org/apache/uniffle/common/netty/client/TransportContext.java

…ansportClientFactory.java make sense Co-authored-by: advancedxy <xianjin@apache.org>

leixm · 2023-03-29T08:29:41Z

common/src/main/java/org/apache/uniffle/common/config/RssClientConf.java

+      .withDescription("netty connect to server time out mills");
+
+  public static final ConfigOption<IOMode> NETTY_IO_MODE = ConfigOptions
+      .key("rss.client.netty.io.mode")


It's ok for me, i will fix it later PRs.

leixm · 2023-03-29T08:30:44Z

common/src/main/java/org/apache/uniffle/common/config/RssClientConf.java

+  public static final ConfigOption<Integer> NETTY_CLIENT_NUM_CONNECTIONS_PER_PEER = ConfigOptions
+      .key("rss.client.netty.client.connections.per.peer")
+      .intType()
+      .defaultValue(2)


Make sense.

leixm · 2023-03-29T08:49:23Z

common/src/main/java/org/apache/uniffle/common/netty/client/RpcResponseCallback.java

+   * <p>After `onSuccess` returns, `response` will be recycled and its content will become invalid.
+   * Please copy the content of `response` if you want to use it after `onSuccess` returns.
+   */
+  void onSuccess(RpcResponse rpcResponse);


In fact, not all response types are RPCResponse, such as getInMemoryShuffleData.

getInMemoryShuffleData result will extends RPCResponse in your design?

GetMemoryShuffleDataResponse will extend Message.

Why do GetMemoryShuffleDataResponse need extend Message?

Seems every Response type will have status and retMsg attr, it can extend RPCResponse, my mistake.

leixm · 2023-03-29T08:50:32Z

We better add some UTs.

jerqi · 2023-03-31T03:11:01Z

We better add some UTs.

Spark don't have UTs, too. Maybe we can use Integration tests to guarantee the correctness when other prs were merged.

leixm · 2023-03-31T03:14:00Z

We better add some UTs.

Spark don't have UTs, too. Maybe we can use Integration tests to guarantee the correctness when other prs were merged.

Ok.

jerqi · 2023-04-01T03:03:06Z

@xumanbu There is one comment left. #771 (comment) Could you address it?

xumanbu · 2023-04-03T03:26:34Z

@xumanbu There is one comment left. #771 (comment) Could you address it?

My mistake.done.

jerqi

LGTM, merged. thanks all

@xumanbu

…#771) ### What changes were proposed in this pull request? 1. add for netty rpc client TransportClient 2. TransportClientFactory for connection pool 3. TransportContext contains the context to create a TransportClientFactory, setup Netty Channel pipelines with a TransportResponseHandler 4. TransportConf for netty transport config create by RssConf ### Why are the changes needed? Fix: apache#584 ### Does this PR introduce _any_ user-facing change? add client configurations and add the ability to reuse netty clients. Todo: update the user documentation after the netty feature is completed @xumanbu ### How was this patch tested? local test Co-authored-by: jam.xu <jam.xu@vipshop.com>

[apache#584] transport client pool for netty

42b4e2b

smallzhongfeng changed the title ~~[#584] transport client pool for netty~~ [#584] feat(netty): transport client pool for netty Mar 27, 2023

fix bug

4d3b3cd

jerqi requested a review from smallzhongfeng March 27, 2023 16:54

jerqi reviewed Mar 28, 2023

View reviewed changes

common/src/main/java/org/apache/uniffle/common/netty/client/TransportContext.java Outdated Show resolved Hide resolved

jerqi requested a review from leixm March 28, 2023 01:58

jerqi changed the title ~~[#584] feat(netty): transport client pool for netty~~ [#584] feat(netty): Add transport client pool for netty Mar 28, 2023

jerqi reviewed Mar 28, 2023

View reviewed changes

common/src/main/java/org/apache/uniffle/common/netty/client/TransportClientFactory.java Outdated Show resolved Hide resolved

common/src/main/java/org/apache/uniffle/common/netty/client/TransportClient.java Show resolved Hide resolved

xumanbu and others added 2 commits March 28, 2023 15:38

fix THROWS_METHOD_THROWS_RUNTIMEEXCEPTION

9a61278

Merge branch 'apache:master' into netty-connection-pool

2c4b7ee

jerqi reviewed Mar 28, 2023

View reviewed changes

common/src/main/java/org/apache/uniffle/common/netty/client/TransportClientFactory.java Outdated Show resolved Hide resolved

Update common/src/main/java/org/apache/uniffle/common/netty/client/Tr…

bd9c2a7

…ansportClientFactory.java

jerqi previously approved these changes Mar 28, 2023

View reviewed changes

remove MessageDecoder& /MessageEncoder

6acf884

xumanbu dismissed jerqi’s stale review via 6acf884 March 28, 2023 09:38

fix EncoderAndDecoderTest

4ef2706

smallzhongfeng reviewed Mar 28, 2023

View reviewed changes

advancedxy reviewed Mar 28, 2023

View reviewed changes

Update common/src/main/java/org/apache/uniffle/common/netty/client/Tr…

b37bb74

…ansportClientFactory.java make sense Co-authored-by: advancedxy <xianjin@apache.org>

leixm requested changes Mar 29, 2023

View reviewed changes

xumanbu added 4 commits March 31, 2023 15:59

fix: opt tools use

f6df780

add TransportClientTest

425bc8d

fix:merge

1192150

fix:checkstyle

3dde8d3

jerqi approved these changes Apr 3, 2023

View reviewed changes

jerqi merged commit bc9aaaa into apache:master Apr 3, 2023

[#584] feat(netty): Add transport client pool for netty #771

[#584] feat(netty): Add transport client pool for netty #771

Conversation

xumanbu commented Mar 27, 2023 • edited Loading

What changes were proposed in this pull request?

Why are the changes needed?

Does this PR introduce any user-facing change?

How was this patch tested?

codecov-commenter commented Mar 27, 2023 • edited Loading

Codecov Report

leixm commented Mar 28, 2023

jerqi left a comment

Choose a reason for hiding this comment

advancedxy commented Mar 28, 2023

xumanbu commented Mar 28, 2023

xumanbu commented Mar 28, 2023

advancedxy commented Mar 28, 2023

xumanbu commented Mar 28, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

advancedxy Mar 28, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

leixm commented Mar 29, 2023

jerqi commented Mar 31, 2023

leixm commented Mar 31, 2023

jerqi commented Apr 1, 2023 • edited Loading

xumanbu commented Apr 3, 2023

jerqi left a comment

Choose a reason for hiding this comment

xumanbu commented Mar 27, 2023 •

edited

Loading

codecov-commenter commented Mar 27, 2023 •

edited

Loading

advancedxy Mar 28, 2023 •

edited

Loading

jerqi commented Apr 1, 2023 •

edited

Loading