How support 20K connections in server PC

[linjson]

I tested a 10K test connection, and then the server crashed. How does grpc-java set up to support more than 20K request connections on a PC

What version of gRPC are you using?

1.12.0

What did you expect to see?

How support 20K connections in server PC

[carl-mastrangelo]

@linjson Can you show us what error you got?

[linjson]

server error: io.grpc.netty.NettyServerTransport.notifyTerminated Transport failed
client error:UNKNOW

[linjson]

@carl-mastrangelo ,How do I test the number of connections tested by the GRPC server
My code:

for(int i=0;i<20000;i++){
	ManagedChannel channel = NettyChannelBuilder.forAddress("192.168.0.166", 9000).negotiationType(NegotiationType.PLAINTEXT).build();


	TestServiceGrpc.TestServiceStub stub = TestServiceGrpc.newStub(channel);
	final StreamObserver<TestRequest> client = stub.simpleTest(new ClientResponseObserver<TestRequest, TestResponse>() {
		...
	});


	TestRequest req = TestRequest.newBuilder().setParam(xx").build();
	client.onNext(req);
	Thread.sleep(50);
}

Is that right?

[carl-mastrangelo]

I would guess you would run out of file descriptors pretty quick with that. It would be very helpful if you included all the error detail you can.

[linjson]

How many connections can a PC support at most

[carl-mastrangelo]

Millions if tuned right. It's outside of scope for gRPC on how to do this. You will need to include more detail about the error for us to help.

[pgrosu]

@linjson This might be of interest to you: https://www.nginx.com/blog/nginx-1-13-10-grpc/

[linjson]

@carl-mastrangelo
Tell me how to correctly test the number of connections GRPC can withstand on a server. Maybe there was a problem with my previous test. My test environment was Windows_7_64

[carl-mastrangelo]

@linjson You need to provide the full stack trace of the error before we can help you.

[linjson]

io.grpc.StatusRuntimeException: UNKNOWN
	at io.grpc.Status.asRuntimeException(Status.java:526)
	at io.grpc.stub.ClientCalls$StreamObserverToCallListenerAdapter.onClose(ClientCalls.java:419)
	at io.grpc.PartialForwardingClientCallListener.onClose(PartialForwardingClientCallListener.java:39)
	at io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:23)
	at io.grpc.ForwardingClientCallListener$SimpleForwardingClientCallListener.onClose(ForwardingClientCallListener.java:40)
	at io.grpc.internal.CensusStatsModule$StatsClientInterceptor$1$1.onClose(CensusStatsModule.java:684)
	at io.grpc.PartialForwardingClientCallListener.onClose(PartialForwardingClientCallListener.java:39)
	at io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:23)
	at io.grpc.ForwardingClientCallListener$SimpleForwardingClientCallListener.onClose(ForwardingClientCallListener.java:40)
	at io.grpc.internal.CensusTracingModule$TracingClientInterceptor$1$1.onClose(CensusTracingModule.java:391)
	at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:471)
	at io.grpc.internal.ClientCallImpl.access$300(ClientCallImpl.java:63)
	at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.close(ClientCallImpl.java:553)
	at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.access$600(ClientCallImpl.java:474)
	at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:591)
	at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
	at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:744)
Caused by: io.netty.channel.ChannelException: Unable to create Channel from class class io.netty.channel.socket.nio.NioSocketChannel
	at io.netty.channel.ReflectiveChannelFactory.newChannel(ReflectiveChannelFactory.java:40)
	at io.netty.bootstrap.AbstractBootstrap.initAndRegister(AbstractBootstrap.java:320)
	at io.netty.bootstrap.AbstractBootstrap.register(AbstractBootstrap.java:234)
	at io.grpc.netty.NettyClientTransport.start(NettyClientTransport.java:222)
	at io.grpc.internal.ForwardingConnectionClientTransport.start(ForwardingConnectionClientTransport.java:32)
	at io.grpc.internal.ForwardingConnectionClientTransport.start(ForwardingConnectionClientTransport.java:32)
	at io.grpc.internal.InternalSubchannel.startNewTransport(InternalSubchannel.java:230)
	at io.grpc.internal.InternalSubchannel.obtainActiveTransport(InternalSubchannel.java:194)
	at io.grpc.internal.ManagedChannelImpl$SubchannelImpl.requestConnection(ManagedChannelImpl.java:1301)
	at io.grpc.PickFirstBalancerFactory$PickFirstBalancer.handleResolvedAddressGroups(PickFirstBalancerFactory.java:80)
	at io.grpc.internal.AutoConfiguredLoadBalancerFactory$AutoConfiguredLoadBalancer.handleResolvedAddressGroups(AutoConfiguredLoadBalancerFactory.java:87)
	at io.grpc.internal.ManagedChannelImpl$NameResolverListenerImpl$1NamesResolved.run(ManagedChannelImpl.java:1177)
	at io.grpc.internal.ChannelExecutor.drain(ChannelExecutor.java:73)
	at io.grpc.internal.ManagedChannelImpl$LbHelperImpl.runSerialized(ManagedChannelImpl.java:1129)
	at io.grpc.internal.ManagedChannelImpl$NameResolverListenerImpl.onAddresses(ManagedChannelImpl.java:1181)
	at io.grpc.internal.DnsNameResolver$1.run(DnsNameResolver.java:238)
	... 3 more
Caused by: java.lang.reflect.InvocationTargetException
	at sun.reflect.GeneratedConstructorAccessor4.newInstance(Unknown Source)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:408)
	at io.netty.channel.ReflectiveChannelFactory.newChannel(ReflectiveChannelFactory.java:38)
	... 18 more
Caused by: io.netty.channel.ChannelException: Failed to open a socket.
	at io.netty.channel.socket.nio.NioSocketChannel.newSocket(NioSocketChannel.java:68)
	at io.netty.channel.socket.nio.NioSocketChannel.<init>(NioSocketChannel.java:85)
	at io.netty.channel.socket.nio.NioSocketChannel.<init>(NioSocketChannel.java:78)
	... 22 more
Caused by: java.net.SocketException: Too many open files
	at sun.nio.ch.Net.socket0(Native Method)
	at sun.nio.ch.Net.socket(Net.java:393)
	at sun.nio.ch.Net.socket(Net.java:386)
	at sun.nio.ch.SocketChannelImpl.<init>(SocketChannelImpl.java:104)
	at sun.nio.ch.SelectorProviderImpl.openSocketChannel(SelectorProviderImpl.java:60)
	at io.netty.channel.socket.nio.NioSocketChannel.newSocket(NioSocketChannel.java:66)
	... 24 more

[carl-mastrangelo]

@linjson You can see that you have too many open files, looking at the error:

Caused by: java.net.SocketException: Too many open files

You need to increase the limit. If you are on Linux, you can check to see the max like this:

~$ ulimit -Hn
4096

~$ ulimit -Sn
1024

And you can set it like this:

~$ ulimit -Sn 4096

You will need at least 50,000 or so.

[ejona86]

Seems like this is answered; closing. @linjson, if it doesn't work you can comment here and we can re-open.