Connection timed out

[sm2017]

@walkor In distributed deployment I see the following error when I start new gateways in a new machine

stream_socket_client(): unable to connect to tcp://10.10.10.50:2320 (Connection timed out)
/vendor/workerman/gateway-worker/src/Lib/Gateway.php:363

10.10.10.50:2320 is a gateway in another machine and the load is very low

When I start new gateway in new machine , all business workers in all machines hangs for 10-15 seconds
I think we have stream_socket_client in business worker too and it's in synchronous mode

[walkor]

Please open /etc/sysctl.conf and add these

net.core.somaxconn = 65535
net.ipv4.tcp_max_syn_backlog = 262144
net.core.netdev_max_backlog = 30000
net.ipv4.tcp_tw_recycle = 0
fs.file-max = 6815744
net.netfilter.nf_conntrack_max = 2621440

settings , then run command sysctl -p.
Finally restart gatewayWorker.

[sm2017]

1- Is establishing connection between register , business workers and gateways in synchronous mode ?
2- Why I have 600K total_request in my register? When a request send to register? I think just in business worker and gateways starts (currently I have 150 gateways and 150 business worker and I know that just few business worker exits per day on restart them selves , less than 10 per day) , So why I have 600K request in register??

[sm2017]

I add settings in /etc/sysctl.conf

Now in start in have the following error

stream_socket_client(): unable to connect to tcp://10.10.10.10:2315 (Connection refused)
vendor/workerman/gateway-worker/src/Lib/Gateway.php:363

10.10.10.10:2315 is a gateway
Also I stop firewalld

[sm2017]

What about

stream_socket_client(): unable to connect to tcp://10.10.10.11:2315 (Operation now in progress)
vendor/workerman/gateway-worker/src/Lib/Gateway.php:363

[sm2017]

After add settings in /etc/sysctl.conf (Connection timed out) solved at startup but it occurred a few in runtimes

[walkor]

May be high load caused the problem.
I'm not sure.

[sm2017]

Who exactly can help me? Linux expert? Network expert?...

[sm2017]

@walkor still having stream_socket_client(): unable to connect to tcp://10.10.10.11:2321 (Connection timed out) issue

• Can you tell me how much is timeout?
• When you connect from a business worker to a gateway?

[walkor]

Default timeout is 3 seconds and can be modified by GatewayWorker\Lib\Gateway::$connectTimeout.
When you call methods Gateway::XXX like Gateway::isUidOnline Gateway:: isOnline Gateway:: getAllClientSessions Gateway::getClientSessionsByGroup Gateway:: getAllClientCount Gateway:: getClientCountByGroup Gateway:: getClientIdByUid Gateway::getSession the business worker will connect to gateway.

[sm2017]

@walkor is the connection keep a lived? Or it is disconnect and when needed it connected again?

[walkor]

it is disconnect and when needed it connected again.

[sm2017]

@walkor , How can I change the behavior?
As you know establishing TCP connection is expensive , I use sendToClient, sendToUid and sendToGroup a lot and it is better that connection keep a-lived during multi method calls

closing and establishing connection is not optimized

[walkor]

sendToClient, sendToUid and sendToGroup will not reconnect.

[MajidJafari]

For +300k concurrent connection:

Set these variables in /etc/sysctl.conf:

fs.file-max = 10000000 
fs.nr_open = 10000000

Also, change these variables in /etc/security/limits.conf:

* soft nofile 10000000
* hard nofile 10000000
root soft nofile 10000000
root hard nofile 10000000

And finally, increase TCP buffers in /etc/sysctl.conf, too:

net.ipv4.tcp_mem = 786432 1697152 1945728
net.ipv4.tcp_rmem = 4096 4096 16777216
net.ipv4.tcp_wmem = 4096 4096 16777216

for more information please refer to https://www.linangran.com/?p=547