net.socket has no setsockopt() method，and How to create a socket whose handle is not null

[dong-king]

What is the problem this feature will solve?

1. My scenario is that the TCP client read speed is slow, so I need to set SO_RCVBUF through setsockopt().
   According to this post, I have implemented setsockopt().
   But can Node.js directly expose the corresponding interface?

2. The normal TCP client implementation code is:

var conn = new net.Socket();  // conn._handle will be null
conn.connect(5555, '127.0.0.1');  // conn._handle will be not null

Only after the socket is connected, its handle is not null.

But because the setting of SO_RCVBUF needs to be before socket connect (client) or socket listen (server), so I do it with the following code:

const TCPWrap = process.binding('tcp_wrap');
const { TCP } = TCPWrap;
const handle  = new TCP(TCPWrap.constants.SOCKET);

var conn = new net.Socket({handle});
console.log(conn._handle)	=> print null
setSocketOption(conn._handle, SOL_SOCKET, SO_RCVBUF, 1024);
conn.connect(5555, '127.0.0.1');

I expected to get a socket with a non-null handle at line 4, but in fact, conn._handle is still null.
Is there any way to allocate the socket handle before the connect operation?

What is the feature you are proposing to solve the problem?

1. Provide setsockeopt() interface;
2. Provides a method to initialize its handle when instantiating net.Socket()

What alternatives have you considered?

No response

[bnoordhuis]

Libuv has uv_recv_buffer_size() and uv_send_buffer_size() functions that get and set SO_RCVBUF and SO_SNDBUF respectively.

They're already used in the dgram module and they could be exposed in the net module as well but I'm skeptical that solves your slow read speed problems. What makes you think it's that?

[dong-king]

They're already used in the dgram module and they could be exposed in the net module as well but I'm skeptical that solves your slow read speed problems. What makes you think it's that?

I will do flow control in the program to ensure that after one frame of data is processed, the next frame of data is read from the socket cache. But I don't want the cached data to be too large. If SO_RCVBUF is not set, up to 1GB of data may be cached.

[bnoordhuis]

I briefly skimmed the linked article. How do you arrive at that 1 GB?

Perhaps Windows is in a class of its own but on Linux and most other Unices the operating system simply scales the send/receive buffers as needed (both up and down.)

I'm not saying there's never a reason to configure them manually, just that there's usually no reason to do so because you're unlikely to outperform the kernel.

And again, what makes you think it's going to fix your slow read speed problem? Configuring a small buffer will ceteris paribus make it worse, not better.

I mean, you're welcome to open a pull request and see how it's received, but it probably isn't the solution to your problem.

[dong-king]

@bnoordhuis
The tcp window size and window size scaling factor of the tcp receiving end affect the tcp sending speed.
If I call the setsockopt function on the tcp receiving end to set the SO_RCVBUF parameter, it will modify the tcp window size and window size scaling factor.

The following is the python code, which calls setsockopt to set the tcp window size and scale factor.
···
import socket
import time

HOST = '127.0.0.1' # The remote host
PORT = 50007 # The same port as used by the server

rcvbufsize = 1024*2
total = 0

with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
s.setsockopt(socket.SOL_SOCKET, socket.SO_RCVBUF, rcvbufsize)
s.connect((HOST, PORT))
s.sendall(b'Hello, world')
while True:
data = s.recv(size)
time.sleep(1)
total = total + len(data)
print('R', total)
···

When the socket establishes a connection, the client socket tells the server that its tcp window size is 2048 and the scale factor is 1. The following is a screenshot of wireshark:

If I do not set setsockopt, then the tcp window size is 10233 and the scale factor is 256:

In fact, my expectation is that node can provide the setsockopt() interface. Many programming languages ​​provide setsockopt interface.
In addition, the handle can be assigned to the socket when the socket is created, instead of when the socket is connected (the handle of the socket obtained by new net.Socket() is null). Because some socket settings need to be configured before connecting, otherwise the settings will not take effect.

[poorren]

Perhaps Windows is in a class of its own but on Linux and most other Unices the operating system simply scales the send/receive buffers as needed (both up and down.)

I'm not saying there's never a reason to configure them manually, just that there's usually no reason to do so because you're unlikely to outperform the kernel.

And again, what makes you think it's going to fix your slow read speed problem? Configuring a small buffer will ceteris paribus make it worse, not better.

I have the same desire to control the buffer size tcp stack.
My reason to have that is the buffered data led to huge delay on two sides of tcp.
The data sender collects enviroment image periodically, yes, it's images. The receiver side consumes very slow for those slow image processing operation. The operation result is serveral seconds or even tens of seconds latency. It seems that window system will increase the buffer gradually, the sender always has chance to send image, the sent data stay in tcp stack until many seconds later. I couldn't figure any easy way to solve that. Do you have any suggestion on that?

[bnoordhuis]

Either of you is welcome to open a pull request. If you can provide before/after benchmark numbers that strengthen your case, that'd definitely help.

my expectation is that node can provide the setsockopt() interface

As a rule of thumb node doesn't expose platform-specific functionality. Networking is handled by libuv, node doesn't deal with sockets directly.

[bnoordhuis]

Closing, no follow-up. Pull request still welcome.