proposal: x/sys/unix: add EpollWaitBlock function to handoff p asap.

[jayantxie]

Background

To support passive detection of connection active state for net.Conn, I attempted to create an independent epoll loop to monitor connection closure signals, thereby setting corresponding flags to extend functionality beyond the net package. This interface will significantly reduce the number of goroutines in multi-connection pool scenarios while minimizing the overhead of asynchronous reader/writer calls.

The codes is here: https://github.com/cloudwego/gopkg/pull/49/files.

However, when finalizing the feature submission, I encountered a tricky issue. I discovered that calling the syscall.EpollWait function is a blocking system call (as connection closures are low-frequency events), which occupies a P until sysmon detects it and hands off the P.

To avoid the latency impact of occupying a P, it is necessary to call runtime.entersyscallblock, but it is an unexported function.

Proposal

Therefore, I propose adding an EpollWaitBlock syscall to the src/syscall package, which would call entersyscallblock to immediately hand off the P.

Associated issue

#57103

[ianlancetaylor]

If we make a change here, I would instead suggest changing all system calls that will typically block to use entersyscallblock.

This change should be to the x/sys/unix package (and perhaps the x/sys/windows package). I don't see a clear reason to change the syscall package.

[jayantxie]

Currently, only the syscall package provides the EpollWait function, while the x/sys/unix package does not. Do you suggest we modify syscall.EpollWait in-place to call entersyscallblock, or add a new EpollWait into x/sys/unix?

[ianlancetaylor]

What platform are you using? I see EpollWait in golang.org/x/sys/unix.

[jayantxie]

Oh so sorry, I misread.

[jayantxie]

If EpollWait is defaultly changed to call entersyscallblock, it may cause pollers in libraries like netpoll ( sorry for both from cloudwego :( ) to re-acquire P every time, leading to a decrease in scheduling priority, which in turn affects the latency of request processing.

However, for the connstate module I implemented, epoll_wait is likely to block, and the decrease in poller priority does not have a substantial impact, because the scenario of determining connection active state is not sensitive to latency.

This is indeed a trade-off, so I still prefer to add an interface like EpollWaitBlock. If that's not possible, I'll consider implementing the poller in a C thread, like in this PR: https://github.com/cloudwego/gopkg/pull/52/files#diff-b08595bcb80b68a403100260380937c6309c49eff1121b5150be5bd32f9d378f