Unix.select fast path misses events if the list of fds is greater than FD_SETSIZE (typically 64) #7276
Original bug ID: 7276
On Windows there's a problem with the Unix.select fast path used when all
With the default FD_SETSIZE of 64, if the sockets list is of length 65+
A possible fix appears to be to fall back to the slow path for lists of fds greater
diff --git a/otherlibs/win32unix/select.c b/otherlibs/win32unix/select.c
This is similar to "#5563: harden Unix.select against file descriptors
Steps to reproduce
On Windows (I'm using the cygwin based installer from https://fdopen.github.io/opam-repository-mingw/)
ocamlfind ocamlopt -package unix -linkpkg -o test.exe test.ml
For me the large Unix.select will time out and the program prints "ERROR" and exits with code 1. On OSX and on Windows with the patch above applied, it prints "OK" and exits with code 0.
I encountered this when filtering all network connections from a VM through the Mirage TCP/IP stack and out into the Internet via regular sockets. Occasionally the I/O would stop (but other Lwt timer threads continue), and then after some timeout it would wake up again. I believe this is because the list of active socket connections was > 64 and all the activity was on the fds at the end of the list.
Comment author: @alainfrisch
Can you confirm the following:
Comment author: djs55
Many thanks for resolving this! For the Changes file could you put my real name "David Scott"?
Regarding your two statements, the Microsoft docs for select say:
I'm not enough of a Unix expert to confirm your second statement ("FD_SETSIZE gives an upper bound to fd values that can go into an fd_set") but I think that is correct. I believe Unix-like systems all use low integers as file descriptors (unlike Windows which uses opaque handles, probably pointer values). I believe select on Unix has traditionally used simple bitmaps where including fd "n" means setting bit "n" to 1. Therefore I think on Unix people are more concerned with ensuring the bitmaps are large enough to represent the highest-possible fd value (and the performance implications of frequently scanning large bitmaps).