File Descriptor Leak (can't identify protocol)

[ysbaddaden]

Prax is leaking File Descriptors on each request. Here is the result of an lsof after a few requests:

COMMAND     PID   USER   FD   TYPE             DEVICE SIZE/OFF    NODE NAME
prax-bina 27652 julien  cwd    DIR               0,36     4096 8655908 /home/julien/work/github/prax.cr
prax-bina 27652 julien  rtd    DIR                8,9     4096       2 /
prax-bina 27652 julien  txt    REG               0,36  1602014 8784921 /home/julien/work/github/prax.cr/bin/prax-binary
prax-bina 27652 julien  mem    REG                8,9  1840928 2101998 /lib/x86_64-linux-gnu/libc-2.19.so
prax-bina 27652 julien  mem    REG                8,9    90080 2101109 /lib/x86_64-linux-gnu/libgcc_s.so.1
prax-bina 27652 julien  mem    REG                8,9    14664 2101987 /lib/x86_64-linux-gnu/libdl-2.19.so
prax-bina 27652 julien  mem    REG                8,9   141574 2101999 /lib/x86_64-linux-gnu/libpthread-2.19.so
prax-bina 27652 julien  mem    REG                8,9    31792 2102001 /lib/x86_64-linux-gnu/librt-2.19.so
prax-bina 27652 julien  mem    REG                8,9  1930528 2097469 /lib/x86_64-linux-gnu/libcrypto.so.1.0.0
prax-bina 27652 julien  mem    REG                8,9   387256 2097470 /lib/x86_64-linux-gnu/libssl.so.1.0.0
prax-bina 27652 julien  mem    REG                8,9   149120 2101995 /lib/x86_64-linux-gnu/ld-2.19.so
prax-bina 27652 julien    0u   CHR             136,12      0t0      15 /dev/pts/12
prax-bina 27652 julien    1u   CHR             136,12      0t0      15 /dev/pts/12
prax-bina 27652 julien    2u   CHR             136,12      0t0      15 /dev/pts/12
prax-bina 27652 julien    3u  0000                0,9        0    7345 anon_inode
prax-bina 27652 julien    4u  unix 0x0000000000000000      0t0 1403824 socket
prax-bina 27652 julien    5u  unix 0x0000000000000000      0t0 1403825 socket
prax-bina 27652 julien    6r  FIFO                0,8      0t0 1403826 pipe
prax-bina 27652 julien    7w  FIFO                0,8      0t0 1403826 pipe
prax-bina 27652 julien    8u  sock                0,7      0t0 1403827 can't identify protocol
prax-bina 27652 julien    9u  IPv6            1403828      0t0     TCP *:20559 (LISTEN)
prax-bina 27652 julien   10u  sock                0,7      0t0 1403829 can't identify protocol
prax-bina 27652 julien   11u  IPv6            1403830      0t0     TCP *:20558 (LISTEN)
prax-bina 27652 julien   13u  sock                0,7      0t0 1406150 can't identify protocol
prax-bina 27652 julien   14u  sock                0,7      0t0 1406152 can't identify protocol
prax-bina 27652 julien   15u  sock                0,7      0t0 1406154 can't identify protocol
prax-bina 27652 julien   16u  sock                0,7      0t0 1406156 can't identify protocol
prax-bina 27652 julien   17u  sock                0,7      0t0 1406158 can't identify protocol
prax-bina 27652 julien   18u  sock                0,7      0t0 1406160 can't identify protocol
prax-bina 27652 julien   19u  sock                0,7      0t0 1406162 can't identify protocol
prax-bina 27652 julien   20u  sock                0,7      0t0 1406164 can't identify protocol
prax-bina 27652 julien   21u  sock                0,7      0t0 1405310 can't identify protocol
prax-bina 27652 julien   22u  sock                0,7      0t0 1405315 can't identify protocol
prax-bina 27652 julien   23u  sock                0,7      0t0 1405319 can't identify protocol
prax-bina 27652 julien   24u  sock                0,7      0t0 1405324 can't identify protocol
prax-bina 27652 julien   25u  sock                0,7      0t0 1405328 can't identify protocol
prax-bina 27652 julien   26u  sock                0,7      0t0 1405332 can't identify protocol
prax-bina 27652 julien   27u  sock                0,7      0t0 1407060 can't identify protocol
prax-bina 27652 julien   28u  sock                0,7      0t0 1405374 can't identify protocol
prax-bina 27652 julien   29u  sock                0,7      0t0 1405378 can't identify protocol
prax-bina 27652 julien   30u  sock                0,7      0t0 1408064 can't identify protocol
prax-bina 27652 julien   31u  sock                0,7      0t0 1404805 can't identify protocol
prax-bina 27652 julien   32u  sock                0,7      0t0 1404808 can't identify protocol

When making curl requests one by one, Prax leaks a sock but debug shows that TCPSocket reuses the same file descriptor (eg: 12), indicating that it may not be Server#handle_client that's leaking.

[ysbaddaden]

After some more investigation:

• requesting localhost or a public file doesn't leak any fd;
• starting or restarting an application leaks fds (sock 0,7 can't identify protocol);
• proxying a request to a rack application leaks 1 fd (sock 0,7 can't identify protocol);
• forwarding a request to a port leaks 1 fd (sock 0,7 can't identify protocol);

I have a feeling the problem lies in Application#connect which is used in both cases that are leaking fds, and never in cases where there are no leak. The problem is that the opened TCPSocket are correctly closed...

[ysbaddaden]

Fixed by 3b91e60 until Crystal fixes the problem.