NetfilterQueue() create socket fd and never release

[1ncande5cence]

Hi! Here is my code.

self.out_nfqueue = netfilterqueue.NetfilterQueue()
self.out_nfqueue.bind(self.out_queue_num,self.out_callback)
self.out_nfqueue_socket = socket.fromfd(self.out_nfqueue.get_fd(),socket.AF_UNIX,socket.SOCK_STREAM)
... 
# when I finish
self.out_nfqueue.unbind()
self.out_nfqueue_socket.close()

In line 1, Class NetfilterQueue will create a FileDescriptor in /proc/fd/ . But when I finish these code. this socket will not be released and still in /proc/fd/ . However the FileDescriptor created in line 3 will be released in line 7 nicely.
I need to call the above code many times. I discovered that the FileDescriptor created in line 1 can not be released correctly, which lead to crashes of opening too many filedescriptors ( filedescriptor out of range in select() ).

So how should I solve this problem? Thanks!

[DOWRIGHTTV]

in the class cinit there is support for passing in a fd that it will try to reuse.

so -> self.out_nfqueue = netfilterqueue.NetfilterQueue(sockfd=SOCK_FD)

[oremanj]

@1ncande5cence I am not seeing an FD leak:

>>> import os, netfilterqueue
>>> os.listdir("/proc/self/fd")
['0', '1', '2', '3']
>>> nfq = netfilterqueue.NetfilterQueue()
>>> os.listdir("/proc/self/fd")
['0', '1', '2', '3', '4']
>>> import socket
>>> sock = socket.fromfd(nfq.get_fd(), socket.AF_NETLINK, socket.SOCK_DGRAM)
>>> os.listdir("/proc/self/fd")
['0', '1', '2', '3', '4', '5']
>>> sock.close()
>>> os.listdir("/proc/self/fd")
['0', '1', '2', '3', '4']
>>> del nfq
>>> os.listdir("/proc/self/fd")
['0', '1', '2', '3']
>>>

Note that the FD is allocated in __init__() and is therefore closed in __del__(). You can do multiple sequences of bind()/unbind() on the same NetfilterQueue object reusing the same netlink file descriptor, so it would not be appropriate for unbind() to close the file descriptor. I can provide a close() method or context manager if you need it, but first can you make sure you have actually dropped all references to the NetfilterQueue object before you expect its fd to be closed? Like in your example try del self.out_nfqueue and see if that helps. Or, as I said, you could reuse the same NetfilterQueue object for all of your operations, binding it and unbinding it to different queues as needed.

@DOWRIGHTTV that option doesn't change the FD lifecycle, it just lets you supply an FD instead of allocating one inside __init__. It will still be notionally owned by the NetfilterQueue object and closed when the NetfilterQueue is destroyed, so it would not be correct to pass the same FD to multiple NetfilterQueues, as it would then be closed multiple times when those queues were freed (which would potentially close unrelated files that were opened after the first queue was freed and reused that now-available FD).