Does not support threaded operation

[strictlymike]

Hi,

It would be nice to be able to run this in a thread.

Right now, if I call run() or run_socket() in a thread (daemon or otherwise), the process deadlocks when I call the unbind() method. Omitting the unbind() method "works" but doesn't seem like a clean solution.

Using the non-blocking functionality of the run() method via the optional block argument only allows polling. This is inefficient because instead of waiting on netfilter queue I/O most of the time, the script would be waiting on its polling timer most of the time which would increase delay.

My work-around is to set a timeout on the socket and poll a stop flag whenever a timeout occurs:

def nfq_threadproc():
    while not stop:
        try:
            nfqueue.run_socket(s)
        except socket.timeout:
            pass

nfqueue = NetfilterQueue()
nfqueue.bind(1, handle_pkt)  # Definition omitted for brevity
s = socket.fromfd(nfqueue.get_fd(), socket.AF_UNIX, socket.SOCK_STREAM) # Taken the from example code
s.settimeout(1)
stop = False

t = threading.Thread(target=nfq_threadproc)
t.start()

time.sleep(5)  # Do something else here

stop = True
t.join()

It would be nice if this library used multiplexed I/O to efficiently permit threaded operation. This would entail using something like the select() library call (or something more modern like epoll(), at a cost to backward compatibility) instead of recv(). The interface could allow the user to specify the timeout interval via an optional run()/run_socket() argument or a new setter method. It could then check a flag at each iteration that would be set by the unbind() method to determine when another thread has initiated shutdown of the queue.

Let me know if I'm missing an easier way to accomplish my goal. Otherwise, I’d be happy to write up a gist or try my hand at a pull request for this if you agree it is a deficiency and is worth addressing.

[jllorente]

I commented this yesterday on another issue #28 (comment)
I wrote a wrapper for using the queue with asyncio, check it out if you are interested! Any feedback is welcome :)

[oremanj]

You can use netfilterqueue with select, epoll, or anything else you like, by calling get_fd() and registering that fd with your poller of choice for readability notifications. When the fd shows as readable, call run(False) to process all the pending data.

I'll leave this issue open to track the thread problem, which I haven't investigated yet.

[oremanj]

Right now, if I call run() or run_socket() in a thread (daemon or otherwise), the process deadlocks when I call the unbind() method.

If I'm understanding correctly, you're running run() in one thread but trying to interrupt it by calling unbind() in another? Yeah, that won't work. It is effectively impossible to interrupt a thread that's blocked in read(); not even closing the socket it's reading from works for that. I suggest you use get_fd() to obtain a file descriptor that you can poll for readability, and then run(False) when it shows up as readable. That way you can multiplex in another cross-thread signaling mechanism (such as a pipe) to get the cancellation behavior you're seeking.