Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

read timeout is not always used for read timeout but for "max-packet-delay" #550

Closed
stbuehler opened this issue Jan 17, 2017 · 7 comments
Closed

Comments

@stbuehler
Copy link

Hi,

the documentation for pcap_set_timeout makes it sound like you could set a read timeout, so that pcap_next_ex() returns after that timeout whether any packets were received or not.

On linux (can't speak for the other platforms) it actually sets a different timer: it limits the maximum time a packet gets delayed in the buffer before pcap_next_ex() returns it. If there are no packets at all the "timeout" will never fire.

In other words: the TPACKET3 tp_retire_blk_tov is NOT a poll timeout.

It would be very nice to have a (separate) way to set a real poll timeout (preferably on an active handle, or as parameter to a new pcap_next_ex_with_timeout function), and the documentation should contain the possible interpretation of "read timeout" in pcap_set_timeout.

Afaict the recommended workarounds seem to be:

  1. Use non-blocking "immediate" mode and use your own event loop (poll, select): but libpcap will call poll() again (on linux at least) afaics, wasting syscalls.
  2. Have a timer in a separate thread, call pcap_breakloop when it expires and send the other thread some signal to wake it up.

PS: I hope I didn't miss this bug in one of the existing timeout issues :)

@guyharris
Copy link
Member

guyharris commented Jan 17, 2017

the documentation for pcap_set_timeout makes it sound like you could set a read timeout, so that pcap_next_ex() returns after that timeout whether any packets were received or not.

The documentation for pcap_set_timeout() says you can set a timeout called a "read timeout", so that something times out in some fashion.

The pcap(3PCAP) man page says, with rather a lot of detail intended to let users know that this timeout is NOT guaranteed to time out whether any packets were received or not:

  read timeout
        If,  when  capturing,  packets  are  delivered  as  soon as they
        arrive, the application capturing the packets will be  woken  up
        for  each  packet  as  it arrives, and might have to make one or
        more calls to the operating system to fetch each packet.

        If, instead, packets are not delivered as soon as  they  arrive,
        but are delivered after a short delay (called a "read timeout"),
        more than one packet can be accumulated before the  packets  are
        delivered,  so  that  a single wakeup would be done for multiple
        packets, and each set of calls  made  to  the  operating  system
        would  supply  multiple  packets,  rather  than a single packet.
        This reduces the per‐packet CPU overhead if packets are arriving
        at a high rate, increasing the number of packets per second that
        can be captured.

        The read timeout is required so that an application  won’t  wait
        for  the  operating  system’s  capture  buffer to fill up before
        packets are delivered; if packets are arriving slowly, that wait
        could take an arbitrarily long period of time.

        Not  all  platforms  support  a  read timeout; on platforms that
        don’t, the read timeout is ignored.  A zero value for the  time‐
        out, on platforms that support a read timeout, will cause a read
        to wait forever to allow enough packets to arrive, with no time‐
        out.

        NOTE:  the  read timeout cannot be used to cause calls that read
        packets to return within a limited period of time,  because,  on
        some  platforms, the read timeout isn’t supported, and, on other
        platforms, the timer doesn’t start until  at  least  one  packet
        arrives.   This  means that the read timeout should NOT be used,
        for example, in an interactive application to allow  the  packet
        capture loop to ‘‘poll’’ for user input periodically, as there’s
        no guarantee that a call reading packets will return  after  the
        timeout expires even if no packets have arrived.

        The read timeout is set with pcap_set_timeout().

It would be very nice to have a (separate) way to set a real poll timeout (preferably on an active handle, or as parameter to a new pcap_next_ex_with_timeout function),

What would the use for this feature be?

and the documentation should contain the possible interpretation of "read timeout" in pcap_set_timeout.

...or point to the pcap(3PCAP) section on the read timeout rather than just mentioning pcap(3PCAP) in the "SEE ALSO" section.

@stbuehler
Copy link
Author

The documentation for pcap_set_timeout() says you can set a timeout called a "read timeout", so that something times out in some fashion.

The pcap(3PCAP) man page says, with rather a lot of detail intended to let users know that this timeout is NOT guaranteed to time out whether any packets were received or not:
[...custom definition of "read timeout"...]

It should be obvious that calling something a "read timeout" which actually is NOT a read timeout is a very bad idea, and I still think the pcap_set_timeout() man page should define this more clearly; not necessarily in the long form from pcap(3PCAP), but more than just a "SEE pcap(3PCAP) for our fucked up definition of 'read timeout'"

It would be very nice to have a (separate) way to set a real poll timeout (preferably on an active handle, or as parameter to a new pcap_next_ex_with_timeout function),

What would the use for this feature be?

I'd like to print progress and/or log statistics at regular intervals, and I see no point going multithreading for that if there is a poll() call involved which can easily have a timeout.

I guess going for immediate mode and adding another poll() call would be acceptable in my use case (sadly the rust wrapper doesn't support immediate mode yet).

@guyharris
Copy link
Member

It should be obvious that calling something a "read timeout" which actually is NOT a read timeout is a very bad idea,

The name dates back at least to LBL's libpcap 0.4, but "packet buffer timeout" would make it clearer.

I'd like to print progress and/or log statistics at regular intervals, and I see no point going multithreading for that if there is a poll() call involved which can easily have a timeout.

There isn't necessarily a select() or poll() call involved, so that's not the right way to go here.

I guess going for immediate mode and adding another poll() call would be acceptable in my use case

Going for non-blocking mode and adding another select() or poll() call should work even better (as it preserves the buffering done with BPF and Linux TPACKET_v3). This is what would be done if, for example, a program were reading from a socket and printing progress and/or logging statistics - select() and poll() (and kqueues and epoll() and WaitForMultipleObjects() and so on) are good for handling multiple input sources, and the timeout in those calls can be through of as an additional input source, namely the clock.

@guyharris
Copy link
Member

I've changed the man pages in b4d719c.

@stbuehler
Copy link
Author

I would have preferred a more explicit mention of "please lookup what a packet buffer timeout is" in pcap_set_timeout.3pcap, but I guess this is ok too. Thanks for your work!

@guyharris
Copy link
Member

I would have preferred a more explicit mention of "please lookup what a packet buffer timeout is" in pcap_set_timeout.3pcap

How does

(See pcap(3PCAP) for an explanation of the packet buffer timeout.)

differ from the more explicit mention that you would have preferred?

ljfranklin added a commit to ljfranklin/pcap that referenced this issue Apr 2, 2020
- e.g. 'identifier "_Ctype_struct_pcap_stat" may conflict with
  identifiers generated by cgo' and 'Sprintf call needs 2 args but has 3
  args'
- also adds a 'break' in tests since pcap Next() seems to ignore read
  timeout on my system after all packets have been read (Linux)
  - the-tcpdump-group/libpcap#550
@infrastation
Copy link
Member

For posterity, there is now a FAQ entry about this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

No branches or pull requests

3 participants