Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix FreeBSD build issues #1074

Closed
wants to merge 2 commits into from
Closed

Conversation

kprovost
Copy link

@kprovost kprovost commented Dec 13, 2021

Recent FreeBSD versions started including <net/bpf.h> from <net/if_pflog.h>, which triggered two distinct build issues.

kprovost added 2 commits Dec 13, 2021
We carefully check for various operating system include guards to ensure
that we include either the OS header or our own.
This works fine if the OS header is included first, but if we include
our own first and then the OS header we run into duplicate definitions.

This fixes part of the build issues caused by recent FreeBSD including
<net/bpf.h> from <net/if_pflog.h>.
New FreeBSD versions includes <net/bpf.h> from <net/if_pflog.h>, and
indirectly include <net/dlt.h>. The FreeBSD version lacks
DLT_IEEE802_15_4_TAP, so we really want to use our own version.
freebsd-git pushed a commit to freebsd/freebsd-ports that referenced this issue Dec 22, 2021
Fix build issues since if_pflog.h added a net/bpf.h include

Obtained from:	the-tcpdump-group/libpcap#1074
Sponsored by:	Rubicon Communications, LLC ("Netgate")
@guyharris
Copy link
Member

guyharris commented Jan 25, 2022

The change that did that was:

commit 6d4baa0d011cb3e78b4b08415568e71c0aab00fe
Author: Kristof Provost <kp@FreeBSD.org>
Date:   Thu Dec 2 08:22:34 2021 +0100

    if_pflog: fix packet length
    
    There were two issues with the new pflog packet length.
    The first is that the length is expected to be a multiple of
    sizeof(long), but we'd assumed it had to be a multiple of
    sizeof(uint32_t).
    
    The second is that there's some broken software out there (such as
    Wireshark) that makes incorrect assumptions about the amount of padding.
    That is, Wireshark assumes there's always three bytes of padding, rather
    than however much is needed to get to a multiple of sizeof(long).
    
    Fix this by adding extra padding, and a fake field to maintain
    Wireshark's assumption.
    
    Reported by:    Ozkan KIRIK <ozkan.kirik@gmail.com>
    Tested by:      Ozkan KIRIK <ozkan.kirik@gmail.com>
    MFC after:      1 week
    Sponsored by:   Rubicon Communications, LLC ("Netgate")
    Differential Revision:  https://reviews.freebsd.org/D33236

Note that what Wireshark is doing wrong is bothering to even try to dissect LINKTYPE_PFLOG; it's not a good LINKTYPE_ value, as it doesn't have a single specification, it's "whatever the OS that wrote the file decided to use, subject to change without notice". If it were up to me, at this point, I'd rip the code for it out of tcpdump and Wireshark and leave it up to particular OSes to add in dissection appropriate for the version of the OS with which they're packaging Wireshark in their packaging system.

Note: any file format or packet format that includes a long is broken unless it contains sufficient information to indicate how long a long is. This includes packet formats in capture files.

(THIS is why, when somebody proposes a new LINKTYPE_ value, I try to get them to nail down all the details; I want to give them as little freedom as possible to change things. Every time you require a packet dissector to use a heuristic to dissect a link-layer type, God kills a kitten. Please, think of the kittens.)

So why did this change have to include net/bpf.h?

@infrastation
Copy link
Member

infrastation commented Jan 26, 2022

(See also the-tcpdump-group/tcpdump#587.)

@kprovost
Copy link
Author

kprovost commented Jan 26, 2022

6d4baa0d011cb3e78b4b08415568e71c0aab00fe includes net/bpf.h because it now uses BPF_WORDALIGN() in PFLOG_HDRLEN.

@guyharris
Copy link
Member

guyharris commented Jan 29, 2022

6d4baa0d011cb3e78b4b08415568e71c0aab00fe includes net/bpf.h because it now uses BPF_WORDALIGN() in PFLOG_HDRLEN.

Given that struct pfloghdr is written to files that may be read on other machines, the only sane definition of it is one in which the size and layout of the structure is the same on all machines.

None of the data types in the structure are long, so none of them either 1) require alignment greater than 32-bit alignment or 2) have a size that depends on whether your programming environment is ILP32 or LP64 (FreeBSD doesn't use other environments such as LLP64).

Thus, the only way in which the size and layout of the structure would differ between ILP32 and LP64 environments would be if it were to be aligned on a 32-bit boundary in an ILP32 environment and a 64-bit boundary in an LP64 environment, as that would require its size to be a multiple of 8 in an LP64 environment but only a multiple of 4 in an ILP32 environment.

As long as 1) neither IF_NAMESIZE nor PFLOG_RULESET_NAME_SIZE change and 2) neither uid_t nor pid_t are extended to be 64-bit, the size and layout of the structure will be fixed, and the padding require to align it on an 8-byte boundary on all platforms will always be 4 bytes.

So:

  1. It's a good thing that Wireshark forced you to add the padding, to force the structure to be the same on ILP32 and LP64 platforms, so that it's a sane header for use in files such as pcap and pcapng files.
  2. sizeof(struct pfloghdr) should be sufficient, as the size should be the same on all platforms.

Furthermore:

  1. BPF_WORDALIGN() rounds the size up to a multipel fo 4; given that the structure is not "packed", and has uint32_t members, is there any reason why the size would ever not be a multiple of 4?
  2. Also,BPF_WORDALIGN() doesn't round it up to a multiple of 8, so I'm not sure what purpose it's serving here.

So I don't see that it's necessary.

If there's still anything compiler-dependent or ILP32 vs. LP64-dependent about the structure, that needs to be fixed, so that all LINKTYPE_PFLOG files using the structure look Exactly. The. Same. Reardless. Of Whether. They're. Written. By. An. ILP32. Host. Or. An. LP64. Host.

While we're at it, if there will ever be a reason why that structure might change, whether it's a structure member being added or removed, a structure member having its size change (including array-size changes), a change to the numerical values of AF_ definitions (probably unlikely), or a change to the numerical values for action or reason, and it cannot be ensured that the structure layout can always be determined based on the length field value, then, unless every such change will result in a new LINKTYPE_ value being assigned, there should be a field in the structure specifying a version number for the structure.

If no changes to the numerical values in question, and all changes to the structure result in an increase in the structure size, the header field will suffice.

And, given that struct pfloghdr in current FreeBSD is different from struct pfloghdr in:

  • current DragonFly BSD;
  • current NetBSD;
  • current OpenBSD;
  • macOS as of xnu-7195.50.7.100.1;

how about, when we finally have a FreeBSD struct pfloghdr that will not change in ways that require either 1) a new LINKTYPE_ or 2) heuristics to allow the structure format to be determined, we add a new LINKTYPE_FREEBSD_PFLOG, and we can have all software that reads pcap or pcapng files with LINKTYPE_FREEBSD_PFLOG able to read all such files from all current or past versions of software that generates them and able to be modified to handle all future versions - and make it possible to write programs to read those formats without having to include if_pflog.h from the OS.

(Then we can work on handling the other versions, and ensuring that they don't require any more than one LINKTYPE_ value per OS or any heuristics - and not require including if_pflog.h.)

guyharris added a commit that referenced this issue Jan 30, 2022
Don't rely on the OS's pflog include files to give the pflog header
length; instead, fetch it from the length field of the header, and round
up to a multiple of 4.  That way, it'll handle headers for OSes other
than the one for which we're compiling code, and will handle different
header lengths for any given OS if, for example, a new version of the OS
adds more fields to the end.

Don't rely on the OS's pflog include files to define direction types,
reason types, action types, or the layout of the header; instead, define
them ourselves in a header of our own, with #ifs to select the ones that
are only on some platforms.  That way, it'll handle some fields and
field values (the ones common to all OSes with pflog) on all OSes, even
ones without pflog.

This should also clean up the FreeBSD build issues reported in #1074, as
we no longer include <net/if_pflog.h>.
@guyharris
Copy link
Member

guyharris commented Jan 30, 2022

This should be fixed by 2757ddc and 80dbee1.

@guyharris guyharris closed this Jan 30, 2022
guyharris added a commit that referenced this issue Feb 21, 2022
Don't rely on the OS's pflog include files to give the pflog header
length; instead, fetch it from the length field of the header, and round
up to a multiple of 4.  That way, it'll handle headers for OSes other
than the one for which we're compiling code, and will handle different
header lengths for any given OS if, for example, a new version of the OS
adds more fields to the end.

Don't rely on the OS's pflog include files to define direction types,
reason types, action types, or the layout of the header; instead, define
them ourselves in a header of our own, with #ifs to select the ones that
are only on some platforms.  That way, it'll handle some fields and
field values (the ones common to all OSes with pflog) on all OSes, even
ones without pflog.

This should also clean up the FreeBSD build issues reported in #1074, as
we no longer include <net/if_pflog.h>.

(cherry picked from commit 2757ddc)
ovsrobot pushed a commit to ovsrobot/dpdk that referenced this issue Mar 10, 2022
This is something caught in UNH FreeBSD env.

For some reason [1], the pcap/bpf.h header started to define _BPF_H_.

It happens that the bpf_impl.h internal DPDK header uses this define as
an internal guard.
This triggers a build failure in bpf_convert.c which can't find
RTE_BPF_LOG macro.

Fix the include guard to use the filename and remove _.

1: the-tcpdump-group/libpcap#1074

Fixes: 94972f3 ("bpf: add BPF loading and execution framework")
Cc: stable@dpdk.org

Signed-off-by: David Marchand <david.marchand@redhat.com>
Signed-off-by: 0-day Robot <robot@bytheb.org>
dpdk-replication pushed a commit to DPDK/dpdk that referenced this issue Mar 14, 2022
This is something caught in UNH FreeBSD env.

For some reason [1], the pcap/bpf.h header started to define _BPF_H_.

It happens that the bpf_impl.h internal DPDK header uses this define as
an internal guard.
This triggers a build failure in bpf_convert.c which can't find
RTE_BPF_LOG macro.

Fix the include guard to use the filename and remove _.

1: the-tcpdump-group/libpcap#1074

Fixes: 94972f3 ("bpf: add BPF loading and execution framework")
Cc: stable@dpdk.org

Signed-off-by: David Marchand <david.marchand@redhat.com>
Acked-by: Stephen Hemminger <stephen@networkplumber.org>
bluca pushed a commit to bluca/dpdk-stable that referenced this issue Mar 15, 2022
[ upstream commit 63f39a430a0a7a8b893ffbf88cd452dbc7b97c97 ]

This is something caught in UNH FreeBSD env.

For some reason [1], the pcap/bpf.h header started to define _BPF_H_.

It happens that the bpf_impl.h internal DPDK header uses this define as
an internal guard.
This triggers a build failure in bpf_convert.c which can't find
RTE_BPF_LOG macro.

Fix the include guard to use the filename and remove _.

1: the-tcpdump-group/libpcap#1074

Fixes: 94972f3 ("bpf: add BPF loading and execution framework")

Signed-off-by: David Marchand <david.marchand@redhat.com>
Acked-by: Stephen Hemminger <stephen@networkplumber.org>
kevintraynor pushed a commit to kevintraynor/dpdk-stable that referenced this issue Mar 16, 2022
[ upstream commit 63f39a430a0a7a8b893ffbf88cd452dbc7b97c97 ]

This is something caught in UNH FreeBSD env.

For some reason [1], the pcap/bpf.h header started to define _BPF_H_.

It happens that the bpf_impl.h internal DPDK header uses this define as
an internal guard.
This triggers a build failure in bpf_convert.c which can't find
RTE_BPF_LOG macro.

Fix the include guard to use the filename and remove _.

1: the-tcpdump-group/libpcap#1074

Fixes: 94972f3 ("bpf: add BPF loading and execution framework")

Signed-off-by: David Marchand <david.marchand@redhat.com>
Acked-by: Stephen Hemminger <stephen@networkplumber.org>
kevintraynor pushed a commit to kevintraynor/dpdk-stable that referenced this issue Mar 21, 2022
[ upstream commit 63f39a430a0a7a8b893ffbf88cd452dbc7b97c97 ]

This is something caught in UNH FreeBSD env.

For some reason [1], the pcap/bpf.h header started to define _BPF_H_.

It happens that the bpf_impl.h internal DPDK header uses this define as
an internal guard.
This triggers a build failure in bpf_convert.c which can't find
RTE_BPF_LOG macro.

Fix the include guard to use the filename and remove _.

1: the-tcpdump-group/libpcap#1074

Fixes: 94972f3 ("bpf: add BPF loading and execution framework")

Signed-off-by: David Marchand <david.marchand@redhat.com>
Acked-by: Stephen Hemminger <stephen@networkplumber.org>
gmuthukr pushed a commit to gmuthukr/dpdk that referenced this issue Apr 6, 2022
This is something caught in UNH FreeBSD env.

For some reason [1], the pcap/bpf.h header started to define _BPF_H_.

It happens that the bpf_impl.h internal DPDK header uses this define as
an internal guard.
This triggers a build failure in bpf_convert.c which can't find
RTE_BPF_LOG macro.

Fix the include guard to use the filename and remove _.

1: the-tcpdump-group/libpcap#1074

Fixes: 94972f3 ("bpf: add BPF loading and execution framework")
Cc: stable@dpdk.org

Signed-off-by: David Marchand <david.marchand@redhat.com>
Acked-by: Stephen Hemminger <stephen@networkplumber.org>
dmiller-nmap pushed a commit to nmap/libpcap that referenced this issue May 10, 2022
Don't rely on the OS's pflog include files to give the pflog header
length; instead, fetch it from the length field of the header, and round
up to a multiple of 4.  That way, it'll handle headers for OSes other
than the one for which we're compiling code, and will handle different
header lengths for any given OS if, for example, a new version of the OS
adds more fields to the end.

Don't rely on the OS's pflog include files to define direction types,
reason types, action types, or the layout of the header; instead, define
them ourselves in a header of our own, with #ifs to select the ones that
are only on some platforms.  That way, it'll handle some fields and
field values (the ones common to all OSes with pflog) on all OSes, even
ones without pflog.

This should also clean up the FreeBSD build issues reported in the-tcpdump-group#1074, as
we no longer include <net/if_pflog.h>.

(cherry picked from commit 2757ddc)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants