Embed GPS data in individual packets via PCAPPPI

[dracode]

I was playing around with the GPS support and I think there's a relatively simple change that could make GPS data much more useful: tag each packet with the geo-coordinates where it was captured.

The .pcap and .pcapng format supports Per-Packet Information (PPI), which allows you to include additional meta-data about the packet. One of the supported types of metadata is GPS. This is supported in Scapy and Wireshark.
It's possible to have Kismet output PPI GPS data if you set log_types=kismet,pcapppi.
The full spec for this can be found here.

Having this data per-packet makes doing geo-filtering your packets trivially easy. Example:

$ tshark -r example_ppi_gps.pcapng -Y wlan.ssid -Tfields -e ppi_gps.lat -e ppi_gps.lon -e "_ws.col.Info" 
39.941125       -75.162175      Beacon frame, SN=1287, FN=0, Flags=........C, BI=100, SSID="DIRECT-53-HP ENVY Pro 6400"

A packet captured with Kismet's pcapppi:
example_ppi_gps.pcapng.zip

Thoughts?

[dracode]

I have written some Python that will construct this PPI header.
This is an excerpt from a larger program, so it won't run as-is, but I think it is sufficient to demonstrate the principle. I've tried to comment as much as possible to make it easy to understand.
This probably would be much simpler with Scapy, but the purpose here is to demonstrate the construction of the bytes that make up the PPI headers in a way that would be easy to port to C.

# PPI-GEOLOCATION spec says to store lat/longs as ints with a fixed 3 digits before the decimal and 7 after
def floatToFixed37(num):
    return int((num+180.0) * 10000000)

def floatToFixed64(num):
    return int((num+180000.0) * 10000)


def example(packetData):
        # Construct the PPI data fields
        ppiData = b''

        # Construct 802.11-Common PPI data header
        ppiRadioTap = b''
        ppiRadioTap += struct.pack('<HH', 2, 20) # 2 is the 802.11-Common tag, 20 is the length of the data
        ppiRadioTap += struct.pack('<QHHHHBBbb', tsfTimer, 0, dataRate, freq, 0, 0, 0, rssi, noise) # TSF Timer, Flags, Rate, Channel-Freq, Channel-Flags, FHSS-Hopset, FHSS-Pattern, dBm-Antsignal, dBm-Antnoise
        ppiData += ppiRadioTap

        # Construct the PPI-GEOLOCATION data fields
        gpsData = b''
        gpsFieldMask = 0
        if(self.currentGpsLat != None and self.currentGpsLon != None):
            gpsData += struct.pack('<I', floatToFixed37(self.currentGpsLat))
            gpsData += struct.pack('<I', floatToFixed37(self.currentGpsLon))
            gpsFieldMask |= 0b00000110 # Lat/Long fields present
        if(self.currentGpsAlt != None):
            gpsData += struct.pack('<I', floatToFixed64(self.currentGpsAlt))
            gpsFieldMask |= 0b00001000 # Altitude field present

        if(len(gpsData)):
            gpsHdr = struct.pack('<BBHI', 2, 0, 8+len(gpsData), gpsFieldMask) # base_geotag_header
            gpsData = gpsHdr + gpsData

            ppiData += struct.pack('<HH', 30002, len(gpsData)) # 30002 is the PPI Vendor Tag for PPI-GEOLOCATION
            ppiData += gpsData

        # Construct the PPI packet header
        ppiLen = 8 + len(ppiData)
        # PPI headers are always little-endian
        ppiPH = struct.pack('<BBHI', 0, 0, ppiLen, self.origDLT) # version, flags, length, encapsulated DLT

        # Trim the packet contents to keep everything within the valid snaplen
        if(ppiLen+capturedLen > self.snaplen):
            capturedLen = self.snaplen - ppiLen
            packetData = packetData[:capturedLen]

        # Write the new libpcap packet header
        self.outFH.write(timestamp)
        self.outFH.write(struct.pack('<II', ppiLen+capturedLen, ppiLen+origPacketLen))

        # Write the PPI header/data
        self.outFH.write(ppiPH)
        self.outFH.write(ppiData)

        # Write the packet contents
        self.outFH.write(packetData)

[ZerBea]

Code snippet is pcap only.

BTW:
Hcxdumptool is not a passive dumper. First and foremost is the interaction with the target.
We don't have the time (CPU cycles) to interact with the target and to process GPS data on the fly.

[ZerBea]

The .pcap and .pcapng format supports Per-Packet Information (PPI).

The links you have referenced describe PCAPPPI while hcxdumptool uses pcapng, only.
Please attach a link to a pcapng documentation describing an official pcapng GEO struct. I didn't find any information about it

My reference of the PCAP Next Generation (pcapng) Capture File Format:
https://github.com/IETF-OPSAWG-WG/draft-ietf-opsawg-pcap
https://www.ietf.org/staging/draft-tuexen-opsawg-pcapng-02.html#name-custom-options

[ZerBea]

I've analyzed the attached pcapng file.
It does not(!) contain a pcapng block containing GEAO information.
Instead it include an encapsulated PPI header follwed by a RADIOTAP header and IEEE 802.11 frames:

Frame 1: 507 bytes on wire (4056 bits), 507 bytes captured (4056 bits) on interface unknown, id 0
PPI version 0, 76 bytes
802.11 radio information
IEEE 802.11 Beacon frame, Flags: ........C
IEEE 802.11 Wireless Management

via hcxpcapngtool:

$ hcxpcapngtool example_ppi_gps.pcapng
...
link layer header type...................: DLT_PPI (192).
...

via capinfos:

$ capinfos example_ppi_gps.pcapng
...
Interface #0 info:
                     Encapsulation = Per-Packet Information header (97 - ppi)
...

When dissolving the encapsulation remains the pcapppi format:

$ tshark -r example_ppi_gps.pcapng -F pcap -w stripped.pcap
$ tshark -r stripped.pcap -Y wlan.ssid -Tfields -e ppi_gps.lat -e ppi_gps.lon -e "_ws.col.Info" 
39.941125	-75.162175	Beacon frame, SN=1287, FN=0, Flags=........C, BI=100, SSID="DIRECT-53-HP ENVY Pro 6400"
$ capinfos stripped.pcap
...
Interface #0 info:
                     Encapsulation = Per-Packet Information header (97 - ppi)

[ZerBea]

There is a pull request about this:
IETF-OPSAWG-WG/draft-ietf-opsawg-pcap#51
and an issue report:
IETF-OPSAWG-WG/draft-ietf-opsawg-pcap#48

[ZerBea]

I took a closer look at the pcapppi header:

PPI version 0, 76 bytes
    Version: 0
    Flags: 0x00
        .... ...0 = Alignment: Not aligned
        0000 000. = Reserved: 0x00
    Header length: 76
    DLT: 105
    GPS: Lat:39,941125  Lon:-75,162175  Alt:-22,000000 
        Header revision: 1
        Header pad: 0
        Header length: 32
        Present: 0x200000ae, Lat, Lon, Alt, GPStime, error_h, AppId
            .... .... .... .... .... .... .... ...0 = GPSFlags: False
            .... .... .... .... .... .... .... ..1. = Lat: True
            .... .... .... .... .... .... .... .1.. = Lon: True
            .... .... .... .... .... .... .... 1... = Alt: True
            .... .... .... .... .... .... ...0 .... = Alt-gnd: False
            .... .... .... .... .... .... ..1. .... = GPStime: True
            .... .... .... .... .... .... .0.. .... = fractime: False
            .... .... .... .... .... .... 1... .... = error_h: True
            .... .... .... .... .... ...0 .... .... = error_v: False
            .... .... .... .... .... ..0. .... .... = error_t: False
            ...0 .... .... .... .... .... .... .... = Description: False
            ..1. .... .... .... .... .... .... .... = AppId: True
            .0.. .... .... .... .... .... .... .... = AppData: False
            0... .... .... .... .... .... .... .... = Ext: False
        Latitude: 39,941125
        Longitude: -75,162175
        Altitude: -22
        GPSTimestamp: Feb 16, 2022 18:51:51.000000000 UTC
        Horizontal Error (m): 241
        Application Specific id: 0x00534c57
    802.11-Common
    Aggregation Extension

This is a horrible construct from days gone by, to handle additional GEO information what the pcap format can't do by default.
Even the kismet developers suggest to stop using pcapppi:
https://www.kismetwireless.net/docs/readme/logging/logging/

Unfortunately KISMET is using its own implementation which is not a standard:
https://www.kismetwireless.net/docs/dev/pcapng_gps/

[dracode]

Sorry about that. I told Wireshark to re-save the file as a pcapng but I didn't actually verify that it did so. That's annoying.

Yes, there are a few other ideas on how to embed GPS data in a pcap, but as you have noted with the Kismet example, they just invented their own format that despite having been published for nearly 3 years, no other tool supports. The other pcapng geo-data addition has been languishing since 2018 with no sign of actual adoption.
Although PPI has its flaws, it's still the only way to convey this data that is adopted by more than one single tool. That's why I think it's the right choice, unless/until one of the better standards is adopted -- but given their 3-5 year history, this does not seem likely to happen anytime soon.

Since Wireshark let me down, I have hand-crafted the example into an actual pcapng here:
actual_pcapng_ppi_gps.zip

[ZerBea]

Still old PCAPPPI style:

$ capinfos actual_pcapng_ppi_gps.pcapng
...
Capture application: Kismet
Number of interfaces in file: 1
Interface #0 info:
                     Name = wlan0
                     Encapsulation = Per-Packet Information header (97 - ppi)
                     Capture length = 65535
                     Time precision = microseconds (6)
                     Time ticks per second = 1000000
                     Number of stat entries = 0
                     Number of packets = 1
...

using the horrible PPI header:

Frame 1: 507 bytes on wire (4056 bits), 507 bytes captured (4056 bits) on interface wlan0, id 0
PPI version 0, 76 bytes
802.11 radio information
IEEE 802.11 Beacon frame, Flags: ........C
IEEE 802.11 Wireless Management

[ZerBea]

This is pure pcapng format as recorded by the interface:

Capture application: hcxdumptool 6.3.2-164-g86dc158
Number of interfaces in file: 1
Interface #0 info:
                     Name = wlp22s0f0u9u3
                     Encapsulation = IEEE 802.11 plus radiotap radio header (23 - ieee-802-11-radiotap)
                     Capture length = 1024
                     Time precision = nanoseconds (9)
                     Time ticks per second = 1000000000
                     Time resolution = 0x09
                     Number of stat entries = 0
                     Number of packets = 1

Please take a look at the capture length.
By this commit:
8a752a7
I've reduced it to 1024 (from former 65535) to save CPU cycles. The time saved would be nullified by adding an additional PCAPPPI header.

This is the current work flow:

receive ieee802.11 frame (inclusive RADIOTAP header) from interface and save it to pcapng file as it is
receive NMEA from GPS device and save it to pcapng file as it is

[ZerBea]

The PCAPPPI workflow:

receive ieee802.11 frame (inclusive RADIOTAP header)

receive NMEA from GPS device
- do a tag walk through all fields
- build GPS header, add revision number, calculate length and padding
- do floating mathematics and add the result to GPS data
- build PPI header, add version number, flags, calculate length
 
build pcapng block containing PCAPPPI header and ieee802.11 frame (inclusive RADIOTAP header)
save this block to pcapng file

[ZerBea]

BTW:
Although PPI has its flaws, it's still the only way to convey this data that is adopted by more than one single tool. That's why I think it's the right choice, unless/until one of the better standards is adopted -
I fully agree. That is the best solution for tools using pcap formats, because pcap can't handle this information.

With regard to pcapng it is a big step backwards.
Due to the limited pcap format (and ugly workarounds to compensate this limitations), I've removed the entire pcap stuff in 2018 (release of hcxdumpool 4.2.0).

[ZerBea]

Maybe somebody can add a hcxdumptool NMEA dissector to Wireshark similar to this one.
https://github.com/fkie-cad/maritime-dissector/blob/master/README.md
I don't have the time to start such a project.

[dracode]

Yes, a snaplen of 1k should be plenty for what this tool does.

If performance in real-time is at a premium, maybe it makes more sense to do the extra calculation after the fact. It could be done to post-process the .pcapng file with a Scapy script, parse out the NMEA sentence, then emit a new .pcapng with the PPI headers added.

[ZerBea]

Performance is by far the top priority and form always follows function.

[ZerBea]

To draw a price tag, I've added some time measurement to hcxdumptool:

		else if(events[i].data.fd == fd_gps)
			{
			clock_gettime(CLOCK_PROCESS_CPUTIME_ID, &tsin);
			process_nmea0183();
			clock_gettime(CLOCK_PROCESS_CPUTIME_ID, &tsout);
			printf("%ld nsec\n", tsout.tv_nsec - tsin.tv_nsec);
			}

Now hcxdumptool shows the time that has elapsed to process an entire NMEA0183 string:

$ sudo ./hcxdumptool --nmea_dev=/dev/ttyACM0 --nmea_pcapng
...
Initialize main scan loop...
30688 nsec
12564 nsec
2966 nsec
4589 nsec
2685 nsec
3546 nsec
2275 nsec
3456 nsec
17082 nsec
16691 nsec
12563 nsec
2775 nsec
2254 nsec
3577 nsec
3556 nsec
3596 nsec
2264 nsec
3456 nsec
11983 nsec
9909 nsec
1763 nsec
2946 nsec
2955 nsec
1813 nsec
2805 nsec
15850 nsec
4599 nsec
2624 nsec
3566 nsec
3337 nsec
2334 nsec
^C
0 ERROR(s) during runtime
117 Packet(s) captured by kernel
0 Packet(s) dropped by kernel
1 SHB written to pcapng dumpfile
1 IDB written to pcapng dumpfile
1 ECB written to pcapng dumpfile
39 EPB written to pcapng dumpfile
31 NMEA sentence(s) received from device
3 ECB NMEA written to pcapng dumpfile
7 GPWPL record(s) written to file

exit on sigterm

How about your PCAP PPI code?

[ZerBea]

Now measuring handling of NMEA0183 and handling of incoming ieee802.11 packets (on my fast coding machine):

$ sudo ./hcxdumptool --nmea_dev=/dev/ttyACM0 --nmea_pcapng
...
Initialize main scan loop...
24416 nsec (process ieee802.11)
26600 nsec process NMEA0183
7444 nsec (process ieee802.11)
17382 nsec process NMEA0183
2886 nsec process NMEA0183
14768 nsec process NMEA0183
2976 nsec process NMEA0183
1583 nsec process NMEA0183
2875 nsec process NMEA0183
12002 nsec process NMEA0183
2836 nsec process NMEA0183
2245 nsec process NMEA0183
2945 nsec process NMEA0183
3006 nsec process NMEA0183
2955 nsec process NMEA0183
2735 nsec process NMEA0183
2825 nsec process NMEA0183
11943 nsec (process ieee802.11)
4899 nsec (process ieee802.11)
2445 nsec (process ieee802.11)
82735 nsec (process ieee802.11)
26610 nsec (process ieee802.11)
4529 nsec (process ieee802.11)
88727 nsec (process ieee802.11)
10119 nsec (process ieee802.11)
40336 nsec (process ieee802.11)
42059 nsec (process ieee802.11)
9659 nsec (process ieee802.11)
4218 nsec (process ieee802.11)
36518 nsec (process ieee802.11)
10640 nsec (process ieee802.11)
17473 nsec (process ieee802.11)
17002 nsec (process ieee802.11)
...
^C

0 ERROR(s) during runtime
3274 Packet(s) captured by kernel
87 Packet(s) dropped by kernel
1 SHB written to pcapng dumpfile
1 IDB written to pcapng dumpfile
1 ECB written to pcapng dumpfile
272 EPB written to pcapng dumpfile
196 NMEA sentence(s) received from device
14 ECB NMEA written to pcapng dumpfile
54 GPWPL record(s) written to file

exit on sigterm

We are close to the limit.
I think you can imagine what happens on a Rapberry Pi Zero 1 while doing a tag walk through an entire NME0183 sentences, doing float mathematics, calculating a PCAP PPI header and wrap a pacapng EPB around this construct.

[ZerBea]

I forgot to mention.
Please take a look at this:
87 Packet(s) dropped by kernel
The kernel is at its limit, too and starts to drop packets. Monitor mode is very CPU time consuming.

[theweefies]

I have c code that i just provided to aircrack team for writing pcap ppi and radio tap headers. Dude, there is a way to do this that is simple, provides low-ish loss (ns), and also, geo data is important to some users.

[ZerBea]

You mentioned it: pcap ppi (not pcapng)

One of the great benefits of pcang is to use comment fields to store additional data related to the received frame.
Adding an additional ppi header and wrapping a pcapng header around this (horrible) construct is more than one step backwards.

The kismet way is much better, but unfortunately it is not official.
https://www.kismetwireless.net/docs/dev/pcapng_gps/#kismet-gps-block

If you need pcapng embedded GPS data this is the place to discuss it:
IETF-OPSAWG-WG/draft-ietf-opsawg-pcap#48

BTW:
airodump-ng is a passive dumper. If you accept a packet loss, you can add everything to it.

To make this absolutely clear: Hcxdumtool is designed to run on systems like a RaspBerry Pi Zero!
Once hcxdumptool got a request it must(!) respond just in time! If that doesn't happen the entire attack is unsuccessful!
And most importantly, in case of an attack against a CLIENT connected to an AP, it must respond (much) faster than the AP!!!

[theweefies]

Yeah, pcapng is definitely a more up-to-date solution. I just honestly like the simplicity of the older pcap ppi radiotap/geoheaders. And it is relatively simple to code in c if the data already exists and is being utilized already in the program in terms of gps data and radio interface measurements.

Again, you make great points IRT the critical nature of timing in the attack interaction between AP and CLIENT.

[ZerBea]

There is no real need to process GPS data on the fly.
Instead it is much faster to:
record wifi traffic on the fly
record GPS data on the fly (not necessary for it to be done on the same device - e.g. could be an external GPS tracker)

And when finished, sync timestamps of both records and merge data off-line (e,g. by tools like GPS Babel).
That's my goal.
I've already tested this (Raspberry Pi running hcxdumtool and Garmin eTrex 30x) - and it is working like a charm!

[theweefies]

You make some good points. There is an important distinction to be made between the purpose of hcxdumptool and airodump-ng. And i completely understand the necessity of speed and timing IRT what hcxdumptool is trying to accomplish.

I'm digging into the source of hcxdumptool today to familiarize myself today and trying to understand how data is processed and managed - that being said, you wrote this, so i will defer to your judgment for sure.

Your solution for a post-dump sync via timestamps is probably a better option for this suite, and most certainly will not affect timing.

[ZerBea]

Some other differences between hcxdumptool and airodump-ng/aireplay-ng

interactive vs passive
nsec vs usec time resolution 
pcapng vs cap
intelligent attacks (e.g. PMKID, AP-LESS) vs simple deauthentication attacks
BPF vs soft coded filter lists

The aircrack-ng suite is an amazing suite and easy to use especially for beginners.
I do not recommend beginners to use hcxdumptool/hcxtools.

[ZerBea]

I have c code that i just provided to aircrack team for writing pcap ppi and radio tap headers.
Can you please comment some time measurement results (nsec)?

[theweefies]

I'll try to do some tests with the airodump changes and see what shakes out, and I'll post them here, or can send them in direct message/email if still interested

[ZerBea]

Feel free to fork hcxdumptool and to add the PPI support.
Feel free too ask your questions - may I can help. You can get my mail via git API.

The GPS parts are wrapped by:

#ifdef HCXNMEAOUT
...
#381 

GPS read is done here:
https://github.com/ZerBea/hcxdumptool/blob/master/hcxdumptool.c#L2655

GPS pcapng block write is here:
https://github.com/ZerBea/hcxdumptool/blob/master/hcxdumptool.c#L1001

Sooner or later (I think sooner), I'm going to remove the entire GPS code from hcxdumptool in favor of the off-line solution mentioned above.
It is so easy (sync GPS track [gpx] and hcxdumptool timestamp [pcapng]) when doing this off-line.

Since 6.3.0 hcxdumptool is going back to the roots: discover weak points in 802.11 by doing fast attacks.

[theweefies]

Awesome, thanks for posting that man. I may try to work this anyway in a fork, and if its something you want to include for those that are okay with sacrificing the time/processing speed to have the ppi option, awesome, otherwise, no big deal.

Once i get it done, i'll post here. I've got another python script that you can run against cap and .gps and netxml logs and create the ppi headers with gps doing the time sync thing (with a user-modifiable threshold option), let me know if you're interested in that. Very willing to provide that, although i know there is a push to go pcapng versus old ppi.