Segmentation Fault when launching drone

[mcmillanda]

Getting a seg fault when first launching drone on a Balena Fin running Raspbian Buster. It was compiled natively. Based on debug output shown below it seg faults after cap file creation. File is created in /tmp file with no content.

pi@raspberrypi:~ $ sudo drone -d
Version: 1.0
Revision: 2966884
PCAP Lib: libpcap version 1.8.1
Retrieving the device list from the local machine

0. wwan0
   txStats: collection start
   cap file = /tmp/Drone.GSNvjm
   Segmentation fault

[jarednm]

I've encountered the same behavior on a Raspberry Pi. Building with qmake -config debug as the build instructions suggest didn't add any extra information to the segmentation fault.

[pstavirs]

With a debug build, run drone using gdb. Provide output of 'bt full' and the file/line of the crash alongwith any other info that you find. Srivats

[pstavirs]

Can you try the following patch and see if it fixes the problem?

diff --git a/server/linuxport.cpp b/server/linuxport.cpp
index b468dcf..82f03e1 100644
--- a/server/linuxport.cpp
+++ b/server/linuxport.cpp
@@ -217,6 +217,11 @@ void LinuxPort::populateInterfaceInfo()
     }

     nl_addr *addr = rtnl_link_get_addr(link);
+    if (!addr) {
+        qWarning("rtnetlink mac addr not found for %s", name());
+        return;
+    }
+
     if (nl_addr_get_family(addr) != AF_LLC) {
         qWarning("unexpected mac family found for %s:%d",
                 name(), nl_addr_get_family(addr));

[mcmillanda]

The patch fixed the seg fault, and drone is now running on the Balena Fin. Thank you for your quick fix!

[pstavirs]

@mcmillanda : Thanks for verifying that the patch fixes the crash.

This was really a shot in the dark from my side, so to better understand the issue could you provide me the drone -d output log along with the output of ifconfig -a?

[mcmillanda]

I now see that it is important to note that this device is using modem manager and network manager to manage an LTE chipset attached via serial interface. Because of this the MAC address of wwan0 is not present. In order to run modem manager and network manager, openresolv and dhcpd5 were purged from the pi. With the patch applied drone ran without issue, but any frames generated were invalid due to the MAC address not being present. In the end, I attached a separate device running Ostinato to the Ethernet port on the pi and used it as a LAN by creating NAT rules in iptables.

Here is the drone -d output:

pi@raspberrypi:~ $ sudo drone -d
Version: 1.0
Revision: 2966884fc83e
PCAP Lib: libpcap version 1.8.1
Retrieving the device list from the local machine

0. wwan0
txStats: collection start
cap file = /tmp/Drone.WUWSEM
rtnetlink mac addr not found for wwan0
adding dev to all ports list <wwan0>
setRateAccuracy: rate accuracy set to High - busy wait
1. any
 (Pseudo-device that captures on all interfaces)

txStats: collection start
cap file = /tmp/Drone.phYbox
rtnetlink link not found for any
adding dev to all ports list <any>
setRateAccuracy: rate accuracy set to High - busy wait
2. lo
txStats: collection start
cap file = /tmp/Drone.EXtyRY
zero mac for lo - skipping
adding dev to all ports list <lo>
setRateAccuracy: rate accuracy set to High - busy wait
3. eth0
txStats: collection start
cap file = /tmp/Drone.gHyudy
adding dev to all ports list <eth0>
Mac: b827eb41c494
setRateAccuracy: rate accuracy set to High - busy wait
4. nflog
 (Linux netfilter log (NFLOG) interface)

Error setting direction(0) nflog: Setting direction is not implemented on this platform

PortMonitor: Error opening port nflog: nflog: Can't listen on group group index: Operation not permitted

PcapTxThread: Error opening port nflog: nflog: Can't listen on group group index: Operation not permitted

txStats: collection start
cap file = /tmp/Drone.peUqHN
rtnetlink link not found for nflog
adding dev to all ports list <nflog>
PortManager: unable to open nflog. Skipping!
In ~LinuxPort
In ~PcapPort
RxStats stop requested but is not running!
Receive stop requested but is not running!
txStats: collection end
4. nfqueue
 (Linux netfilter queue (NFQUEUE) interface)

Error setting direction(0) nfqueue: Setting direction is not implemented on this platform

PortMonitor: Error opening port nfqueue: nfqueue: Can't listen on group group index: Operation not permitted

PcapTxThread: Error opening port nfqueue: nfqueue: Can't listen on group group index: Operation not permitted

txStats: collection start
cap file = /tmp/Drone.uyxSfd
rtnetlink link not found for nfqueue
adding dev to all ports list <nfqueue>
PortManager: unable to open nfqueue. Skipping!
In ~LinuxPort
In ~PcapPort
RxStats stop requested but is not running!
Receive stop requested but is not running!
txStats: collection end
4. usbmon1
 (USB bus number 1)

PortMonitor: Error opening port usbmon1: usbmon1: Can't open USB bus file /sys/kernel/debug/usbmon/1t: No such file or directory

PortMonitor: Error opening port usbmon1: usbmon1: Can't open USB bus file /sys/kernel/debug/usbmon/1t: No such file or directory

PcapTxThread: Error opening port usbmon1: usbmon1: Can't open USB bus file /sys/kernel/debug/usbmon/1t: No such file or directory

txStats: collection start
cap file = /tmp/Drone.UcemMj
rtnetlink link not found for usbmon1
adding dev to all ports list <usbmon1>
PortManager: unable to open usbmon1. Skipping!
In ~LinuxPort
In ~PcapPort
RxStats stop requested but is not running!
Receive stop requested but is not running!
txStats: collection end
nlmsg_flags = 301
netlink required buffer size = 2640
buffer size 2640 enough for netlink
if: lo(1)
if: eth0(2)
buffer size 2640 enough for netlink
if: wwan0(3)
if: wwan1(4)
buffer size 2640 enough for netlink
port count = 3

stats for 3 ports setup
Port eth0: ifIndex 2
host(add):  mac=b827eb41c494
Updater: User-Agent: Mozilla/5.0 (Raspbian GNU/Linux 10 (buster)) Drone/1.0 (Qt/5.11.3)
ComboProtocol: protoNumber = 208, 0x16beff8 <--> 0x16b5c08
ComboProtocol: protoNumber = 206, 0x16c1b40 <--> 0x16be9e8
ComboProtocol: protoNumber = 206, 0x16c1e68 <--> 0x16c1c98
ComboProtocol: protoNumber = 207, 0x16c4ce8 <--> 0x16cf040
ComboProtocol: protoNumber = 305, 0x16c29e8 <--> 0x16d9db0
ComboProtocol: protoNumber = 304, 0x16da160 <--> 0x16da238
ComboProtocol: protoNumber = 303, 0x16da3a0 <--> 0x16da4d0
ComboProtocol: protoNumber = 306, 0x16da730 <--> 0x16da890
The server is running on 0.0.0.0: 7878
Version: 1.0
Revision: 2966884fc83e
Updater: Host update.ostinato.org not found

Here is sanitized output of ifconfig -a:

pi@raspberrypi:~ $ ifconfig -a
eth0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        ether b8:27:eb:41:c4:94  txqueuelen 1000  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1000  (Local Loopback)
        RX packets 3348  bytes 410740 (401.1 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 3348  bytes 410740 (401.1 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

wwan0: flags=4305<UP,POINTOPOINT,RUNNING,NOARP,MULTICAST>  mtu 1428
        inet X.X.X.X  netmask 255.255.255.252  destination X.X.X.X
        inet6 X:X:X:X:X:X:X:X  prefixlen 64  scopeid 0x0<global>
        unspec 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00  txqueuelen 1000  (UNSPEC)
        RX packets 7585  bytes 668219 (652.5 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 12080  bytes 1078068 (1.0 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

wwan1: flags=4098<BROADCAST,MULTICAST>  mtu 1500
        ether 72:56:56:73:ed:f8  txqueuelen 1000  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

[pstavirs]

@mcmillanda It's indeed the lack of a mac address (technically, link layer address) on wwan0 that was causing the crash.

With the patch applied drone ran without issue, but any frames generated were invalid due to the MAC address not being present. In the end, I attached a separate device running Ostinato to the Ethernet port on the pi and used it as a LAN by creating NAT rules in iptables.

You can still have Ostinato directly generate frames on wwan0 by using the Advanced Protocol Selection_ tab and select just the protocols that you need excluding the mac address.

I'm curious - does this device have a link layer protocol? How does a IP/UDP packet look on the wire for this device in terms of frame format?

[mcmillanda]

@pstavirs I was able to use Ostinato on my balenaFin using your suggested method of modifying the protocols under the Advanced Protocol Selection tab. I had to remove both the MAC and Eth II protocols to get this to work.

The modem manager application allows me to communicate directly with the LTE chipset via serial interface. Since LTE does not use Ethernet for the link layer protocol over the air interface, a MAC address is not necessary. Confusingly, LTE uses a protocol titled Media Access Control (MAC) and Radio Link Control (RLC) as link layer protocols.

Thanks for your assistance.