Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

nexmon blindness bug (brcmf_cfg80211_nexmon_set_channel) #267

Open
evilsocket opened this issue Oct 12, 2019 · 71 comments

Comments

@evilsocket
Copy link
Owner

@evilsocket evilsocket commented Oct 12, 2019

every once in a while, nexmon dies with:

[ 4341.527847] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4101, -110
[ 4344.327806] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4097, -110
[ 4347.127853] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4098, -110
[ 4349.927917] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4099, -110
[ 4352.728074] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4100, -110
[ 4355.527970] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4101, -110
[ 4358.328022] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4102, -110
[ 4361.208095] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4103, -110
[ 4364.008157] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4104, -110
[ 4366.808218] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4105, -110
[ 4369.608431] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4097, -110
[ 4372.408345] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4098, -110
[ 4375.288408] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4099, -110
[ 4378.088474] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4100, -110
[ 4380.891399] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4101, -110

And only a reboot can fix the wifi, this is why the mon_max_blind_epochs parameter exists, to reboot the board when this happens.

Ideally we should document this known issue, the configuration and some day maybe fix it.

@evilsocket evilsocket added this to the 1.0.0 milestone Oct 12, 2019
@evilsocket evilsocket self-assigned this Oct 12, 2019
@evilsocket

This comment has been minimized.

Copy link
Owner Author

@evilsocket evilsocket commented Oct 12, 2019

@evilsocket

This comment has been minimized.

Copy link
Owner Author

@evilsocket evilsocket commented Oct 12, 2019

@evilsocket evilsocket changed the title document nexmon blindness bug nexmon blindness bug Oct 12, 2019
@evilsocket evilsocket added hardware and removed doc todo labels Oct 12, 2019
@evilsocket evilsocket removed this from the 1.0.0 milestone Oct 12, 2019
@Leajian

This comment has been minimized.

Copy link
Contributor

@Leajian Leajian commented Oct 13, 2019

You could try reloading the driver instead of rebooting the pi, using :

modprobe -r brcmfmac
modprobe brcmfmac

If this still fails, then a reboot will fix this.
There is a great discussion about that problem here : https://www.bountysource.com/issues/56252669-wlan-freezes-in-raspberry-pi-3b

@evilsocket

This comment has been minimized.

Copy link
Owner Author

@evilsocket evilsocket commented Oct 13, 2019

tried that way, it doesn't always work, the only reliable way is rebooting

@JRWR

This comment has been minimized.

Copy link
Contributor

@JRWR JRWR commented Oct 14, 2019

So, The Nexmon firmware is a little picky on how its interfaced, How are your bringing up the mon interface in linux before bettercap gets to it?

@evilsocket

This comment has been minimized.

Copy link
Owner Author

@evilsocket evilsocket commented Oct 14, 2019

iw phy phy0 interface add mon0 type monitor && ifconfig mon0 up

@evilsocket

This comment has been minimized.

@evilsocket

This comment has been minimized.

Copy link
Owner Author

@evilsocket evilsocket commented Oct 14, 2019

(from re4son monstart script)

@JRWR

This comment has been minimized.

Copy link
Contributor

@JRWR JRWR commented Oct 15, 2019

I just noticed on the Nexmon Repo that the bcm43455c0 does not support wifi frame injection. That might be the reason for the drivers crashing as they are not handling the requests to do frame injection correctly from bettercap.

@JRWR

This comment has been minimized.

Copy link
Contributor

@JRWR JRWR commented Oct 15, 2019

Ok, So I looked, We are using the older version of the firmware provided by Nexmon, I think Re4son kernel does it as its the default for Nexmon to use that. Patches and Commits from the Nexmon Project show 7.45.189 as the latest version you can use (The base firmware comes from the OEM, not the RPI foundation) as of RC4, we are using version 7.45.154 of the bcm43455c0 firmware This issue should only affect 3B+ and 4s, the 3B and the 0W use the same wifi chip and are listed as supporting injection.

@JRWR

This comment has been minimized.

Copy link
Contributor

@JRWR JRWR commented Oct 15, 2019

More Details, I've looked into how the Re4son kernel builder pulls down its firmware, its even /worse/ Looks like it pulls from https://github.com/Re4son/re4son-nexmon as its source of nexmon patches... its years out of date. I'm doing some prototyping to update this now.

@JRWR

This comment has been minimized.

Copy link
Contributor

@JRWR JRWR commented Oct 15, 2019

Good News, The Nexmon Patches with the stock kernel works well, Here is my DMesg output after running for 10 minutes, I will be running it for the next 24 hours to see if its stable,

[    5.128852] brcmfmac: loading out-of-tree module taints kernel.
[    5.128864] brcmfmac: loading out-of-tree module taints kernel.
[    5.181306] brcmfmac: F1 signature read @0x18000000=0x15264345
[    5.190412] brcmfmac: brcmf_fw_alloc_request: using brcm/brcmfmac43455-sdio for chip BCM4345/6
[    5.190907] usbcore: registered new interface driver brcmfmac
[    5.566003] brcmfmac: brcmf_sdio_bus_preinit: before brcmf_sdio_debugfs_create
[    5.569776] brcmfmac: brcmf_fw_alloc_request: using brcm/brcmfmac43455-sdio for chip BCM4345/6
[    5.597271] brcmfmac: brcmf_c_preinit_dcmds: Firmware: BCM4345/6 wl0: Oct 15 2019 20:30:25 version 7.45.189 (nexmon.org: -4) FWID 01-e1db26e2
[    5.675296] brcmfmac: brcmf_bus_started: before brcmf_debugfs_add_entry
[    8.495009] brcmfmac: brcmf_vif_add_validate: Attempt to add a MONITOR interface...
[    8.495024] brcmfmac: brcmf_mon_add_vif: brcmf_mon_add_vif called
[    8.495028] brcmfmac: brcmf_mon_add_vif: Adding vif "mon0"

Here is my uname

Linux pwnagotchi 4.19.75-v7+ #1270 SMP Tue Sep 24 18:45:11 BST 2019 armv7l GNU/Linux

@JRWR

This comment has been minimized.

Copy link
Contributor

@JRWR JRWR commented Oct 15, 2019

And I failed, You get a little more debug output this time overall..

[ 5387.511962] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4097, -110
[ 5390.551934] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4098, -110
[ 5393.601927] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4099, -110
[ 5396.631949] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4101, -110
[ 5399.671984] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4102, -110
[ 5402.711975] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4103, -110
[ 5405.752020] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4104, -110
[ 5408.791991] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4105, -110
[ 5411.831985] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4106, -110
[ 5414.872013] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53284, -110
[ 5417.911966] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53288, -110
[ 5420.951985] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53296, -110
[ 5423.511967] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4099, -110
[ 5426.312006] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4100, -110
[ 5429.111938] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4103, -110
[ 5431.911970] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4105, -110
[ 5434.711974] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53308, -110
[ 5437.511963] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53348, -110
[ 5440.311927] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53356, -110
[ 5443.111922] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53360, -110
[ 5445.911981] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53368, -110
[ 5448.711973] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53372, -110
[ 5451.511925] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53380, -110
[ 5454.311928] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4099, -110
[ 5457.111929] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4100, -110
[ 5459.911983] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4103, -110
[ 5462.711974] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4105, -110
[ 5465.511924] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53308, -110
[ 5468.311926] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53348, -110
[ 5471.111925] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53356, -110
[ 5473.912031] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53360, -110
[ 5476.471932] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4102, -110
[ 5479.271932] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4103, -110
[ 5482.081987] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4104, -110
[ 5484.871994] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4105, -110
[ 5487.671927] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53292, -110
[ 5490.471930] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53300, -110
[ 5493.271977] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53304, -110
[ 5496.071923] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53348, -110
[ 5498.872004] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53364, -110
[ 5501.671976] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53372, -110
[ 5504.471928] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53376, -110
[ 5507.031989] brcmfmac: brcmf_proto_bcdc_query_dcmd: brcmf_proto_bcdc_msg failed w/status -110
[ 5507.032002] brcmfmac: brcmf_cfg80211_get_channel: chanspec failed (-110)
[ 5509.591930] brcmfmac: brcmf_proto_bcdc_query_dcmd: brcmf_proto_bcdc_msg failed w/status -110
[ 5509.591939] brcmfmac: brcmf_cfg80211_get_tx_power: error (-110)
[ 5512.151989] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4102, -110
[ 5514.952027] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4103, -110

Edit:
Added brcmfmac.debug=30 to /boot/cmdline.txt to try and get a better look at what the driver is doing, It enables full trace mode for the drivers, I have attacked a full debug log up to where it strops responding.
syslog.gz

@evilsocket

This comment has been minimized.

Copy link
Owner Author

@evilsocket evilsocket commented Oct 16, 2019

yep when that happens even trying to change channel manually doesn't work, i think it's the heat

@JRWR

This comment has been minimized.

Copy link
Contributor

@JRWR JRWR commented Oct 16, 2019

Based off this photo, There is no TIM under the wifi can at all... Since its used for shielding RF, this is of no surprise. Someone should get a IR Camera on the board and take a look.

image

Also You notice that White IC in the top left, Thats the same chip they used to show people the SuperMicro Implant... lulz

@JRWR

This comment has been minimized.

Copy link
Contributor

@JRWR JRWR commented Oct 16, 2019

So, I'm looking at the datasheet, for the CYW43455 It states that the Max temp for operation is 120C while under normal loads on a 4 layer board Of course Max oper temp is +85C The thing only puts out 1.2W but I did notice that on the older RPI0W Chips it had a self limiter for overheat, This one is not stated in the data sheet as having one

@evilsocket

This comment has been minimized.

Copy link
Owner Author

@evilsocket evilsocket commented Oct 17, 2019

so you're saying that it's the chinese sabotaging our wifi pwning, right?

@JRWR

This comment has been minimized.

Copy link
Contributor

@JRWR JRWR commented Oct 17, 2019

Maybe.... Until I can get a proper Temp readout of the die itself while under the heavy load, We wont know.

@andrewbeard

This comment has been minimized.

Copy link
Contributor

@andrewbeard andrewbeard commented Nov 11, 2019

Yup, the package is installed. For some reason it sticks everything in linux-headers-4.19.66-Re4son-v7+ and not linux-headers-4.19.66-Re4son-v7l+, though. It looks like that might just be an endianess flag, so I'd be a little surprised if the source was different. I don't think I have an actual kernel config to tell it otherwise, though.

@andrewbeard

This comment has been minimized.

Copy link
Contributor

@andrewbeard andrewbeard commented Nov 11, 2019

Just to add another complication is looks like was just a new kalipi kernel update pushed to the apt repos. Info looks the same, so I would be shocked if the problem didn't continue:

abeard@pwnpi4:/usr/src/nexmon $ uname -a
Linux pwnpi4 4.19.81-Re4son-v7l+ #1 SMP Wed Nov 6 13:10:35 AEDT 2019 armv7l GNU/Linux
abeard@pwnpi4:/usr/src/nexmon $ dmesg | grep 7.45
[ 2.736258] brcmfmac: brcmf_c_preinit_dcmds: Firmware: BCM4345/6 wl0: Oct 8 2018 09:18:41 version 7.45.154 (nexmon.org: 2.2.2-195-g01e0-1) FWID 01-4fbe0b04

@DrSchottky

This comment has been minimized.

Copy link

@DrSchottky DrSchottky commented Nov 11, 2019

What happen if you try ignore vermagic by adding -f to insmod?

@andrewbeard

This comment has been minimized.

Copy link
Contributor

@andrewbeard andrewbeard commented Nov 11, 2019

No luck:

root@pwnpi4:/usr/src/nexmon/patches/bcm43455c0/7_45_189/nexmon# insmod -f brcmfmac_4.19.y-nexmon/brcmfmac.ko
insmod: ERROR: could not insert module brcmfmac_4.19.y-nexmon/brcmfmac.ko: Invalid module format

@andrewbeard

This comment has been minimized.

Copy link
Contributor

@andrewbeard andrewbeard commented Nov 11, 2019

So from what I can tell the source code for the kernels should be identical. I need the kernel config for the v7l+ version, though. None of the config extraction options seem to be built in, so I'm kind of at a loss as to where I can get it from.

@andrewbeard

This comment has been minimized.

Copy link
Contributor

@andrewbeard andrewbeard commented Nov 11, 2019

It seems like this may be an issue specific to the pi4. From what I can tell it's the only system that uses the v7l+ kernel. The build issue is probably the same thing that happened here:
seemoo-lab/nexmon#321

I'm kind of dead in the water at the moment without being able to find the right kernel source or config. It looks like the kernel is being installed from an apt repo here:
http://http.re4son-kernel.com/re4son/

But the re4son website is pretty out of date and so I'm failing miserably trying to find the source package that corresponds to the installed kernel so far. It doesn't not appear that there's a package for the kalipi full kernel source, just the headers (that I already have, and don't include v7l+).

@DrSchottky

This comment has been minimized.

Copy link

@DrSchottky DrSchottky commented Nov 12, 2019

Yes, v7l+ (where l stands for lpae) targets Rpi4 beacuse of its larger RAM. The problem is that the kalipi-kernel-headers package is incomplete since it contains only headers for v7+.
If you look at re4son GH repo you can find their kernel and build script, so you should (never tried) to build a preper headers deb for your arch.
But imho switching to a 3rd party kernel is more a hassle than a benefit, that's why I asked Simone if it's needed.

@DrSchottky

This comment has been minimized.

Copy link

@DrSchottky DrSchottky commented Nov 12, 2019

@andrewbeard I patched the buildscripts to include v7l+ headers in the package.
Try this
It's not tested (I don't have a kalipi setup available atm) and the builder has many bugs, so it's an attempt.

@andrewbeard

This comment has been minimized.

Copy link
Contributor

@andrewbeard andrewbeard commented Nov 12, 2019

COLLECTING STATISTICS read /usr/src/nexmon/STATISTICS.md for more information
COMPILING src/version.c => obj/version.o (details: log/compiler.log)
GENERATING LINKER FILE gen/nexmon.pre => gen/nexmon.ld
GENERATING LINKER FILE gen/nexmon.pre => gen/flashpatches.ld
LINKING OBJECTS => gen/patch.elf (details: log/linker.log, log/linker.err)
GENERATING MAKE FILE gen/nexmon.pre => gen/nexmon.mk
GENERATING MAKE FILE gen/nexmon.pre => gen/flashpatches.mk
APPLYING FLASHPATCHES gen/flashpatches.mk => brcmfmac43455-sdio.bin (details: log/flashpatches.log)
APPLYING PATCHES gen/nexmon.mk => brcmfmac43455-sdio.bin (details: log/patches.log)
BUILDING DRIVER for kernel 4.19 brcmfmac_4.19.y-nexmon/brcmfmac.ko (details: log/driver.log)
/bin/sh: 1: scripts/basic/fixdep: Exec format error
make[2]: *** [scripts/Makefile.build:303: /usr/src/nexmon/patches/bcm43455c0/7_45_189/nexmon/brcmfmac_4.19.y-nexmon/chip.o] Error 2
make[2]: *** Waiting for unfinished jobs....
/bin/sh: 1: scripts/basic/fixdep: Exec format error
make[2]: *** [scripts/Makefile.build:303: /usr/src/nexmon/patches/bcm43455c0/7_45_189/nexmon/brcmfmac_4.19.y-nexmon/cfg80211.o] Error 2
make[1]: *** [Makefile:1519: module/usr/src/nexmon/patches/bcm43455c0/7_45_189/nexmon/brcmfmac_4.19.y-nexmon] Error 2
make: *** [Makefile:52: brcmfmac.ko] Error 2

root@pwnpi4:/usr/src/nexmon/patches/bcm43455c0/7_45_189/nexmon# file /usr/src/linux-headers-4.19.81-Re4son-v7l+/scripts/basic/fixdep
/usr/src/linux-headers-4.19.81-Re4son-v7l+/scripts/basic/fixdep: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, BuildID[sha1]=f96cf37e4ab3abdfa90880655b262c4ae72c937a, for GNU/Linux 3.2.0, not stripped

It looks like the scripts folder in the headers has locally compiled binaries, but since the headers were cross-compiled on an x64 system it's causing the nexmon build to shit the bed.

@hackabean

This comment has been minimized.

Copy link

@hackabean hackabean commented Nov 12, 2019

Hey @DrSchottky I would like to help if I can as well.

Hardware:
Raspberry Pi 0w

Output from uname -a:
Linux pwnagotchi 4.19.66-Re4son+ #1 Sun Aug 18 13:42:02 AEST 2019 armv6l GNU/Linux

dmesg | grep 7.45:
[ 14.339006] brcmfmac: brcmf_c_preinit_dcmds: Firmware: BCM43430/1 wl0: Aug 7 2017 00:46:29 version 7.45.41.46 (r666254 CY) FWID 01-f8a78378

modinfo /lib/modules/4.19.66-Re4son+/kernel/drivers/net/wireless/broadcom/brcm80211/brcmfmac/brcmfmac.ko:

filename: /lib/modules/4.19.66-Re4son+/kernel/drivers/net/wireless/broadcom/brcm80211/brcmfmac/brcmfmac.ko
license: Dual BSD/GPL
description: Broadcom 802.11 wireless LAN fullmac driver.
author: Broadcom Corporation
firmware: brcm/brcmfmac4373-sdio.bin
firmware: brcm/brcmfmac4356-sdio.bin
firmware: brcm/brcmfmac4354-sdio.bin
firmware: brcm/brcmfmac43455-sdio.bin
firmware: brcm/brcmfmac43430-sdio.bin
firmware: brcm/brcmfmac43430a0-sdio.bin
firmware: brcm/brcmfmac4339-sdio.bin
firmware: brcm/brcmfmac43362-sdio.bin
firmware: brcm/brcmfmac4335-sdio.bin
firmware: brcm/brcmfmac43341-sdio.bin
firmware: brcm/brcmfmac43340-sdio.bin
firmware: brcm/brcmfmac4334-sdio.bin
firmware: brcm/brcmfmac4330-sdio.bin
firmware: brcm/brcmfmac4329-sdio.bin
firmware: brcm/brcmfmac43241b5-sdio.bin
firmware: brcm/brcmfmac43241b4-sdio.bin
firmware: brcm/brcmfmac43241b0-sdio.bin
firmware: brcm/brcmfmac43143-sdio.bin
firmware: brcm/brcmfmac4373.bin
firmware: brcm/brcmfmac43569.bin
firmware: brcm/brcmfmac43242a.bin
firmware: brcm/brcmfmac43236b.bin
firmware: brcm/brcmfmac43143.bin
srcversion: 389E0869FFB2AAD06E0D34E
alias: usb:v04B4p0BDCddcdscdpiciscipin
alias: usb:v0A5Cp0BDCddcdscdpiciscipin
alias: usb:v043Ep3101ddcdscdpiciscipin
alias: usb:v04B4pBD29ddcdscdpiciscipin
alias: usb:v13B1p0039ddcdscdpiciscipin
alias: usb:v0A5CpBD27ddcdscdpiciscipin
alias: usb:v0A5CpBD1Fddcdscdpiciscipin
alias: usb:v0A5CpBD17ddcdscdpiciscipin
alias: usb:v0A5CpBD1Eddcdscdpiciscipin
alias: sdio:cv02D0d4373
alias: sdio:cv02D0d4356
alias: sdio:cv02D0d4354
alias: sdio:cv02D0dA9BF
alias: sdio:cv02D0d4345
alias: sdio:cv02D0dA9A6
alias: sdio:cv02D0d4339
alias: sdio:cv02D0d4335
alias: sdio:cv02D0dA9A4
alias: sdio:cv02D0dA962
alias: sdio:cv02D0dA94D
alias: sdio:cv02D0dA94C
alias: sdio:cv02D0d4334
alias: sdio:cv02D0d4330
alias: sdio:cv02D0d4329
alias: sdio:cv02D0d4324
alias: sdio:cv02D0dA887
depends: brcmutil,cfg80211
intree: Y
name: brcmfmac
vermagic: 4.19.66-Re4son+ mod_unload modversions ARMv6 p2v8
parm: txglomsz:Maximum tx packet chain size [SDIO] (int)
parm: debug:Level of debug output (int)
parm: p2pon:Enable legacy p2p management functionality (int)
parm: feature_disable:Disable features (int)
parm: alternative_fw_path:Alternative firmware path (string)
parm: fcmode:Mode of firmware signalled flow control (int)
parm: roamoff:Do not use internal roaming engine (int)
parm: iapp:Enable partial support for the obsoleted Inter-Access Point Protocol (int)
parm: ignore_probe_fail:always succeed probe for debugging (int)

@andrewbeard

This comment has been minimized.

Copy link
Contributor

@andrewbeard andrewbeard commented Nov 12, 2019

I figured I'd give it one more jackass try and replace all the x64 script binaries with ARM ones from my local build tree. They shouldn't be dependent on the patchset, right?

Module built (yay!), but still wouldn't insert (boo):

  COLLECTING STATISTICS read /usr/src/nexmon/STATISTICS.md for more information
  COMPILING src/version.c => obj/version.o (details: log/compiler.log)
  GENERATING LINKER FILE gen/nexmon.pre => gen/nexmon.ld
  GENERATING LINKER FILE gen/nexmon.pre => gen/flashpatches.ld
  LINKING OBJECTS => gen/patch.elf (details: log/linker.log, log/linker.err)
  GENERATING MAKE FILE gen/nexmon.pre => gen/nexmon.mk
  GENERATING MAKE FILE gen/nexmon.pre => gen/flashpatches.mk
  APPLYING FLASHPATCHES gen/flashpatches.mk => brcmfmac43455-sdio.bin (details: log/flashpatches.log)
  APPLYING PATCHES gen/nexmon.mk => brcmfmac43455-sdio.bin (details: log/patches.log)
  BUILDING DRIVER for kernel 4.19 brcmfmac_4.19.y-nexmon/brcmfmac.ko (details: log/driver.log)
  COPYING brcmfmac43455-sdio.bin => /lib/firmware/brcm/brcmfmac43455-sdio.bin
  RELOADING brcmfmac
insmod: ERROR: could not insert module brcmfmac_4.19.y-nexmon/brcmfmac.ko: Invalid module format
make: *** [Makefile:190: install-firmware] Error 1
root@pwnpi4:/usr/src/nexmon/patches/bcm43455c0/7_45_189/nexmon# insmod -f brcmfmac_4.19.y-nexmon/brcmfmac.ko 
insmod: ERROR: could not insert module brcmfmac_4.19.y-nexmon/brcmfmac.ko: Invalid module format

The modules themselves look pretty damn similar. I'm really at a loss what the difference could be:

root@pwnpi4:/usr/src/nexmon/patches/bcm43455c0/7_45_189/nexmon# modinfo brcmfmac_4.19.y-nexmon/brcmfmac.ko > /tmp/newmod
root@pwnpi4:/usr/src/nexmon/patches/bcm43455c0/7_45_189/nexmon# modinfo /lib/modules/4.19.81-Re4son-v7l+/kernel/drivers/net/wireless/broadcom/brcm80211/brcmfmac/brcmfmac.ko > /tmp/oldmod
root@pwnpi4:/usr/src/nexmon/patches/bcm43455c0/7_45_189/nexmon# diff /tmp/newmod /tmp/oldmod
1c1
< filename:       /usr/src/nexmon/patches/bcm43455c0/7_45_189/nexmon/brcmfmac_4.19.y-nexmon/brcmfmac.ko
---
> filename:       /lib/modules/4.19.81-Re4son-v7l+/kernel/drivers/net/wireless/broadcom/brcm80211/brcmfmac/brcmfmac.ko
13a14
> firmware:       brcm/brcmfmac43341-sdio.bin
27c28
< srcversion:     A208FF46FF4FDC4A48B10E7
---
> srcversion:     389E0869FFB2AAD06E0D34E
54a56
> intree:         Y
@DrSchottky

This comment has been minimized.

Copy link

@DrSchottky DrSchottky commented Nov 12, 2019

This header thing is becoming a pain in the ass.
Since pwnagotchi can be installed on a working system why don't we burn a clean Raspbian Lite image, do a couple of nexmon tests and then install the software?

@andrewbeard

This comment has been minimized.

Copy link
Contributor

@andrewbeard andrewbeard commented Nov 12, 2019

Something I just noticed, that failedinstall process looks like it did overwrite the firmware in /lib/firmware/brcm/brcmfmac43455-sdio.bin and when I rebooted the pi it did load the new firmware.

[    2.664068] brcmfmac: brcmf_fw_alloc_request: using brcm/brcmfmac43455-sdio for chip BCM4345/6
[    2.673602] brcmfmac: brcmf_c_preinit_dcmds: Firmware: BCM4345/6 wl0: Nov 12 2019 09:40:03 version 7.45.189 (nexmon.org: 2.2.2-273-gaaa80e-dirty-7) FWID 01-e1db26e2

Unfortunately the channel change errors now start immediately, about 14 second after I boot:

[   14.299243] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53286, -52
[   15.312016] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53290, -52
[   16.325362] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53294, -52
[   30.549909] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53286, -52
[   31.567560] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53290, -52
[   32.584209] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53294, -52
[   62.261185] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53286, -52
[   63.278705] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53290, -52
[   64.299974] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53294, -52
[   78.550006] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53286, -52
[   79.568249] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53290, -52
[   80.586455] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53294, -52
[  128.297310] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53286, -52
[  131.399334] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53286, -52
[  134.503219] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53286, -52
[  137.631244] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53286, -52
[  140.753593] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53286, -52

It looks like new firmware + old module is even worse than old firmware + old module.

@andrewbeard

This comment has been minimized.

Copy link
Contributor

@andrewbeard andrewbeard commented Nov 12, 2019

@DrSchottky Sounds like a good plan. I'll see if I've got a spare sd card I can start from.

@andrewbeard

This comment has been minimized.

Copy link
Contributor

@andrewbeard andrewbeard commented Nov 12, 2019

OK, I've got a fresh Raspbian lite image + the nexmon driver installed with the 7.45.189 firmware. What do you need me to try?

@DrSchottky

This comment has been minimized.

Copy link

@DrSchottky DrSchottky commented Nov 12, 2019

install aircrack-ng, remove wpasupplicant, activate mon0 and let airodump run for a while.
If it won't crash we can consider monitor mode working and go to the next step (you can also repeat the test with nexmon 7.45.154)

@andrewbeard

This comment has been minimized.

Copy link
Contributor

@andrewbeard andrewbeard commented Nov 13, 2019

So, I tried a couple things last night:

  • airodump seems to run without error for 6+ hours, but it picks up very little. Few stations detected.
  • I tried running bettercap but it produces an error when initializing the interface complaining that it couldn't set the channel
  • I tried to debug that a bit and boiled it down to this command bettercap is running:
root@pwnpi4-ng:/home/abeard# iwconfig mon0 channel 1
Error for wireless request "Set Frequency" (8B04) :
    SET failed on device mon0 ; Device or resource busy.

So, the good news is that it isn't locking up. The bad news is that I'm not really sure I'm getting to the fully functional point (even without trying injection).

@DrSchottky

This comment has been minimized.

Copy link

@DrSchottky DrSchottky commented Nov 13, 2019

Ok, airodump can't find many stations because it can't do channel hopping.
So reboot your device and make sure wpa_supplicant isn't running/isn't installed and start mon0 this way

sudo ip link set dev wlan0 down
sudo iwconfig wlan0 channel 6
sudo iw phy `iw dev wlan0 info | gawk '/wiphy/ {printf "phy" $2}'` interface add mon0 type monitor
sudo ip link set dev mon0 up
sudo iwconfig mon0 channel 6

Now you should have mon0 set on channel 6 (and you should be able to change the channel with iwconfig), so start airdoump telling it to work on both 2.4 and 5.8 bands
sudo airodump-ng --band abg mon0

@andrewbeard

This comment has been minimized.

Copy link
Contributor

@andrewbeard andrewbeard commented Nov 13, 2019

@andrewbeard

This comment has been minimized.

Copy link
Contributor

@andrewbeard andrewbeard commented Nov 14, 2019

25 hours running airodump with no issues. Running bettercap right now as a test. What should I try next?

@DrSchottky

This comment has been minimized.

Copy link

@DrSchottky DrSchottky commented Nov 14, 2019

Perfect.
I'd try to turn it into a complete pwnagotchi by following the manual install procedure

@andrewbeard

This comment has been minimized.

Copy link
Contributor

@andrewbeard andrewbeard commented Nov 14, 2019

[95101.861954] brcmfmac: brcmf_sdio_hostmail: mailbox indicates firmware halted
[95123.800146] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4097, -110
[95126.360187] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4102, -110
[95128.920228] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4099, -110
[95131.480295] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4097, -110
[95134.520352] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4098, -110
[95137.560407] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4100, -110
[95140.600478] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4102, -110
[95143.640511] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4103, -110
[95146.680603] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4105, -110
[95149.720655] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4107, -110
[95152.760713] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4108, -110
[95155.800785] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4109, -110
[95158.840877] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53292, -110
[95161.880885] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53296, -110
[95164.920952] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53308, -110
[95167.960983] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53352, -110
[95171.001052] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53360, -110
[95174.041139] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53364, -110
[95177.081197] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53368, -110
[95180.121272] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53376, -110
[95183.161338] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53384, -110
[95186.201382] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4097, -110
[95189.241458] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4098, -110
[95191.801500] brcmfmac: brcmf_proto_bcdc_query_dcmd: brcmf_proto_bcdc_msg failed w/status -110
[95191.801517] brcmfmac: brcmf_cfg80211_get_channel: chanspec failed (-110)
[95194.361561] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4100, -110
[95207.321832] brcmfmac: brcmf_proto_bcdc_query_dcmd: brcmf_proto_bcdc_msg failed w/status -110
[95207.321844] brcmfmac: brcmf_cfg80211_get_channel: chanspec failed (-110)
[95209.881888] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4097, -110
[95212.921901] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4098, -110
[95215.961962] brcmfmac: brcmf_proto_bcdc_query_dcmd: brcmf_proto_bcdc_msg failed w/status -110
[95215.961969] brcmfmac: brcmf_cfg80211_get_channel: chanspec failed (-110)
[95218.522018] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4099, -110

I'm going to reboot it and try again from a fresh start, but it looks like it went blind after less than an hour.

@andrewbeard

This comment has been minimized.

Copy link
Contributor

@andrewbeard andrewbeard commented Nov 15, 2019

I tried disabling the advertise, deauth, and associate options just to see if removing all the packet injection stuff would make a difference. The short version is that it doesn't. airodump never causes it to lock up, but bettercap driven by pwnagochi causes it to die almost immediately.

@DrSchottky

This comment has been minimized.

Copy link

@DrSchottky DrSchottky commented Nov 15, 2019

@andrewbeard is bettercap standalone (w/ injection enabled but not driven by pwnagotchi) causing any problem?
@evilsocket is bettercap doing any kind of injection under the hood or anything special?

@evilsocket

This comment has been minimized.

Copy link
Owner Author

@evilsocket evilsocket commented Nov 15, 2019

@DrSchottky

This comment has been minimized.

Copy link

@DrSchottky DrSchottky commented Nov 15, 2019

@evilsocket ok, but andrew told he turned off deauth/association, so I meant in addition to those.
I want to figure out what are the differences between airodump and bettercap (w/o injections) to understand why one works while the other crashes the fw.

@evilsocket

This comment has been minimized.

Copy link
Owner Author

@evilsocket evilsocket commented Nov 15, 2019

well there's also pwngrid that's injecting stuff on the same interface for the mesh protocol, but this bug happened way before pwngrid or the mesh were a thing ... maybe it might be the channel hopping? bettercap uses iwconfig to hop and that might be stressful for the rpi0w every few milliseconds ... it'd be interesting to try to disable hopping all together (by setting ai.enabled to false and personality channels to just 1 or whatever).

@DrSchottky

This comment has been minimized.

Copy link

@DrSchottky DrSchottky commented Nov 15, 2019

By default airodump hops every 250ms, so the only difference that comes to my mind is that you do it through wext, while airodump uses (AFAIK) nl.
@andrewbeard could you try to disable hopping as evlisocket suggested?

@evilsocket

This comment has been minimized.

Copy link
Owner Author

@evilsocket evilsocket commented Nov 15, 2019

@DrSchottky yes i think it's not the timing but the methodology ... i also don't think airodump uses calls to the iwconfig binary as i do in bettercap, pretty sure there's a cleaner way to do it :D

@DrSchottky

This comment has been minimized.

Copy link

@DrSchottky DrSchottky commented Nov 15, 2019

@evilsocket exactly, it uses a totally different software stack (wext vs nl80211).
If turning off the hopping solves the problem the easiest fix is to replace iwconfig with iw in bettercap.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
6 participants
You can’t perform that action at this time.