stuck while rebooting when driver is loaded? #36
Comments
|
The board hangs when rebooting with the IoT 8.4 image, don't know exactly what is causing it yet: |
|
Unfortunately, your logs do not show anything about what is happening. I'm not that familiar with any of the special boot operations for Beaglebone and similar devices. Is it possible to get any boot details? |
|
@lwfinger nothing really special, it's ti's v4.4.x branch, with a debian-ish kitchen sink config: https://github.com/RobertCNelson/ti-linux-kernel-dev/blob/ti-linux-4.4.y/patches/defconfig We build rtl8723bu with the default settings and package up 8723bu.ko into a deb package. https://github.com/rcn-ee/rtl8723bu/blob/master/build.sh#L9 My SanCloud variant is at work, so i can't test this on other kernel's till monday. Regards, |
mpvader
added a commit
to victronenergy/meta-victronenergy
that referenced
this issue
Sep 18, 2016
Make sure to add this commit from upstream in the openembedded-core mete: openembedded/openembedded-core@afcea61 Itś needed to prevent these errors during compile: WARNING: "wiphy_free" [/media/venusbuilds/wipbbbjethro/build/tmp-glibc/work/beaglebone-ve-linux-gnueabi/rtl8723bu/0.0-r0/git/8723bu.ko] undefined! WARNING: "cfg80211_unlink_bss" [/media/venusbuilds/wipbbbjethro/build/tmp-glibc/work/beaglebone-ve-linux-gnueabi/rtl8723bu/0.0-r0/git/8723bu.ko] undefined! [many more] and later, when using the ko file, there errors in the bootlog: [ 4.554499] 8723bu: no symbol version for cfg80211_scan_done [ 4.560468] 8723bu: Unknown symbol cfg80211_scan_done (err -22) ... many more, resulting in module not loading unless doing a manual modprobe -f OPEN ISSUES 0) I added lib80211 to the defconfig at some point while trying to fix above issue. Probably isn't needed anymore. Todo: test & remove 1) Replace the hack in the rtl8723bu Makefile with something proper 2) Reboot doesn't work lwfinger/rtl8723bu#36 3) Kernel deadlocks right after boot: [...] [ 3.544110] udevd[677]: starting version 182 [ 4.125070] cfg80211: Calling CRDA to update world regulatory domain [ 4.499920] RTL871X: module init start [ 4.503973] RTL871X: rtl8723bu v4.3.6.11_12942.20141204_BTCOEX20140507-4E40 [ 4.511273] RTL871X: rtl8723bu BT-Coex version = BTCOEX20140507-4E40 [ 7.276123] cfg80211: Calling CRDA to update world regulatory domain [ 7.816563] RTL871X: rtw_ndev_init(wlan0) [ 7.823760] usbcore: registered new interface driver rtl8723bu [ 7.829887] RTL871X: module init ret=0 [ 8.275916] EXT4-fs (mmcblk1p3): re-mounted. Opts: data=ordered bootlogd: cannot allocate pseudo tty: No such file or directory [ 8.419817] random: dd urandom read with 105 bits of entropy available Populating dev cache [ 10.432973] cfg80211: Calling CRDA to update world regulatory domain hwclock: can't open '/dev/misc/rtc': No such file or directory Thu Sep 15 23:48:31 UTC 2016 hwclock: can't open '/dev/misc/rtc': No such file or directory Starting watchdog [ 11.915814] watchdog watchdog0: watchdog did not stop! [ 13.582859] cfg80211: Calling CRDA to update world regulatory domain [ 14.169740] random: nonblocking pool is initialized update-rc.d: /etc/init.d/run-postinsts exists during rc.d purge (continuing) Removing any system startup links for run-postinsts ... /etc/rcS.d/S99run-postinsts INIT: Entering runlevel: 5 Configuring network interfaces... [ 14.520549] net eth0: initializing cpsw version 1.12 (0) [ 14.603248] net eth0: phy found : id is : 0x4dd072 [ 14.608427] libphy: PHY 4a101000.mdio:01 not found [ 14.613487] net eth0: phy 4a101000.mdio:01 not found on slave 1 [ 14.625689] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready udhcpc (v1.23.2) started Sending discover... [ 16.732824] cfg80211: Calling CRDA to update world regulatory domain Sending discover... [ 19.882724] cfg80211: Calling CRDA to update world regulatory domain Sending discover... [ 23.032729] cfg80211: Calling CRDA to update world regulatory domain No lease, forking to background [ 23.865391] c_can_platform 481d0000.can can1: setting BTR=1c05 BRPE=0000 done. Starting system message bus: Setting up watches. Watches established. /var/run/dbus/ CREATE system_bus_socket hwclock: can't open '/dev/misc/rtc': No such file or directory Starting syslogd/klogd: done Starting php-fpm [ 25.190096] c_can_platform 481d0000.can can1: setting BTR=1c05 BRPE=0000 [ 25.197402] IPv6: ADDRCONF(NETDEV_CHANGE): can1: link becomes ready [ 25.991495] c_can_platform 481d0000.can can1: setting BTR=1c05 BRPE=0000 [ 25.998758] IPv6: ADDRCONF(NETDEV_CHANGE): can1: link becomes ready [ 26.182841] cfg80211: Calling CRDA to update world regulatory domain done Starting Connection Manager Starting Hiawatha Web Server: [ 26.981252] c_can_platform 481d0000.can can1: setting BTR=1c05 BRPE=0000 [ 26.988575] IPv6: ADDRCONF(NETDEV_CHANGE): can1: link becomes ready hiawatha. dbus[1832]: [system] Activating service name='fi.w1.wpa_supplicant1' (using servicehelper) Starting crond: dbus[1832]: [system] Successfully activated service 'fi.w1.wpa_supplicant1' OK [ 27.518031] [ 27.519624] ============================================= [ 27.525281] [ INFO: possible recursive locking detected ] [ 27.530944] 4.1.15-venus #1 Tainted: G O [ 27.536236] --------------------------------------------- [ 27.541892] connmand/1959 is trying to acquire lock: [ 27.547094] (pmutex){+.+...}, at: [<bf0ef664>] usbctrl_vendorreq+0x7c/0x274 [8723bu] [ 27.555649] [ 27.555649] but task is already holding lock: [ 27.561764] (pmutex){+.+...}, at: [<bf0ee454>] netdev_open+0x28/0x50 [8723bu] [ 27.569468] [ 27.569468] other info that might help us debug this: [ 27.576311] Possible unsafe locking scenario: [ 27.576311] [ 27.582516] CPU0 [ 27.585075] ---- [ 27.587632] lock(pmutex); [ 27.590565] lock(pmutex); [ 27.593497] [ 27.593497] *** DEADLOCK *** [ 27.593497] [ 27.599704] May be due to missing lock nesting notation [ 27.599704] [ 27.606822] 2 locks held by connmand/1959: [ 27.611111] #0: (rtnl_mutex){+.+.+.}, at: [<c042d050>] devinet_ioctl+0xd0/0x7d4 [ 27.618996] #1: (pmutex){+.+...}, at: [<bf0ee454>] netdev_open+0x28/0x50 [8723bu] [ 27.627136] [ 27.627136] stack backtrace: [ 27.631706] CPU: 0 PID: 1959 Comm: connmand Tainted: G O 4.1.15-venus #1 [ 27.639914] Hardware name: Generic AM33XX (Flattened Device Tree) [ 27.646326] [<c0014844>] (unwind_backtrace) from [<c0012994>] (show_stack+0x10/0x14) [ 27.654455] [<c0012994>] (show_stack) from [<c0067fe0>] (__lock_acquire+0x1454/0x1da0) [ 27.662759] [<c0067fe0>] (__lock_acquire) from [<c006922c>] (lock_acquire+0xac/0x12c) [ 27.670979] [<c006922c>] (lock_acquire) from [<c04bffd4>] (mutex_lock_interruptible_nested+0x48/0x3bc) [ 27.680833] [<c04bffd4>] (mutex_lock_interruptible_nested) from [<bf0ef664>] (usbctrl_vendorreq+0x7c/0x274 [8723bu]) [ 27.692079] [<bf0ef664>] (usbctrl_vendorreq [8723bu]) from [<bf11e9f0>] (usb_read8+0x30/0x3c [8723bu]) [ 27.702011] [<bf11e9f0>] (usb_read8 [8723bu]) from [<bf0c1178>] (rtw_read8+0x18/0x1c [8723bu]) [ 27.711204] [<bf0c1178>] (rtw_read8 [8723bu]) from [<bf11ceb0>] (rtl8723bu_hal_init+0x30/0xb4c [8723bu]) [ 27.721338] [<bf11ceb0>] (rtl8723bu_hal_init [8723bu]) from [<bf1026b4>] (rtw_hal_init+0x18/0x154 [8723bu]) [ 27.731727] [<bf1026b4>] (rtw_hal_init [8723bu]) from [<bf0ee20c>] (_netdev_open+0x64/0x284 [8723bu]) [ 27.741561] [<bf0ee20c>] (_netdev_open [8723bu]) from [<bf0ee45c>] (netdev_open+0x30/0x50 [8723bu]) [ 27.751143] [<bf0ee45c>] (netdev_open [8723bu]) from [<c03c8884>] (__dev_open+0xb8/0x11c) [ 27.759726] [<c03c8884>] (__dev_open) from [<c03c8b08>] (__dev_change_flags+0x94/0x144) [ 27.768123] [<c03c8b08>] (__dev_change_flags) from [<c03c8bd0>] (dev_change_flags+0x18/0x48) [ 27.776978] [<c03c8bd0>] (dev_change_flags) from [<c042d64c>] (devinet_ioctl+0x6cc/0x7d4) [ 27.785559] [<c042d64c>] (devinet_ioctl) from [<c03ae70c>] (sock_ioctl+0x1d8/0x2b0) [ 27.793593] [<c03ae70c>] (sock_ioctl) from [<c011c39c>] (do_vfs_ioctl+0x3f0/0x5c0) [ 27.801533] [<c011c39c>] (do_vfs_ioctl) from [<c011c5a0>] (SyS_ioctl+0x34/0x5c) [ 27.809201] [<c011c5a0>] (SyS_ioctl) from [<c000f1e0>] (ret_fast_syscall+0x0/0x54) Checking available software versions... [ 28.191668] EXT4-fs (mmcblk1p2): mounted filesystem with ordered data mode. Opts: (null) Starting simple-upnpd [ 31.095134] RTL871X: RTW_ADAPTIVITY_EN_AUTO, chplan:0x20, Regulation:0,0 [ 31.102220] RTL871X: RTW_ADAPTIVITY_MODE_NORMAL [ 32.523712] IPv6: ADDRCONF(NETDEV_UP): wlan0: link is not ready [ 32.536569] cfg80211: Calling CRDA to update world regulatory domain [ 32.604106] c_can_platform 481d0000.can can1: setting BTR=1c05 BRPE=0000 [ 32.611164] c_can_platform 481d0000.can: obtain a copy of previously claimed pinctrl [ 32.619469] IPv6: ADDRCONF(NETDEV_CHANGE): can1: link becomes ready [ 33.212819] c_can_platform 481d0000.can can1: setting BTR=1c05 BRPE=0000 [ 33.220096] c_can_platform 481d0000.can: obtain a copy of previously claimed pinctrl [ 33.291001] IPv6: ADDRCONF(NETDEV_CHANGE): can1: link becomes ready [ 33.704635] RTL871X: nolinked power save enter [ 39.083257] RTL871X: RTW_ADAPTIVITY_EN_AUTO, chplan:0x20, Regulation:0,0 [ 39.090326] RTL871X: RTW_ADAPTIVITY_MODE_NORMAL [ 40.204562] RTL871X: nolinked power save leave [ 40.425420] RTL871X: rtw_set_802_11_connect(wlan0) fw_state=0x00000008 [ 40.432584] cfg80211: Calling CRDA to update world regulatory domain [ 40.498385] RTL871X: start auth [ 40.503755] RTL871X: auth success, start assoc [ 40.512793] RTL871X: rtw_cfg80211_indicate_connect(wlan0) BSS not found !! [ 40.520058] RTL871X: assoc success [ 40.524101] IPv6: ADDRCONF(NETDEV_CHANGE): wlan0: link becomes ready [ 40.956425] RTL871X: send eapol packet [ 40.978771] RTL871X: send eapol packet [ 40.994451] RTL871X: set pairwise key camid:4, addr:98:fc:11:ae:b5:04, kid:0, type:AES bgscan simple: Failed to enable signal strength monitoring [ 41.024333] RTL871X: set group key camid:5, addr:98:fc:11:ae:b5:04, kid:1, type:AES [ 43.582737] cfg80211: Calling CRDA to update world regulatory domain [ 46.732739] cfg80211: Exceeded CRDA call max attempts. Not calling CRDA
mpvader
added a commit
to victronenergy/meta-victronenergy
that referenced
this issue
Sep 19, 2016
Make sure to add this commit from upstream in the openembedded-core mete: openembedded/openembedded-core@afcea61 It's needed to prevent these errors during compile: WARNING: "wiphy_free" [/media/venusbuilds/wipbbbjethro/build/tmp-glibc/work/beaglebone-ve-linux-gnueabi/rtl8723bu/0.0-r0/git/8723bu.ko] undefined! WARNING: "cfg80211_unlink_bss" [/media/venusbuilds/wipbbbjethro/build/tmp-glibc/work/beaglebone-ve-linux-gnueabi/rtl8723bu/0.0-r0/git/8723bu.ko] undefined! [many more] and later, when using the ko file, there errors in the bootlog: [ 4.554499] 8723bu: no symbol version for cfg80211_scan_done [ 4.560468] 8723bu: Unknown symbol cfg80211_scan_done (err -22) ... many more, resulting in module not loading unless doing a manual modprobe -f OPEN ISSUES 0) I added lib80211 to the defconfig at some point while trying to fix above issue. Probably isn't needed anymore. Todo: test & remove 1) Replace the hack in the rtl8723bu Makefile with something proper 2) Reboot doesn't work lwfinger/rtl8723bu#36 3) Kernel deadlocks right after boot: [...] [ 3.544110] udevd[677]: starting version 182 [ 4.125070] cfg80211: Calling CRDA to update world regulatory domain [ 4.499920] RTL871X: module init start [ 4.503973] RTL871X: rtl8723bu v4.3.6.11_12942.20141204_BTCOEX20140507-4E40 [ 4.511273] RTL871X: rtl8723bu BT-Coex version = BTCOEX20140507-4E40 [ 7.276123] cfg80211: Calling CRDA to update world regulatory domain [ 7.816563] RTL871X: rtw_ndev_init(wlan0) [ 7.823760] usbcore: registered new interface driver rtl8723bu [ 7.829887] RTL871X: module init ret=0 [ 8.275916] EXT4-fs (mmcblk1p3): re-mounted. Opts: data=ordered bootlogd: cannot allocate pseudo tty: No such file or directory [ 8.419817] random: dd urandom read with 105 bits of entropy available Populating dev cache [ 10.432973] cfg80211: Calling CRDA to update world regulatory domain hwclock: can't open '/dev/misc/rtc': No such file or directory Thu Sep 15 23:48:31 UTC 2016 hwclock: can't open '/dev/misc/rtc': No such file or directory Starting watchdog [ 11.915814] watchdog watchdog0: watchdog did not stop! [ 13.582859] cfg80211: Calling CRDA to update world regulatory domain [ 14.169740] random: nonblocking pool is initialized update-rc.d: /etc/init.d/run-postinsts exists during rc.d purge (continuing) Removing any system startup links for run-postinsts ... /etc/rcS.d/S99run-postinsts INIT: Entering runlevel: 5 Configuring network interfaces... [ 14.520549] net eth0: initializing cpsw version 1.12 (0) [ 14.603248] net eth0: phy found : id is : 0x4dd072 [ 14.608427] libphy: PHY 4a101000.mdio:01 not found [ 14.613487] net eth0: phy 4a101000.mdio:01 not found on slave 1 [ 14.625689] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready udhcpc (v1.23.2) started Sending discover... [ 16.732824] cfg80211: Calling CRDA to update world regulatory domain Sending discover... [ 19.882724] cfg80211: Calling CRDA to update world regulatory domain Sending discover... [ 23.032729] cfg80211: Calling CRDA to update world regulatory domain No lease, forking to background [ 23.865391] c_can_platform 481d0000.can can1: setting BTR=1c05 BRPE=0000 done. Starting system message bus: Setting up watches. Watches established. /var/run/dbus/ CREATE system_bus_socket hwclock: can't open '/dev/misc/rtc': No such file or directory Starting syslogd/klogd: done Starting php-fpm [ 25.190096] c_can_platform 481d0000.can can1: setting BTR=1c05 BRPE=0000 [ 25.197402] IPv6: ADDRCONF(NETDEV_CHANGE): can1: link becomes ready [ 25.991495] c_can_platform 481d0000.can can1: setting BTR=1c05 BRPE=0000 [ 25.998758] IPv6: ADDRCONF(NETDEV_CHANGE): can1: link becomes ready [ 26.182841] cfg80211: Calling CRDA to update world regulatory domain done Starting Connection Manager Starting Hiawatha Web Server: [ 26.981252] c_can_platform 481d0000.can can1: setting BTR=1c05 BRPE=0000 [ 26.988575] IPv6: ADDRCONF(NETDEV_CHANGE): can1: link becomes ready hiawatha. dbus[1832]: [system] Activating service name='fi.w1.wpa_supplicant1' (using servicehelper) Starting crond: dbus[1832]: [system] Successfully activated service 'fi.w1.wpa_supplicant1' OK [ 27.518031] [ 27.519624] ============================================= [ 27.525281] [ INFO: possible recursive locking detected ] [ 27.530944] 4.1.15-venus #1 Tainted: G O [ 27.536236] --------------------------------------------- [ 27.541892] connmand/1959 is trying to acquire lock: [ 27.547094] (pmutex){+.+...}, at: [<bf0ef664>] usbctrl_vendorreq+0x7c/0x274 [8723bu] [ 27.555649] [ 27.555649] but task is already holding lock: [ 27.561764] (pmutex){+.+...}, at: [<bf0ee454>] netdev_open+0x28/0x50 [8723bu] [ 27.569468] [ 27.569468] other info that might help us debug this: [ 27.576311] Possible unsafe locking scenario: [ 27.576311] [ 27.582516] CPU0 [ 27.585075] ---- [ 27.587632] lock(pmutex); [ 27.590565] lock(pmutex); [ 27.593497] [ 27.593497] *** DEADLOCK *** [ 27.593497] [ 27.599704] May be due to missing lock nesting notation [ 27.599704] [ 27.606822] 2 locks held by connmand/1959: [ 27.611111] #0: (rtnl_mutex){+.+.+.}, at: [<c042d050>] devinet_ioctl+0xd0/0x7d4 [ 27.618996] #1: (pmutex){+.+...}, at: [<bf0ee454>] netdev_open+0x28/0x50 [8723bu] [ 27.627136] [ 27.627136] stack backtrace: [ 27.631706] CPU: 0 PID: 1959 Comm: connmand Tainted: G O 4.1.15-venus #1 [ 27.639914] Hardware name: Generic AM33XX (Flattened Device Tree) [ 27.646326] [<c0014844>] (unwind_backtrace) from [<c0012994>] (show_stack+0x10/0x14) [ 27.654455] [<c0012994>] (show_stack) from [<c0067fe0>] (__lock_acquire+0x1454/0x1da0) [ 27.662759] [<c0067fe0>] (__lock_acquire) from [<c006922c>] (lock_acquire+0xac/0x12c) [ 27.670979] [<c006922c>] (lock_acquire) from [<c04bffd4>] (mutex_lock_interruptible_nested+0x48/0x3bc) [ 27.680833] [<c04bffd4>] (mutex_lock_interruptible_nested) from [<bf0ef664>] (usbctrl_vendorreq+0x7c/0x274 [8723bu]) [ 27.692079] [<bf0ef664>] (usbctrl_vendorreq [8723bu]) from [<bf11e9f0>] (usb_read8+0x30/0x3c [8723bu]) [ 27.702011] [<bf11e9f0>] (usb_read8 [8723bu]) from [<bf0c1178>] (rtw_read8+0x18/0x1c [8723bu]) [ 27.711204] [<bf0c1178>] (rtw_read8 [8723bu]) from [<bf11ceb0>] (rtl8723bu_hal_init+0x30/0xb4c [8723bu]) [ 27.721338] [<bf11ceb0>] (rtl8723bu_hal_init [8723bu]) from [<bf1026b4>] (rtw_hal_init+0x18/0x154 [8723bu]) [ 27.731727] [<bf1026b4>] (rtw_hal_init [8723bu]) from [<bf0ee20c>] (_netdev_open+0x64/0x284 [8723bu]) [ 27.741561] [<bf0ee20c>] (_netdev_open [8723bu]) from [<bf0ee45c>] (netdev_open+0x30/0x50 [8723bu]) [ 27.751143] [<bf0ee45c>] (netdev_open [8723bu]) from [<c03c8884>] (__dev_open+0xb8/0x11c) [ 27.759726] [<c03c8884>] (__dev_open) from [<c03c8b08>] (__dev_change_flags+0x94/0x144) [ 27.768123] [<c03c8b08>] (__dev_change_flags) from [<c03c8bd0>] (dev_change_flags+0x18/0x48) [ 27.776978] [<c03c8bd0>] (dev_change_flags) from [<c042d64c>] (devinet_ioctl+0x6cc/0x7d4) [ 27.785559] [<c042d64c>] (devinet_ioctl) from [<c03ae70c>] (sock_ioctl+0x1d8/0x2b0) [ 27.793593] [<c03ae70c>] (sock_ioctl) from [<c011c39c>] (do_vfs_ioctl+0x3f0/0x5c0) [ 27.801533] [<c011c39c>] (do_vfs_ioctl) from [<c011c5a0>] (SyS_ioctl+0x34/0x5c) [ 27.809201] [<c011c5a0>] (SyS_ioctl) from [<c000f1e0>] (ret_fast_syscall+0x0/0x54) Checking available software versions... [ 28.191668] EXT4-fs (mmcblk1p2): mounted filesystem with ordered data mode. Opts: (null) Starting simple-upnpd [ 31.095134] RTL871X: RTW_ADAPTIVITY_EN_AUTO, chplan:0x20, Regulation:0,0 [ 31.102220] RTL871X: RTW_ADAPTIVITY_MODE_NORMAL [ 32.523712] IPv6: ADDRCONF(NETDEV_UP): wlan0: link is not ready [ 32.536569] cfg80211: Calling CRDA to update world regulatory domain [ 32.604106] c_can_platform 481d0000.can can1: setting BTR=1c05 BRPE=0000 [ 32.611164] c_can_platform 481d0000.can: obtain a copy of previously claimed pinctrl [ 32.619469] IPv6: ADDRCONF(NETDEV_CHANGE): can1: link becomes ready [ 33.212819] c_can_platform 481d0000.can can1: setting BTR=1c05 BRPE=0000 [ 33.220096] c_can_platform 481d0000.can: obtain a copy of previously claimed pinctrl [ 33.291001] IPv6: ADDRCONF(NETDEV_CHANGE): can1: link becomes ready [ 33.704635] RTL871X: nolinked power save enter [ 39.083257] RTL871X: RTW_ADAPTIVITY_EN_AUTO, chplan:0x20, Regulation:0,0 [ 39.090326] RTL871X: RTW_ADAPTIVITY_MODE_NORMAL [ 40.204562] RTL871X: nolinked power save leave [ 40.425420] RTL871X: rtw_set_802_11_connect(wlan0) fw_state=0x00000008 [ 40.432584] cfg80211: Calling CRDA to update world regulatory domain [ 40.498385] RTL871X: start auth [ 40.503755] RTL871X: auth success, start assoc [ 40.512793] RTL871X: rtw_cfg80211_indicate_connect(wlan0) BSS not found !! [ 40.520058] RTL871X: assoc success [ 40.524101] IPv6: ADDRCONF(NETDEV_CHANGE): wlan0: link becomes ready [ 40.956425] RTL871X: send eapol packet [ 40.978771] RTL871X: send eapol packet [ 40.994451] RTL871X: set pairwise key camid:4, addr:98:fc:11:ae:b5:04, kid:0, type:AES bgscan simple: Failed to enable signal strength monitoring [ 41.024333] RTL871X: set group key camid:5, addr:98:fc:11:ae:b5:04, kid:1, type:AES [ 43.582737] cfg80211: Calling CRDA to update world regulatory domain [ 46.732739] cfg80211: Exceeded CRDA call max attempts. Not calling CRDA
|
@lwfinger I went ahead and made a clean build of the whole system to cut & paste the bootlog for you, but now the behavior has changed (...): It does reboot now, but only after seeing a kernel deadlock message. See full log below. If I move that .ko file out of its normal location to make sure it isn't loaded, all works fine again: no kernel deadlocks. Note that this is a different distro than the IoT one from the Beaglebone organisation. What I am running on the Beaglebone Enhanced is our Venus distro. Similar to @RobertCNelson, I just compile this rtl8723bu driver with its default settings and add it to the rootfs. See the 'add rtl8723bu driver for wifi on bbe' commit for details. The used kernel sources & config is here: For anyone interested, binary installer image can be downloaded here: |
|
As an aside, blacklisting a driver is a lot easier than moving it. In addition, you do not have crda installed in your system. It isn't necessary, but having it would remove all those failure messages from your log. Did the network work is the case above? The possible deadlock warning is something that needs attention, but it usually is benign. In addition, it only appears on removal of the device. The WARNING: CPU: 0 PID: 1791 at /media/venusbuilds/wipbbbjethro/build/tmp-glibc/work-shared/beaglebone/kernel-source/net/wireless/core.c:778 wiphy_unregister+0x2b4/0x2b8 cfg80211 may be responsible for the locking warning. |
|
Here's some more details: call reboot: (without the usb module installed) |
|
I have found the system is more reliable at rebooting when it isn't associated with an AP. |
|
I am a little confused. On reboot, the NIC is never associated with any AP. I understand that you might have problems shutting down while associated. Is that what is happening? |
|
I just found a lock that is not needed and pushed a fix. Does this help? |
|
@lwfinger wrote:
Yes, thanks. I took crda out a while ago, but it can go back in now.
Yes. @sancloud wrote:
there is a bit more nuance to it: just enabling wifi (using connman) is enough to make it get stuck. @lwfinger wrote:
No. (After enabling wifi) it is still stuck at rebooting. Also there is a deadlock error when enabling wifi: And that is where it was stuck again. Also there is still the deadlock, but successful reboot, when rebooting without first enabling wifi. The message is a bit different though: the task holding the lock is no longer I tried the old version again, to make sure this is no coincidence. And indeed, with old image the task holding the lock back to Log of not enabling wifi, and then reboot, with latest (bc9e782) version of rtl8723bu driver: |
|
Using the latest it seems to be stable now at rebooting. |
mpvader
added a commit
to victronenergy/meta-victronenergy
that referenced
this issue
Sep 30, 2016
Make sure to add this commit from upstream in the openembedded-core mete: openembedded/openembedded-core@afcea61 It's needed to prevent these errors during compile: WARNING: "wiphy_free" [/media/venusbuilds/wipbbbjethro/build/tmp-glibc/work/beaglebone-ve-linux-gnueabi/rtl8723bu/0.0-r0/git/8723bu.ko] undefined! WARNING: "cfg80211_unlink_bss" [/media/venusbuilds/wipbbbjethro/build/tmp-glibc/work/beaglebone-ve-linux-gnueabi/rtl8723bu/0.0-r0/git/8723bu.ko] undefined! [many more] and later, when using the ko file, there errors in the bootlog: [ 4.554499] 8723bu: no symbol version for cfg80211_scan_done [ 4.560468] 8723bu: Unknown symbol cfg80211_scan_done (err -22) ... many more, resulting in module not loading unless doing a manual modprobe -f OPEN ISSUES 0) I added lib80211 to the defconfig at some point while trying to fix above issue. Probably isn't needed anymore. Todo: test & remove 1) Replace the hack in the rtl8723bu Makefile with something proper 2) Reboot doesn't work lwfinger/rtl8723bu#36 3) Kernel deadlocks right after boot: [...] [ 3.544110] udevd[677]: starting version 182 [ 4.125070] cfg80211: Calling CRDA to update world regulatory domain [ 4.499920] RTL871X: module init start [ 4.503973] RTL871X: rtl8723bu v4.3.6.11_12942.20141204_BTCOEX20140507-4E40 [ 4.511273] RTL871X: rtl8723bu BT-Coex version = BTCOEX20140507-4E40 [ 7.276123] cfg80211: Calling CRDA to update world regulatory domain [ 7.816563] RTL871X: rtw_ndev_init(wlan0) [ 7.823760] usbcore: registered new interface driver rtl8723bu [ 7.829887] RTL871X: module init ret=0 [ 8.275916] EXT4-fs (mmcblk1p3): re-mounted. Opts: data=ordered bootlogd: cannot allocate pseudo tty: No such file or directory [ 8.419817] random: dd urandom read with 105 bits of entropy available Populating dev cache [ 10.432973] cfg80211: Calling CRDA to update world regulatory domain hwclock: can't open '/dev/misc/rtc': No such file or directory Thu Sep 15 23:48:31 UTC 2016 hwclock: can't open '/dev/misc/rtc': No such file or directory Starting watchdog [ 11.915814] watchdog watchdog0: watchdog did not stop! [ 13.582859] cfg80211: Calling CRDA to update world regulatory domain [ 14.169740] random: nonblocking pool is initialized update-rc.d: /etc/init.d/run-postinsts exists during rc.d purge (continuing) Removing any system startup links for run-postinsts ... /etc/rcS.d/S99run-postinsts INIT: Entering runlevel: 5 Configuring network interfaces... [ 14.520549] net eth0: initializing cpsw version 1.12 (0) [ 14.603248] net eth0: phy found : id is : 0x4dd072 [ 14.608427] libphy: PHY 4a101000.mdio:01 not found [ 14.613487] net eth0: phy 4a101000.mdio:01 not found on slave 1 [ 14.625689] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready udhcpc (v1.23.2) started Sending discover... [ 16.732824] cfg80211: Calling CRDA to update world regulatory domain Sending discover... [ 19.882724] cfg80211: Calling CRDA to update world regulatory domain Sending discover... [ 23.032729] cfg80211: Calling CRDA to update world regulatory domain No lease, forking to background [ 23.865391] c_can_platform 481d0000.can can1: setting BTR=1c05 BRPE=0000 done. Starting system message bus: Setting up watches. Watches established. /var/run/dbus/ CREATE system_bus_socket hwclock: can't open '/dev/misc/rtc': No such file or directory Starting syslogd/klogd: done Starting php-fpm [ 25.190096] c_can_platform 481d0000.can can1: setting BTR=1c05 BRPE=0000 [ 25.197402] IPv6: ADDRCONF(NETDEV_CHANGE): can1: link becomes ready [ 25.991495] c_can_platform 481d0000.can can1: setting BTR=1c05 BRPE=0000 [ 25.998758] IPv6: ADDRCONF(NETDEV_CHANGE): can1: link becomes ready [ 26.182841] cfg80211: Calling CRDA to update world regulatory domain done Starting Connection Manager Starting Hiawatha Web Server: [ 26.981252] c_can_platform 481d0000.can can1: setting BTR=1c05 BRPE=0000 [ 26.988575] IPv6: ADDRCONF(NETDEV_CHANGE): can1: link becomes ready hiawatha. dbus[1832]: [system] Activating service name='fi.w1.wpa_supplicant1' (using servicehelper) Starting crond: dbus[1832]: [system] Successfully activated service 'fi.w1.wpa_supplicant1' OK [ 27.518031] [ 27.519624] ============================================= [ 27.525281] [ INFO: possible recursive locking detected ] [ 27.530944] 4.1.15-venus #1 Tainted: G O [ 27.536236] --------------------------------------------- [ 27.541892] connmand/1959 is trying to acquire lock: [ 27.547094] (pmutex){+.+...}, at: [<bf0ef664>] usbctrl_vendorreq+0x7c/0x274 [8723bu] [ 27.555649] [ 27.555649] but task is already holding lock: [ 27.561764] (pmutex){+.+...}, at: [<bf0ee454>] netdev_open+0x28/0x50 [8723bu] [ 27.569468] [ 27.569468] other info that might help us debug this: [ 27.576311] Possible unsafe locking scenario: [ 27.576311] [ 27.582516] CPU0 [ 27.585075] ---- [ 27.587632] lock(pmutex); [ 27.590565] lock(pmutex); [ 27.593497] [ 27.593497] *** DEADLOCK *** [ 27.593497] [ 27.599704] May be due to missing lock nesting notation [ 27.599704] [ 27.606822] 2 locks held by connmand/1959: [ 27.611111] #0: (rtnl_mutex){+.+.+.}, at: [<c042d050>] devinet_ioctl+0xd0/0x7d4 [ 27.618996] #1: (pmutex){+.+...}, at: [<bf0ee454>] netdev_open+0x28/0x50 [8723bu] [ 27.627136] [ 27.627136] stack backtrace: [ 27.631706] CPU: 0 PID: 1959 Comm: connmand Tainted: G O 4.1.15-venus #1 [ 27.639914] Hardware name: Generic AM33XX (Flattened Device Tree) [ 27.646326] [<c0014844>] (unwind_backtrace) from [<c0012994>] (show_stack+0x10/0x14) [ 27.654455] [<c0012994>] (show_stack) from [<c0067fe0>] (__lock_acquire+0x1454/0x1da0) [ 27.662759] [<c0067fe0>] (__lock_acquire) from [<c006922c>] (lock_acquire+0xac/0x12c) [ 27.670979] [<c006922c>] (lock_acquire) from [<c04bffd4>] (mutex_lock_interruptible_nested+0x48/0x3bc) [ 27.680833] [<c04bffd4>] (mutex_lock_interruptible_nested) from [<bf0ef664>] (usbctrl_vendorreq+0x7c/0x274 [8723bu]) [ 27.692079] [<bf0ef664>] (usbctrl_vendorreq [8723bu]) from [<bf11e9f0>] (usb_read8+0x30/0x3c [8723bu]) [ 27.702011] [<bf11e9f0>] (usb_read8 [8723bu]) from [<bf0c1178>] (rtw_read8+0x18/0x1c [8723bu]) [ 27.711204] [<bf0c1178>] (rtw_read8 [8723bu]) from [<bf11ceb0>] (rtl8723bu_hal_init+0x30/0xb4c [8723bu]) [ 27.721338] [<bf11ceb0>] (rtl8723bu_hal_init [8723bu]) from [<bf1026b4>] (rtw_hal_init+0x18/0x154 [8723bu]) [ 27.731727] [<bf1026b4>] (rtw_hal_init [8723bu]) from [<bf0ee20c>] (_netdev_open+0x64/0x284 [8723bu]) [ 27.741561] [<bf0ee20c>] (_netdev_open [8723bu]) from [<bf0ee45c>] (netdev_open+0x30/0x50 [8723bu]) [ 27.751143] [<bf0ee45c>] (netdev_open [8723bu]) from [<c03c8884>] (__dev_open+0xb8/0x11c) [ 27.759726] [<c03c8884>] (__dev_open) from [<c03c8b08>] (__dev_change_flags+0x94/0x144) [ 27.768123] [<c03c8b08>] (__dev_change_flags) from [<c03c8bd0>] (dev_change_flags+0x18/0x48) [ 27.776978] [<c03c8bd0>] (dev_change_flags) from [<c042d64c>] (devinet_ioctl+0x6cc/0x7d4) [ 27.785559] [<c042d64c>] (devinet_ioctl) from [<c03ae70c>] (sock_ioctl+0x1d8/0x2b0) [ 27.793593] [<c03ae70c>] (sock_ioctl) from [<c011c39c>] (do_vfs_ioctl+0x3f0/0x5c0) [ 27.801533] [<c011c39c>] (do_vfs_ioctl) from [<c011c5a0>] (SyS_ioctl+0x34/0x5c) [ 27.809201] [<c011c5a0>] (SyS_ioctl) from [<c000f1e0>] (ret_fast_syscall+0x0/0x54) Checking available software versions... [ 28.191668] EXT4-fs (mmcblk1p2): mounted filesystem with ordered data mode. Opts: (null) Starting simple-upnpd [ 31.095134] RTL871X: RTW_ADAPTIVITY_EN_AUTO, chplan:0x20, Regulation:0,0 [ 31.102220] RTL871X: RTW_ADAPTIVITY_MODE_NORMAL [ 32.523712] IPv6: ADDRCONF(NETDEV_UP): wlan0: link is not ready [ 32.536569] cfg80211: Calling CRDA to update world regulatory domain [ 32.604106] c_can_platform 481d0000.can can1: setting BTR=1c05 BRPE=0000 [ 32.611164] c_can_platform 481d0000.can: obtain a copy of previously claimed pinctrl [ 32.619469] IPv6: ADDRCONF(NETDEV_CHANGE): can1: link becomes ready [ 33.212819] c_can_platform 481d0000.can can1: setting BTR=1c05 BRPE=0000 [ 33.220096] c_can_platform 481d0000.can: obtain a copy of previously claimed pinctrl [ 33.291001] IPv6: ADDRCONF(NETDEV_CHANGE): can1: link becomes ready [ 33.704635] RTL871X: nolinked power save enter [ 39.083257] RTL871X: RTW_ADAPTIVITY_EN_AUTO, chplan:0x20, Regulation:0,0 [ 39.090326] RTL871X: RTW_ADAPTIVITY_MODE_NORMAL [ 40.204562] RTL871X: nolinked power save leave [ 40.425420] RTL871X: rtw_set_802_11_connect(wlan0) fw_state=0x00000008 [ 40.432584] cfg80211: Calling CRDA to update world regulatory domain [ 40.498385] RTL871X: start auth [ 40.503755] RTL871X: auth success, start assoc [ 40.512793] RTL871X: rtw_cfg80211_indicate_connect(wlan0) BSS not found !! [ 40.520058] RTL871X: assoc success [ 40.524101] IPv6: ADDRCONF(NETDEV_CHANGE): wlan0: link becomes ready [ 40.956425] RTL871X: send eapol packet [ 40.978771] RTL871X: send eapol packet [ 40.994451] RTL871X: set pairwise key camid:4, addr:98:fc:11:ae:b5:04, kid:0, type:AES bgscan simple: Failed to enable signal strength monitoring [ 41.024333] RTL871X: set group key camid:5, addr:98:fc:11:ae:b5:04, kid:1, type:AES [ 43.582737] cfg80211: Calling CRDA to update world regulatory domain [ 46.732739] cfg80211: Exceeded CRDA call max attempts. Not calling CRDA
mpvader
added a commit
to victronenergy/meta-victronenergy
that referenced
this issue
Oct 9, 2016
Make sure to add this commit from upstream to openembedded-core: openembedded/openembedded-core@afcea61 It's needed to prevent these errors during compile: WARNING: "wiphy_free" [/media/venusbuilds/wipbbbjethro/build/tmp-glibc/work/beaglebone-ve-linux-gnueabi/rtl8723bu/0.0-r0/git/8723bu.ko] undefined! WARNING: "cfg80211_unlink_bss" [/media/venusbuilds/wipbbbjethro/build/tmp-glibc/work/beaglebone-ve-linux-gnueabi/rtl8723bu/0.0-r0/git/8723bu.ko] undefined! [many more] And, without that upstream commit, there are also these errors in the bootlog: [ 4.554499] 8723bu: no symbol version for cfg80211_scan_done [ 4.560468] 8723bu: Unknown symbol cfg80211_scan_done (err -22) [many more] resulting in module not loading unless doing a manual modprobe -f Remaining issues: 1) there is a kernel deadlock warning while the driver is loaded. lwfinger/rtl8723bu#40 2) there is some error message during driver unload/reboot. Less important, and less bad than it used to be: the device would get stuck, but that is fixed already: lwfinger/rtl8723bu#36
mpvader
added a commit
to victronenergy/meta-victronenergy
that referenced
this issue
Oct 10, 2016
Also add this commit from upstream to openembedded-core: openembedded/openembedded-core@afcea61 It is needed to prevent these errors during compile: WARNING: "wiphy_free" [/media/venusbuilds/wipbbbjethro/build/tmp-glibc/work/beaglebone-ve-linux-gnueabi/rtl8723bu/0.0-r0/git/8723bu.ko] undefined! WARNING: "cfg80211_unlink_bss" [/media/venusbuilds/wipbbbjethro/build/tmp-glibc/work/beaglebone-ve-linux-gnueabi/rtl8723bu/0.0-r0/git/8723bu.ko] undefined! [many more] And, without that upstream commit, there are also these errors in the bootlog: [ 4.554499] 8723bu: no symbol version for cfg80211_scan_done [ 4.560468] 8723bu: Unknown symbol cfg80211_scan_done (err -22) [many more] resulting in module not loading unless doing a manual modprobe -f There are two harmless visible kernel deadlock warnings: 1) during loading of the driver. Harmless according to lwfinger, the maintainer: lwfinger/rtl8723bu#40 2) during unloading of the driver. Less bad than it used to be: the device was getting stuck during unload, but that has been fixed: lwfinger/rtl8723bu#36
mpvader
added a commit
to victronenergy/meta-victronenergy
that referenced
this issue
Oct 14, 2016
Also add this commit from upstream to openembedded-core: openembedded/openembedded-core@afcea61 Needed to prevent warnings exactly like mentioned in that commit message. There are -still- two harmless visible kernel deadlock warnings: 1) when loading of the driver. Which is according to the maintainer, lwfinger, harmless: lwfinger/rtl8723bu#40 2) when unloading of the driver. Less bad than it used to be: the device was getting stuck during unload, but that has been fixed: lwfinger/rtl8723bu#36
jhofstee
pushed a commit
to victronenergy/meta-victronenergy
that referenced
this issue
Oct 20, 2016
Also add this commit from upstream to openembedded-core: openembedded/openembedded-core@afcea61 Needed to prevent warnings exactly like mentioned in that commit message. There are -still- two harmless visible kernel deadlock warnings: 1) when loading of the driver. Which is according to the maintainer, lwfinger, harmless: lwfinger/rtl8723bu#40 2) when unloading of the driver. Less bad than it used to be: the device was getting stuck during unload, but that has been fixed: lwfinger/rtl8723bu#36
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Hi,
with the rtl8723bu driver loaded, my device won't reboot anymore. Its 100% reproducible. And solved by booting without loading the driver.
system log output is:
the used device is a beaglebone enhanced from Sancloud, running our own image called Venus. So not the IoT beaglebone image. I am still awaiting feedback to know if this same issue happens on their beaglebone IoT image, as I know they use this same driver.
Any ideas? Is this a known issue?
Matthijs
ps. I have this issue (also) when not working with the concurrent_mode. About which I made that other issue for last night, #35. So thought it best to keep these two issues seperate. Also I have some more compile errors with the concurrent_mode. And fixes for them this time ;-). I'll post that in #35, after doing some more tests.
The text was updated successfully, but these errors were encountered: