Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

deadlock after ed7944f4057a2f96a386267b9e14973ae6e021df on device error #375

Closed
kilspi opened this issue Apr 23, 2021 · 8 comments
Closed

deadlock after ed7944f4057a2f96a386267b9e14973ae6e021df on device error #375

kilspi opened this issue Apr 23, 2021 · 8 comments

Comments

@kilspi
Copy link

kilspi commented Apr 23, 2021

ERROR: Data source 'ace_bt / hci0:name="ace_bt"' ('hci0') encountered an error: IPC connection closed
ALERT: SOURCEERROR Source ace_bt has encountered an error. Kismet will attempt to re-open the source in 5 seconds. (1 failures)
ERROR: Source ace_bt has encountered an error. Kismet will attempt to re-open the source in 5 seconds. (1 failures)
FATAL: Capture source 737251 did not get PING from Kismet for over 15 seconds; shutting down
ERROR: Data source 'ace_Archer_T2U / wlp0s20u2u1:name="ace_Archer_T2U"' ('wlp0s20u2u1') encountered an error: IPC connection closed
ALERT: SOURCEERROR Source ace_Archer_T2U has encountered an error. Kismet will attempt to re-open the source in 5 seconds. (1 failures)
ERROR: Source ace_Archer_T2U has encountered an error. Kismet will attempt to re-open the source in 5 seconds. (1 failures)
Uncaught exception "potential deadlock: mutex devicetracker::devicelist not available within timeout period for op UNKNOWN"
Stack trace (most recent call last) in thread 737208:
#11 Object ", at 0xffffffffffffffff, in
#10 Object "/usr/lib/libc-2.33.so, at 0x7f23b9ffd052, in __clone
#9 Object "/usr/lib/libpthread-2.33.so, at 0x7f23ba55d298, in start_thread
#8 Source "/build/gcc/src/gcc/libstdc++-v3/src/c++11/thread.cc", line 80, in execute_native_thread_routine
#7 Source "/home/chris/src/kismet/packetchain.cc", line 339, in packet_queue_processor
337: for (const auto& pcl : classifier_chain) {
338: if (pcl->callback != nullptr)
> 339: pcl->callback(Globalreg::globalreg, pcl->auxdata, packet);
340: else if (pcl->l_callback != nullptr)
341: pcl->l_callback(packet);
342: }
#6 Source "/home/chris/src/kismet/phy_80211.cc", line 1494, in packet_dot11_common_classifier
1492: bssid_dev =
1493: d11phy->devicetracker->update_common_device(commoninfo,
>1494: dot11info->bssid_mac, d11phy, in_pack,
1495: (update_flags | bflags),
1496: "Wi-Fi Device");
1497: }
#5 Source "/home/chris/src/kismet/devicetracker.cc", line 1232, in update_common_device
1230: // Lock before we look up the device, we can't have two threads making a new
1231: // device record for the same device
>1232: ul_list.lock();
1233:
1234: if ((device = fetch_device_nr(key)) == NULL) {
1235: if (in_flags & UCD_UPDATE_EXISTING_ONLY)
#4 Source "/home/chris/src/kismet/kis_mutex.h", line 284, in lock
281: locked = true;
282:
283: if (!mutex.try_lock_for(std::chrono::seconds(KIS_THREAD_TIMEOUT)))
> 284: throw std::runtime_error(fmt::format("potential deadlock: mutex {} not available within "
285: "timeout period for op {}", mutex.get_name(), op));
286: }
#3 Source "/build/gcc/src/gcc/libstdc++-v3/libsupc++/eh_throw.cc", line 95, in __cxa_throw
#2 Source "/build/gcc/src/gcc/libstdc++-v3/libsupc++/eh_terminate.cc", line 58, in terminate
#1 Source "/build/gcc/src/gcc/libstdc++-v3/libsupc++/eh_terminate.cc", line 48, in __builtin_abort
#0 Source "/home/chris/src/kismet/kismet_server.cc", line 301, in TerminationHandler
299: #ifndef DISABLE_BACKWARD
300: using namespace backward;
> 301: StackTrace st; st.load_here(32);
302: Printer p; p.print(st);
303: #endif
Uncaught exception "std::exception"
Uncaught exception "potential deadlock: mutex devicetracker::devicelist not available within timeout period for op UNKNOWN"
@dragorn
Copy link
Contributor

dragorn commented Apr 23, 2021 via email

@kilspi
Copy link
Author

kilspi commented Apr 23, 2021

was latest GIT from today. Switching back to c580a95 solved the problem immediately

@kismetwireless
Copy link
Owner

OK; unfortunately that commit is absolutely vital for fixing widespread memory corruption in threads.

Made another test commit that drops device-level locking entirely and locks at the device list level for everything. It runs fine here - but so did the previous commits, for multiple days, so it's difficult to test directly.

@kilspi
Copy link
Author

kilspi commented Apr 23, 2021

I tried the test commit, same result, crashes and is restarted after 30-60 seconds, just like before. I had to step back again. Maybe I can try again with gdb on the weekend

@kilspi kilspi changed the title dealock after ed7944f4057a2f96a386267b9e14973ae6e021df on device error deadlock after ed7944f4057a2f96a386267b9e14973ae6e021df on device error Apr 25, 2021
@kilspi
Copy link
Author

kilspi commented May 25, 2021 •
edited

kismet_dump.txt

still happening with 2021-05, logs and dump added

PS sent mail with gdb dump

@kismetwireless
Copy link
Owner

Try the latest commits (or tomorrows nightlies); it looks like there may have been major problems with the timed thread locking mechanisms.

@kilspi
Copy link
Author

kilspi commented May 28, 2021

Looks promising running for over 1h now without crashing

@kilspi kilspi closed this as completed May 31, 2021
@kilspi
Copy link
Author

kilspi commented May 31, 2021

closing this after running three days without deadlocks. thx!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@dragorn @kismetwireless @kilspi