Crash upon connection from Ignition server #62
Comments
|
It has been a long time, but by looking at the code around it, it looks like a race condition when accessing the // static methods to reimplement custom behaviour
/*
config->accessControl.activateSession = &QUaServer::activateSession;
config->accessControl.closeSession = &QUaServer::closeSession;
config->accessControl.getUserRightsMask = &QUaServer::getUserRightsMask;
config->accessControl.getUserAccessLevel = &QUaServer::getUserAccessLevel;
config->accessControl.getUserExecutable = &QUaServer::getUserExecutable;
config->accessControl.getUserExecutableOnObject = &QUaServer::getUserExecutableOnObject;
*/You would have to check if this would break anything else, since I cannot remember the details anymore. |
|
You can also give a try to https://github.com/juangburgos/QCrashHandler if you want to obtain a mini dump that can shed some light into the issue, although I give the same support guarantees as for this library 😅 (welcome to open source). |
|
Thank you for your reply. I have the gdb trace which is enough (thanks for pointing me out to QCrashHandler, though). I will try to see the open62541 code (though it is daunting to me). I have one observation, maybe it will make sense to you. The crash happens when there are many OPC/UA connections already (about 30, over 4 channels — that's what logs say; monitoring some 800 items — sorry, I am not responsible for that...). And right before the crash, there is always a new connection (and another one closing — interestingly; shortly before or shortly after), and a new connection with "revised lifetime of 600.00s" is opened. Do you see any causality there, a pointer where to look? Does that confirm that the session needs to be locked? |
It is daunting, actually I never really went through it all, just by parts. My recomendation, make a local debug session and put a breakpoint in the place where the crash occurs in production. Then once you are stopped at that point, go on and inspect the functions from open62541 that appear up in the stack of the debugger. In the end is C code, and from what I remember, it is well written, good variable names and understandable. See if something stands out, some mutex that gets unlocked before calling the user-land callback. Find all references to variables of interest and explore a llittle bit around. Sorry I cannot do this research for you, I am already retired from the industrial OT world due to low wages and bad working conditions. I moved to the IT side of things, grass is greener if you ever want to make the change 😜 |
|
The traceback is in the log above; I looked at Service_ActivateSession, but will have to look a bit more :) There are no UA_* functions in other threads; can there still be race condition even if the UA server serves from single thread? |
|
I wanted to quickly build the latest master and take a look to see if I could spot something, but it seems master does not build with Qt5 anymore 😆 . Sorry amigo, I think you are on your own on this one, or maybe @sergey-kuzminov can help you out, he seems to be recently using the library. |
I am getting consistent crashes when Ignition client is connecting to QUaServer. Line 775 (where the crash is reported) is this line (a slightly old versionof QUaServer, Dec 2021, there):
QUaServer/src/wrapper/quaserver.cpp
Line 787 in 0d0e5c2
Can someone shed light on this? The code is running on a sit which is difficult to access (in docker, so also not easy to debug interactively, put in extra logs etc). It did not happen before the client upgraded their Ignition software (I would remember that).
This is stack trace from the thread where the crash happens (other threads are in poll, syscall or sleep; and all crashes happen here):
The text was updated successfully, but these errors were encountered: