Bugfix/3169 stack fix #3182

davidnich · 2018-12-13T21:17:40Z

No description provided.

… stack implementation; the HAVE_RUNTIME_THREAD_STACK_TRACE constant is always True, removed associated build options and internal logic

davidnich · 2018-12-13T21:18:39Z

allows the jni module to make exceptions on demand (no cost with no exception) like:

System exception: ../test/jni.qtest:127: NO-FUNCTION: function name 'does_not_exist' does not exist
call stack:
  org.qore.jni.QoreJavaApi.callFunction0() called at QoreJavaApi.java:-2 (Java builtin code)
  org.qore.jni.QoreJavaApi.callFunction() called at QoreJavaApi.java:15 (Java user code)
  org.qore.jni.test.QoreJavaApiTest.callFunctionTest() called at QoreJavaApiTest.java:94 (Java user code)
  QoreJavaApiTest::callFunctionTest() called at ../test/jni.qtest:127 (Qore builtin code)
  Main::exceptionStackTest() called at /Users/david/src/Qorus/test/qlib/QUnit.qm:457 (Qore user code)
  call_function_args() called at /Users/david/src/Qorus/test/qlib/QUnit.qm:457 (Qore builtin code)
  TestCase::run() called at /Users/david/src/Qorus/test/qlib/QUnit.qm:2008 (Qore user code)
  Test::main() called at ../test/jni.qtest:122 (Qore user code)
  Main::constructor() called at :-1 (Qore user code)

…ded language info to exceptions, updates tests and docs

davidnich · 2018-12-14T10:05:40Z

now a native Java example:

QUnit Test "jni test" v1.0
System exception: NativeMethodAccessorImpl.java:-2 (Java): JNI-ERROR: java.lang.reflect.InvocationTargetException (arg: {<InvocationTargetException object>})
call stack:
  java.lang.reflect.Method.invoke() called at Method.java:498 (Java user code)
  sun.reflect.DelegatingMethodAccessorImpl.invoke() called at DelegatingMethodAccessorImpl.java:43 (Java user code)
  sun.reflect.NativeMethodAccessorImpl.invoke() called at NativeMethodAccessorImpl.java:62 (Java user code)
  Method::invoke() called at ../test/jni.qtest:722 (Qore builtin code)
  Main::testCallback() called at /Users/david/src/Qorus/test/qlib/QUnit.qm:457 (Qore user code)
  call_function_args() called at /Users/david/src/Qorus/test/qlib/QUnit.qm:457 (Qore builtin code)
  TestCase::run() called at /Users/david/src/Qorus/test/qlib/QUnit.qm:2008 (Qore user code)
  Test::main() called at ../test/jni.qtest:122 (Qore user code)
  Main::constructor() called at :-1 (Qore user code)
Ran 23 test cases, 23 succeeded (212 assertions)

and Qore from Java from Qore:

System exception: ../test/jni.qtest:127 (Qore): NO-FUNCTION: function name 'does_not_exist' does not exist
call stack:
  org.qore.jni.QoreJavaApi.callFunction0() called at QoreJavaApi.java:-2 (Java builtin code)
  org.qore.jni.QoreJavaApi.callFunction() called at QoreJavaApi.java:15 (Java user code)
  org.qore.jni.test.QoreJavaApiTest.callFunctionTest() called at QoreJavaApiTest.java:94 (Java user code)
  QoreJavaApiTest::callFunctionTest() called at ../test/jni.qtest:127 (Qore builtin code)
  Main::exceptionStackTest() called at /Users/david/src/Qorus/test/qlib/QUnit.qm:457 (Qore user code)
  call_function_args() called at /Users/david/src/Qorus/test/qlib/QUnit.qm:457 (Qore builtin code)
  TestCase::run() called at /Users/david/src/Qorus/test/qlib/QUnit.qm:2008 (Qore user code)
  Test::main() called at ../test/jni.qtest:122 (Qore user code)
  Main::constructor() called at :-1 (Qore user code)

davidnich · 2018-12-14T10:21:58Z

@sejvlond now this is ready for review - call stack info is only created when necessary - when there is a request for the call stack or an exception is raised. Modules providing language integration (ex jni) add their stack info dynamically on demand. There are no more heap allocations made for each call in Qore to maintain a separate heap-allocated call stack - the call stack is stored on each thread's stack.

Also a lang key has been added to the exception info + call stack entries.

The default exception and warning handlers share the same code for formatting the output - it's in one place now, so if we want to rework it to make it more easily machine parsable, we can do so.

Looking forward to your c++ comments :)

…acks

sejvlond · 2018-12-13T21:21:08Z

include/qore/ExceptionSink.h

    DLLEXPORT const QoreProgramLocation& get() const {
        return *loc;
    }

+    //! returns the file name
+    DLLEXPORT const char* getFile() const {
+        return file_str.c_str();


why returning const char * instead of string?

sejvlond · 2018-12-13T21:22:33Z

include/qore/ExceptionSink.h

+    DLLLOCAL QoreStackLocation& operator=(const QoreStackLocation&) = delete;
+
+    //! no move assignment operator
+    DLLLOCAL QoreStackLocation& operator=(QoreStackLocation&&) = delete;


what's the reason for deleting copy and move =operators? it looks that the member pointer is not owned and copies are just shallow copies..

sejvlond · 2018-12-13T21:27:58Z

include/qore/ExceptionSink.h

+    DLLLOCAL QoreStackLocation& operator=(QoreStackLocation&&) = delete;
+
+    //! called when pushed on the stack to set the next location
+    DLLLOCAL void setNext(const QoreStackLocation* next) {


this methods assumes that the QoreStackLocation next will be valid after setNext as long as this class instance exists. But since QoreStackLocation is a simple object holding just a pointer we can copy it and we don't have to be careful that the QoreStackLocation next will be there as long as needed.

I don't understand the problem or how to solve it - can you please give me more info?
next is a pointer that must be valid while the current object is valid, because it's the current last stack location. It will only disappear after the current item has been deleted. If there is a better way to do this, please let me know.

who is the owner of the pointer and when it gets deleted? because the API of this method takes a const* - readonly pointer somewhere - there are no guaranties that the pointer will be valid and still the method saves it into a member and accesses it later. So either the QoreStackLocation class is going to be owner of it, then I would do setNext(std::unique_ptr<const QoreStackLocation> next) or is not, and then it must be internally copied.

otherwise someone can do

QoreStackLocation loc1; { QoreStackLocation loc2; loc1.setNext(&loc2); } loc1.next().smth - FAIL

the same for the member right, it's just a pointer without knowing who owns it, I would go with unique or shared_ptr probably...?
Or as I said in a different comment, maybe the StackLocation should hold only info about the location itself and the linked list should be done by std::list - single responsibility for the object.

sejvlond · 2018-12-14T14:28:05Z

include/qore/ExceptionSink.h

+//! Stack location element abstract class
+/** @since %Qore 0.9
+*/
+class QoreStackLocation {


this class looks like an implementation of a linked list, maybe we can use std::list for it and have QoreStackLocation as a template class for it holding only info about the current Program, statement etc, not holding the list directly.

The reason it's implemented like this with a virtual function to return the next stack location is so that language modules (ex jni) can put a single placeholder on the stack, and then if and when an exception is raised, the object can populate all the stack entries in the language stack, the number of which entries will only be known at runtime in case of an exception.

The function returning the next ptr looks like this:

const QoreStackLocation* QoreJniStackLocationHelper::getNext() const { checkInit(); assert(current < size); return (++current < size) ? this : stack_next; }

So basically the "substack" of Java stack entries is created on demand if the stack is actually traversed due to an exception.

sejvlond · 2018-12-14T14:30:55Z

include/qore/intern/qore_thread_intern.h

+class QoreInternalCallStackLocationHelper : public QoreInternalCallStackLocationHelperBase {
+public:
+    DLLLOCAL QoreInternalCallStackLocationHelper(const QoreProgramLocation& loc, const char* call,
+        qore_call_t call_type) : loc(loc), call(call), call_type(call_type) {


call const char * is stored into a member but who is the owner and how it is forced that the pointer is going to be valid as long as this instance lives?

sejvlond · 2018-12-14T14:35:00Z

include/qore/intern/qore_thread_intern.h

+class QoreProgramStackLocationHelper {
+public:
+    DLLLOCAL QoreProgramStackLocationHelper(QoreStackLocation* stack_loc, const AbstractStatement*& current_stmt,
+        QoreProgram*& current_pgm) :


calling the ctor of QoreProgramStackLocationHelper changes the current_stmt and current_pgm? I don't get it why?

it retrieves the current program and statement so they can be returned by the the QoreStackLocation using that object in its stack position - this functionality is used by the debugger.

davidnich and others added 11 commits December 10, 2018 22:19

reffs #3169 initial new stack handling

c32bcd0

refs #3169 progress on unified thread and exception stack trace support

4cb631c

reffs #3169 removed lots of old code related to the old optional call…

59a60ee

… stack implementation; the HAVE_RUNTIME_THREAD_STACK_TRACE constant is always True, removed associated build options and internal logic

Merge remote-tracking branch 'origin/develop' into bugfix/3169_stack_fix

7259e1a

refs #3169 fixed deadlock in thread stack retrieval

9988ea3

refs #3169 new stack trace API updates

7e546d7

refs #3177 fixed binary module installation with autotools

1871a80

refs #3179 std::string COW issue workaround (#3180)

fbf1e2c

reffs #3169 removed lots of old code related to the old optional call…

4f565d3

… stack implementation; the HAVE_RUNTIME_THREAD_STACK_TRACE constant is always True, removed associated build options and internal logic

refs #3169 new stack trace API updates

a7358a3

refs #3169 finalized the API and fixed C++ issues

330a512

davidnich assigned sejvlond Dec 13, 2018

Merge branch 'develop' into bugfix/3169_stack_fix

8f4e2b5

davidnich added 2 commits December 14, 2018 09:11

refs #3169 refactored default exception handling & warning output, ad…

d7a9902

…ded language info to exceptions, updates tests and docs

refs #3169 fixed test

5172049

davidnich added 2 commits December 14, 2018 11:09

refs #3169 more API updates

c0af588

refs #3169 fixed dbg stmt

5f2c10b

refs #3169 show builtin code location properly in call & exception st…

a74d9ac

…acks

sejvlond reviewed Dec 14, 2018

View reviewed changes

davidnich added 3 commits December 14, 2018 16:01

reffs #3169 updated from review comments

bed7b11

reffs #3169 updated from review feedback

5cd8d36

Merge branch 'develop' into bugfix/3169_stack_fix

915e38a

sejvlond merged commit 64d3a5f into develop Dec 14, 2018

sejvlond deleted the bugfix/3169_stack_fix branch December 14, 2018 16:57

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Bugfix/3169 stack fix #3182

Bugfix/3169 stack fix #3182

davidnich commented Dec 13, 2018

davidnich commented Dec 13, 2018

davidnich commented Dec 14, 2018

davidnich commented Dec 14, 2018 •

edited

sejvlond Dec 13, 2018

davidnich Dec 14, 2018

sejvlond Dec 13, 2018

davidnich Dec 14, 2018

sejvlond Dec 13, 2018

davidnich Dec 14, 2018

sejvlond Dec 14, 2018

sejvlond Dec 14, 2018

sejvlond Dec 14, 2018

davidnich Dec 14, 2018

sejvlond Dec 14, 2018

davidnich Dec 14, 2018

sejvlond Dec 14, 2018

davidnich Dec 14, 2018

Bugfix/3169 stack fix #3182

Bugfix/3169 stack fix #3182

Conversation

davidnich commented Dec 13, 2018

davidnich commented Dec 13, 2018

davidnich commented Dec 14, 2018

davidnich commented Dec 14, 2018 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

davidnich commented Dec 14, 2018 •

edited