Native Java Agent for "Async Call Stacks"

[benjchristensen]

Speaking with Tal Weiss at QConSF (@takipi https://www.takipi.com) it sounds like there may be some efficient approaches to capturing the callstack as longs (using C in native code) at each async invocation point in an Observable sequence so that in event of onError we could rebuild the stacktrace (paying the cost only then) and end up with more useful error information that normal stack traces offer in async and composed systems like Rx.

This could enhance what we can do with https://github.com/ReactiveX/RxJavaDebug but also allow an "RxException" that contains the graph and callstack at each method invocation point along the way. It won't necessarily be a simple stack since an Observable can split and merge, but we should be able to either capture the full graph or the path to the error.

[DylanSale]

This would be amazing. Would it work on Android though? I assume not, due
to it not being the JVM.

Perhaps a native extension could also be developed on android.
On 6 Nov 2014 08:02, "Ben Christensen" notifications@github.com wrote:

Speaking with Tal Weiss at QConSF (@takipi https://github.com/takipi
https://www.takipi.com) it sounds like there may be some efficient
approaches to capturing the callstack as longs (using C in native code) at
each async invocation point in an Observable sequence so that in event of
onError we could rebuild the stacktrace (paying the cost only then) and
end up with more useful error information that normal stack traces offer in
async and composed systems like Rx.

This could enhance what we can do with
https://github.com/ReactiveX/RxJavaDebug but also allow an "RxException"
that contains the graph and callstack at each method invocation point along
the way. It won't necessarily be a simple stack since an Observable can
split and merge, but we should be able to either capture the full graph or
the path to the error.

—
Reply to this email directly or view it on GitHub
#1823.

[benjchristensen]

I doubt this particular option would be available on Android. I don't know enough about Dalvik at this time to understand options. If Dalvik exposes hooks or APIs that allow this type of thing we could definitely take advantage of it.

[headinthebox]

What is the trick to do that?

[benjchristensen]

More on this topic can be found here: http://www.takipiblog.com/double-agent-java-vs-native-agents/

It goes beyond my skill set so can only conceptually discuss it. Apparently the concept is:

• at each call site capture a pointer to memory in C with the callstack information
• store this pointer in the Java object (Observable operator instance?)
• if debug occurs (manually or via onError) read through the pointers and capture the callstacks for each operator
• build a "callstack graph" for the Observable chain (expensive, memory allocation, getting code lines etc)
• if more than x seconds passes free up the memory of those callstacks in C that are being referenced by pointers

Apparently this can be done very efficiently in C++ deep in the JVM with a native agent. In Java however it is quite expensive to call Thread.getStackTrace() as it done many native calls and allocates a large array of strings, so we definitely don't want to be doing that on every operator invocation in the happy path.

[akarnokd]

PR is welcome, although it might be better to have such tool in a separate project - not sure how we'd build and release a separate agent jar with the current infrastructure.

[iNoles]

Android does have NDK (C and C++) with JNI access too.

[akarnokd]

Closing as very old and being a complicated topic. Other async solutions also struggle with this and I haven't seen any solution for the either despite the support from their major IDE vendor.