Unable to step into hello-libs

[bessermt]

I am running Android Studio 2.2 Preview 4 on Linux Mint 17.2 64 bit using a Nexus 5 device.

I am able to debug into

Java_com_example_hellolibs_MainActivity_stringFromJNI(JNIEnv *env, jobject thiz),

but I am unable to debug into

GMATH_EXPORT unsigned gpower(unsigned n)

GPERF_EXPORT uint64_t GetTicks(void)

The only difference I can see is that these are located in a module, but maybe there is some other problem.

How can I debug into these C++ functions?

[ggfan]

which branch( master or master-cmake) you are using? please also check whether you are building the 2 libs in your app ( that is: app/build.gradle). Recently they are changed to use binary directly for libs, so by default it is not built together with app. please let me know the above info.

[bessermt]

I am on branch "master". I don't know how to check whether I am building the 2 libs as you describe. I am doing a full rebuild if that answers the question. It sounds like you are claiming it should work, but maybe I need to do some special building of the modules?

[bessermt]

I looked at how to build modules as standalone, but I'm not sure how to our do that. I did add a #error to the code in each function I'm attempting to debug into and did a Build | Rebuild Project. The #errors did stop the compiler, so I am quite confused by your comment about "by default not built together with app". If not built, then I would expect the #error to not fail in a Rebuild Project. Can you please explain with some more detail?

[ggfan]

yes, master should still be building the libs when app module is being built. master-cmake branch is not building anymore. build or not mainly depends the following in app/build.gradle

tasks.whenTaskAdded { task ->
if (task.name.contains('compile')) {
task.dependsOn ':gmath:distributeLib'
task.dependsOn ':gperf:distributeLib'
}
}

the setting for these 2 libs are treated as external, app does not know where the symbols are actually. I just put them together and let gradle to trigger a build for lib, but app module does NOT depends on the build location of gmath and gperf; app only depends on libs in directory distribution/gmath and distribution/gperf; and those libs are copied from release build:
task(distributeLib, type : Copy) {
// trigger build library
dependsOn assemble // <=== this one cause both debug and release to be built, but...
into '../distribution/gmath/'
from('src/main/jni/gmath.h') {
into 'include/'
}
from('build/outputs/native/release/lib') {
into 'lib/' //<== only the release lib is copied (and released to customers)
}
}

so app module will not be able to debug. If you want to really debug into a lib, it is much easier to build together as app AND make app's jni depends on lib module; that way it builds and knows where the symbols are. All samples in the repo using app-native-glue are built and used that way, and traceable:
Teapot is one of them:
https://github.com/googlesamples/android-ndk/blob/master/Teapot/app/build.gradle

thought I copied the debug libs, I was wrong.

This sample(hello-libs) is to show how to user 3rd party libs, not in-app libs. In lib-lib ones are in teapot/moreTeapot/native-activities .
hope this clarifies things. If you are not very deep into gradle experimental yet, maybe you could check out cmake branch (master-cmake), and it is guaranteed long time support and could only be better and better. Gradle experimental might be stale for a while before taking off big time.

[bessermt]

Some of this makes sense, but I still have a lot of confusion. My goal is to use a 3rd party open source library "as is" but debug into it so I can learn how it works. The library is both Java and C++ NDK.

Is hello-libs a good starting point to learn how to build a 3rd party C++ library and debug into it?

How do I tell Android Studio where to find the source symbols for the library?

[ggfan]

since you already have the source code, that matches teapot's use of app-native-glue, you could directly follow that path. it will be much smoother

in your situation, I would recommend cmake branch, teapot sample. read through the cmakelists.txt you should be fine.

[bessermt]

Thanks for the tip. I'm still concerned this path won't be easy because I don't have much (if any) control over their test project. You can take a look at:

https://github.com/mapbox/mapbox-gl-native/tree/master/platform/android

Note they have a test app named: MapboxGLAndroidSDKTestApp
and they have the library: MapboxGLAndroidSDK

I would need to debug the MapboxGLAndroidSDKTestApp and trace into MapboxGLAndroidSDK's C++ code. Still think your suggestion is a good idea for someone that knows little about Gradle and Android Studio?

[bessermt]

I took a look at the teapot example app. Android Studio detected some problem building and then suggested a fix. I said okay, and the project doesn't load anymore. I'll rollback and say no to the fix when it asks again.

Can you clarify if I'll need to modify the library code or the app code? I'm unclear on what is the C++ library code in the teapot example. It looks more like a C++ android app to me and not a Java app calling a C++ library. I'm new so maybe I'm not reading it correctly.

I think getting the hello-libs to debug into the C++ library is a better proxy for the problem I'm solving. It looks just like the Mapbox project structure, but simpler. Mapbox is a library written in C++ and I can't debug it, exactly the same as hello-libs in Android Studio. . Can you help me figure out how to fix the hello-libs project to work with debugging? If not, how can I change teapot to be a C++ library called from a Java Android app?

Thanks.

[ggfan]

for how to make mapbox debuggable with android studio, it might be easier for you to get in touch with that repo owner.

I could try to get hellp-libs to be debuggable from app. But it is a twist and not the very purpose of this sample.

Here it is:
`
apply plugin: 'com.android.model.application'

// Root of 3rd party lib(s): location could be anywhere on the host system
def lib_distribution_root = '../distribution'
model {

android {
    compileSdkVersion = 23
    buildToolsVersion = '23.0.2'

    defaultConfig {
        applicationId = 'com.example.hellolibs'
        minSdkVersion.apiLevel = 13
        targetSdkVersion.apiLevel = 23
        versionCode = 1
        versionName = '1.0'
    }
    ndk {
        platformVersion = 21
        moduleName = 'hello-libs'
        toolchain = 'clang'
        stl = 'gnustl_static'
        cppFlags.addAll(['-std=c++11',
                         "-I" + file("${lib_distribution_root}/gperf/include"),
                         "-I" + file("${lib_distribution_root}/gmath/include")])
        ldLibs.addAll(['android', 'log'])
        // build a default combined apk including all ABIs.
        // abiFilters.addAll(['x86'])
    }
    sources {
        main {
            jni {
                dependencies {
                    project ':gmath' linkage 'static'
                    // if gperf were *.a, change shared --> static
                    project ':gperf' linkage 'static'
                }
            }
        }
    }
    buildTypes {
        release {
            minifyEnabled false
            proguardFiles.add(file('proguard-android.txt'))
        }
    }
}

}

dependencies {
println rootProject.getName()
compile fileTree(dir: 'libs', include: ['*.jar'])
compile 'com.android.support:appcompat-v7:23.3.0'
}
`

replace your app/build.gradle, with the above content; then resync, clean project, and re-build app project
set a breakpoint to gmath / gperf, or trace code into it from hello-libs.cpp.

I used ndk-r12, android studio 2.2 preview 4 ( download from canary ).

you could see that it is changed to build static libs, and dependency is "project", not lib anymore. No change on lib source code or gradle scripts; try it out.

[ggfan]

Oops, I clicked the wrong button and closed it

[bessermt]

Thanks @ggfan,

I have contacted the Mapbox team many times over the last year to find someone that could help me. In the end they admitted no one knew how to debug their code in Android. They debug on other platforms and hope it works on Android. When I saw hello-libs, this looked like the simplest example I could find. An added bonus was it failed to debug exactly the same as Mapbox does, so I figured if I could figure out how to change hello-libs to allow debugging, I could do the same to Mapbox's build. I only get a few moments at night to look into this stuff, so I'll get back with questions once I get a good look at what you have posted.

Thanks for all the help. Stay tuned.

[ggfan]

I had super shallow scan of their code, felt: they build lib into AAR format, and your application will link with aar. the jni code is inside aar, so your app may not directly calling jni code inside sdk; app calls sdk's java interface, and sdk's java interface call into its own jni interface; jni interface functions are injected into java class at run time inside jni_onLoad() ( I might be wrong ). it is going to be a interesting route :-)

[bessermt]

I tried you suggested build.gradle for the app and it lets me debug into the C++. So now all I need is to figure out what you did and why it works so I can attempt to apply it to the Mapbox Android Studio project. I'll likely have questions as I go. Thanks for the help.

[ggfan]

sure. I might be slow for the next 3 weeks ( taking a break ), will try to keep up.

[bessermt]

No worries @ggfan, any help you can provide is appreciated. I'll leave a trail here (unless you prefer elsewhere) of questions.

First question is:

What is the difference between these 2?:
apply plugin: 'com.android.model.native'
[comment] this builds a shared native lib, .so file , it needs to be tucked inside final apk
apply plugin: 'com.android.model.library'
[comment] this builds a java lib (.aar), it is very java oriented, and it is meant to be static. So it DOES generate a static lib for your native code, *.a, in your model.library module[grep it you will be able to find it after build your apk]. With this, I see there are 2 ways for app to use the generated static lib:

• in-app build: force your app to build model.library in the same proj environment ( like Viusal Studio's workspace). this way your app knows where the generated static lib is so saving some tedious gradle scripts to configure libs. you could see this usage inside native-activity/teapot/moreTeapot [any sample using native glue ] with dependency clause in app's:
  dependencies { project ':nativeactivity' linkage 'static' }
  your header files will be exported, in LIBRARY module, with
  exportedHeaders { srcDir "${ndkDir}/sources/android/native_app_glue" }
  No need for to do anything on app side for header files anymore.
• Out of app build:
  build is same for model.library; but since it is out of app build, all app has is a lib [ with different ABIs], and one or more header files. some extra work need to be done and the sample is hello-libs where app needs to be told

1. where the lib is for each ABI
2. where the header file is
3. config for debug symbols etc [I would try to avoid this one: I need learn it] if you like to trace into lib code.
   From here it is generic pros and cons regarding static lib and shared libs
   [done my editing]

I seem to get further with the library plugin over native, but by that I simply mean I'm getting an error further down the build.gradle during the sync assuming it is parsed and errors reported from top to bottom. For now I'll do my best to follow http://tools.android.com/tech-docs/new-build-system/gradle-experimental and use com.android.model.library.

[bessermt]

I've moved forward making changes. I've had to take guesses.

For example, there was an error for android.libraryVariants.all which appeared to be solved by adding the "model" surrounding it.

model{
android.libraryVariants.all { variant ->
def name = variant.buildType.name
def checkstyle = project.tasks.create "checkstyle${name.capitalize()}", Checkstyle
checkstyle.dependsOn variant.javaCompile
checkstyle.source variant.javaCompile.source
checkstyle.classpath = project.fileTree(variant.javaCompile.destinationDir)
checkstyle.exclude('/BuildConfig.java')
checkstyle.exclude('/R.java')
checkstyle.exclude('/com/almeros/android/multitouch/')
project.tasks.getByName("check").dependsOn checkstyle
}
}

The documentation "http://tools.android.com/tech-docs/new-build-system/gradle-experimental" isn't clear for this use case, but does state: "The DSL for modifying variants and their tasks is very, very limited right now." so I'm not confident this was the right choice.

After some more changes, the following appeared:

./mapbox-gl-native/platform/android/MapboxGLAndroidSDK/build.gradle
Error:(252, 0) Could not find property 'android' on task ':MapboxGLAndroidSDK:androidJavadocs'.
Open File

The build.gradle file starting on line 251 is:

task androidJavadocs(type: Javadoc) {
    source = android.sourceSets.main.java.sourceFiles
    classpath = files(android.bootClasspath)
}

I'm not sure the correct way to fix this. I could guess, but the more guessing I do, the less confidence I have. Do you know the correct way to fix this error?

[bessermt]

Take a look at these instructions:

https://github.com/mapbox/mapbox-gl-native/blob/master/platform/android/DISTRIBUTE.md

This looks like the C++ code is built using a makefile. The C++ source doesn't appear to be listed in the Android Studio projects. Maybe I should avoid changing the build.gradle files entirely and figure out how to tell Android Studio how to find the source files and symbols?

Is this task simply too hard for anyone to accomplish. The Mapbox people don't know how to do it and if they can't, maybe I'm just wasting my time?

[bessermt]

I left a question at:

http://stackoverflow.com/questions/38296840/android-studio-gradle-project-sync-failed-cause-extensibledynamicobject

Gradle project sync failed: Error:Cause: org.gradle.api.internal.ExtensibleDynamicObject

The files are also posted there. I have no idea on how to get past the error.

[bessermt]

If I add .apiLevel to the minSdkVersion and targetSdkVersion in the library, AND put sources to sourceSets {sources.main.res.srcDirs += 'src/main/res-public'} I get a different error:

Error:(43, 1) A problem occurred configuring project ':MapboxGLAndroidSDK'.

Exception thrown while executing model rule: android { ... } @ MapboxGLAndroidSDK/build.gradle line 33, column 5
Attempt to read a write only view of model of type 'org.gradle.model.ModelMap<org.gradle.language.base.FunctionalSourceSet>' given to rule 'android { ... } @ MapboxGLAndroidSDK/build.gradle line 33, column 5'

Not sure this is a better or worse error.

[bessermt]

I applied all the changes I could think to apply but I'm still getting errors and have no idea how to fix them.

Here's the files I've changed:

https://github.com/bessermt/mapbox-gl-native/blob/master/platform/android/build.gradle

https://github.com/bessermt/mapbox-gl-native/blob/master/platform/android/gradle/wrapper/gradle-wrapper.properties

https://github.com/bessermt/mapbox-gl-native/blob/master/platform/android/MapboxGLAndroidSDK/build.gradle

https://github.com/bessermt/mapbox-gl-native/blob/master/platform/android/MapboxGLAndroidSDKTestApp/build.gradle

Using git diff -w to ignore whitespace will be helpful in seeing the differences since the model {} causes it too appear as if there are far more changes that actually exist.

If you can help me get past this I would very much appreciate the help.

[rschiu]

Hi bessermt,
I think you can solve the "Attempt to read a write only view of model of type" error by replacing
sources.main.res.srcDirs += 'src/main/res-public'
with
sources.main.res.srcDirs.add('src/main/res-public')

[bessermt]

Hi @rschiu, thanks for the suggestion. I pushed the changes, but it didn't fix the problem. Feel free to take a look at the changes I posted. I expect @ggfan to return from vacation next week. I hope he'll also have a suggestion.

[bessermt]

Hi @ggfan,

My notes say you should be back from vacation. Can you take a look at:

https://github.com/bessermt/mapbox-gl-native

Thanks for all your help so far.

[ggfan]

yes, will get to it with help from rschiu, I will get to it(will be slow since I am in a conference this week ).
@bessermt most of your usage is not beyond my knowledge of it. for that specific error, if I do:
sources { main { res { source { srcDir 'src/main/res-public' } } } }
, it does not complain anymore, but other complaints coming out like
No signature of method: com.android.build.gradle.managed.AndroidConfig.lintOptions()
maybe you could try out the above, see if you get to the next error?

[bessermt]

Hi @ggfan, I used your suggestion and it appears to get past the error and as you indicated, creates a new error:

Error:(52, 1) A problem occurred configuring project ':MapboxGLAndroidSDK'.

Exception thrown while executing model rule: android { ... } @ MapboxGLAndroidSDK/build.gradle line 33, column 5
No signature of method: com.android.build.gradle.managed.AndroidConfig.compileOptions() is applicable for argument types: (build_ej6bt6za0oplgezep6o1v50v7$_run_closure5$_closure18$_closure22) values: [build_ej6bt6za0oplgezep6o1v50v7$_run_closure5$_closure18$_closure22@24458ff2]

If I compile, I get a better indication of the location of the error, which is here:

compileOptions {
sourceCompatibility JavaVersion.VERSION_1_7
targetCompatibility JavaVersion.VERSION_1_7
}

I will take a look at the error in more detail this weekend, but I have no experience in this area, so any help is still appreciated.

[ggfan]

Oh, you never need those anymore: those are java7 for old android studio, new one could take java8, just delete them. I should have said that.

got a question not related to this: is mapbox better than google maps?

also I like to recommend your to CMake, rather than gradle-experimental, may we drink tea or coffee for this to get you on CMake? gradle-experimental is on a risky path my personal feel

[graph]

Based on my current experience. That will build in Debug, but put it in the Release folder. On Monday when I'm back at work I'll try it and see which folder it puts results in. When building from Android Studio it creates 2 folders in .externalNativeBuild a Debug & release. Debug Folder is empty. Release folder is filled and fully compiled. What's interesting is when I do "Build APK" from the menu it fully compiles the "Debug" folder as expected, When I do "Run" or "Run & Debug" it compiles Release only.

I'll try -DCMAKE_BUILD_TYPE=Debug and see if it makes a difference to debugging and report back next week.

[graph]

Just tried it -DCMAKE_BUILD_TYPE=Debug does not help in debugging a library project. And it still builds in the release folder.

[graph]

A hacky solution is in the main project do this:

        // default config
        externalNativeBuild {
            cmake {
                abiFilters'armeabi-v7a'
                arguments "-DANDROID_PLATFORM=android-${platformVersion}",
                        '-DANDROID_TOOLCHAIN=clang', '-DANDROID_STL=gnustl_shared'
            }
        }
    // in android {}
    externalNativeBuild {
        cmake {
            path "../libModule/src/main/cpp/CMakeLists.txt"
        }
    }

and I can debug just fine. So far it's fine. This has the disadvantage of compiling everything twice.

[ggfan]

thank you for reporting the issue, I will replicate the issue later in the week and go from there

[graph]

Ok thanks I added this bug report as it's NDK bug not a google samples bug.

[ggfan]

added back the link to the equivalent studio bug:
https://code.google.com/p/android/issues/detail?id=222276

this coming back as: add a sample to showcase the usage

[ggfan]

@bessermt : your cooked hello-jni ( one app, one lib ) project could be debugged with the following modifications (doc is at http://tools.android.com/tech-docs/new-build-system/user-guide#TOC-Product-flavors):

"final" version of the solution:

1. in app's build.gradle, change dependency to
   compile project(path: ':layerlib', configuration: 'debug')
   so debug lib build will be used
2. on lib's build.gradle, enable debug version lib to be built and copied (published) to apk
   android.publishNonDefault true

added to vulkan validation's layer build:
https://github.com/googlesamples/android-vulkan-tutorials/tree/master/tutorial03_traceable_layers

works on my system, please check it out. thank you for chasing after it!

[ggfan]

@graph, @bessermt like to let go of this one, may you kindly check it out? thanks

[graph]

Hi I wanted to check out the samples fully before replying but it looks like I won't have time for a while. The core problem of the issue is in the build system which you already linked to. If the workarounds you have added work in the samples and you have tested them out then I think it's ok to close this issue. I briefly scanned the hello-libs and don't see how to apply the workaround right away to my project. But it was only a very brief scan I probably missed it. Perhaps add better comments to so it's clear. I look forward for the core of the debugging issue to be fixed.

Thank you.

[graph]

Reporting back in. Looking at the Vulkan sample, and doing a quick test. It was very easy to apply the workaround and get debugging to work.

Thank you

[ggfan]

thank you for your confirmation! please help to close this issue if it works for you ( this one is important and useful one, like to make sure you get it working) thanks

[bessermt]

I'm not sure if this means this issue was solved. I had given up and started learning CMake to see if I can build the Mapbox code using CMake. I've made very little progress so far, so should go back to using the Gradle Experimental version?

[graph]

@bessermt The core of the issue has not been fixed, only workarounds. CMake just been integrated into android so it's wise to expect bugs. Go with back to experimental until bugs are resolved. Or you can help report bugs.

@ggfan What would you like me to confirm to help close this issue? It's not a googlesamples issue it's one related with the build tools. I think keep it open until Release version of a native library from a dependent project gets packaged even for a debug build variant is fixed. Or you can close it too, I don't know. If closed I could see people resubmitting this bug until the dependant issue is fixed.

[ggfan]

Since there is already a workaround for this, the priority for this into the real fix into gradle build ( debug version packing the debug lib ) may be relatively low. Also this is more or less debugging required feature: after development phase and releasing product, it is not needed.

I would still recommend CMake for the case that starting a new project and experimental is also new to you: the material for cmake is much much more to pick it up, and it does provide a certain level of familiarity to C/C++ developers. When you have lot of things to take care of ( C/C++, shader languages/spir-v, Java, gradle etc), remove extra one might be helpful. Sure we would still be here to help; in the end, the build system is small part of whole project, pushing it to a small corner of your minds and keeping it there so you could spend big time to the core of your project, that might be helpful overall for your project. That would be mine motivation to switch to cmake for new projects as you mentioned in the other thread: C/C++ developers uses more cmake and are more familiar to cmake, Android Studio's cmake is to serve the industry, and your projects dictate the industry ( not the other way around -- you will not quote me for this sentence here :-))

[graph]

Since there is already a workaround for this, the priority for this into the real fix into gradle build ( debug version packing the debug lib ) may be relatively low. Also this is more or less debugging required feature: after development phase and releasing product, it is not needed.

@ggfan With the work around I have to change the gradle file every time I want to switch between debug/release. This is a critical bug and should be fixed properly. It adds unintuitive training that needs to be done which will slow down migration to cmake. Please keep this at highest priority and urgently fix it.

[ggfan]

that is true: whenever want to debug it, have to switch it.

[edkimmel]

If I'm understanding where this issue has gotten to so far:
Original problem: only C++ code in the application module is debuggable.
Potential solutions:

• Build the native code as part of the application's build.gradle
• Force a debug compilation of the library module (This was not working for me with gradle 2.2)

I have to agree that neither solution above is very nice. An application building its debug configuration should build debug libraries (if they are exposed) and should be able to step through the code.

Almost all libraries are published with a prebuilt shared lib and Java bindings for it. I don't see any downside to libraries published with their full source code being debuggable. In a release build, the library will be stripped, and the signed keystore will prevent debugging.

[ggfan]

thanks for the summary! Eventual solution could be something as:

1. when building for release, release libs will be packed
2. when building for debug, debug version is packed

Android Studio IDE should handle this because it knows which build it is; it should eventually inside Android Studio

[bessermt]

It appears the best option for now is as @edkimmel says:"Build the native code as part of the application's build.gradle". I'm not sure how to do this. The source for the library I want is complicated and contains code Android as well as other platforms. You can see how complicated at: https://github.com/mapbox/mapbox-gl-native. I'm still learning the layout, but I won't be allowed to restructure the code because it's not mine to reorganize. I have to include it as-is.

If you can point me how to do this, that would be appreciated.

[ggfan]

it is complex package, with mandatory to create account, significant amount of efforts needed to understand current build. It would be more feasible to discuss your specific questions after you started working on it: the libs need to be converted to either Android Studio's cmake or ndk-build, and build together with your app, that way the debugging symbols will be configured automatically.

[ggfan]

@graph: modify the above workaround on app side with:
debugCompile project(path: ':layerlib', configuration: 'debug')
releaseCompile project(path: ':layerlib', configuration: 'release')
(lib side needs stay the same )
this way you may not need to manually switch between release / debug version anymore. could you confirm it? thanks

[ggfan]

added wiki for this one, closing it for now. we will need address mapbox in different thread, do let me know for any concerns:
https://github.com/googlesamples/android-ndk/wiki
thanks

[graph]

Ok, I haven't had time to test the post above. Just letting you know I haven't forgotten about it.

[graph]

@ggfan is there a way to send private messages regarding android issues? Thank you.

[ggfan]

I need to find out what forums / channels and let you know. if it is urgent, you could use my email, I will ask team mates for it. Internally we have various groups for specific tops ( I am also in my discovering phase ). Again if it is your company product related, probably find the contact person and send info to that person might be the best ( I might be wrong for this)

[graph]

Yeah let me know where to post it. It's a documentation bug report. Nothing urgent, you can edit and remove your email from that post. Your teams might already know. It's not a very heated topic but I have seen some negative threads around it. Thank you.

[ggfan]

@graph the channel is: https://code.google.com/p/android/issues/entry?template=Developer%20Documentation