Firebase Test Lab example that demos Jacoco support

[bootstraponline]

It'd be nice to have an example of running Espresso on Firebase Test Lab with Jacoco code coverage support.

[AndreSand]

If Firebase team adds this feature, I will be happy to tested :)
I tried to integrate this before but no luck. here are some tutorials I followed:
https://medium.com/mobile-testing/generating-code-coverage-from-espresso-tests-running-in-firebase-test-lab-e98edcea7bdf
https://stackoverflow.com/questions/40726208/how-to-get-code-coverage-reports-from-google-firebase-for-android-espresso-tests

[shobhitagarwal1612]

Try out this blog post shared by @bootstraponline
https://proandroiddev.com/unified-code-coverage-for-android-revisited-44789c9b722f
It definitely helped me a lot

[AndreSand]

@shobhitagarwal1612 good tutorial, but @bootstraponline wants to integrate Jacoco codecov reporting with Firebase Test Lab runs.

[bootstraponline]

Jacoco on Firebase Test Lab works today. There's no official sample though. I think it'd help if Google maintained one.

[prolificcoder]

I wrote these posts

https://medium.com/mobile-testing/generating-code-coverage-from-espresso-tests-running-in-firebase-test-lab-e98edcea7bdf
https://stackoverflow.com/questions/40726208/how-to-get-code-coverage-reports-from-google-firebase-for-android-espresso-tests

Did they no work? @bootstraponline how is your setup different?

[JoseAlcerreca]

These samples are for the Android Testing Support Library. Firebase and Jacoco are outside of that scope. We use FTL in android-architecture though.

[bootstraponline]

Did they no work? @bootstraponline how is your setup different?

Yeah, they work. Was hoping for something Google maintained. Sounds like that's the Android architecture repo.

[langsmith]

Anyone figure this out? I'm trying to retrieve Jacoco code coverage reports of test app instrumentation tests run in a Firebase emulator and then passing those reports on to codecov.io

[sblepa]

@langsmith Where you able to find a solution to the issue? I'm struggling with the same thing

[langsmith]

Nope, haven't solved it yet @sblepa 😞 I think I've read every official documentation page, Github thread, Stack Overflow post, and blog post related to some mix of Jacoco, CircleCI, Firebase, Codecov. 😄 Searching for others' code on Github also didn't really turn up anything.

Nothing out there that covers the integration of all four together though 😢

[ToxicBakery]

I was able to get Firebase, Jacoco, and Codecov to play nicely together for merged unit test and instrumentation test coverage. If you run into any issues following what I did, feel free to post them and I'll try to help.

https://github.com/Rajawali/Rajawali

If you read through the circle config you'll find the block for generating the androidTest apk and then passing it to Firebase for execution. After execution, the coverage report is copied back and then a custom gradle task is run for merging the previously run unit test coverage with the instrumentation test coverage. After that, codecov works as expected.

Note this leaves out several important pieces of information such as creating the dummy apk which is still a requirement for library projects presumably because no one at firebase has ever bothered trying to use test lab from a CI environment. The dummy apk, as documented in various SO posts and github issues, needs to have a main activity, permissions to write to the sd card, and must have test coverage enabled in the gradle config. Next, you need to set up a service account in your google api console which I would love to explain here but that's about 156 "simple" steps that really needs an entire blog post to explain. Finally, if you follow any of the articles about unifying code coverage you may find they do not work for you depending on your build setup. In my case build/intermediates/classes is the wrong location leading to an empty coverage report. In my project, the location is in build/intermediates/javac/... which is a simple tweak to the unified code coverage instructions.

Going forward, I'll work on rolling the gcloud calls into gradle so that the build script can lose some weight but for now it is what it is. Additionally, this setup will only work when testing against a single device. The gcloud copy calls can be tweaked to work around that I just haven't gotten to it yet.

To any Googlers reading this, can you please help us out and document this process? It's a bit insane that it's this much work to figure out how to use tools you make us pay for (re Firebase for any appreciable amount of testing). The fact that everyone, myself included, has to piece this all together from a variety of blog posts, SO posts, and github issues would make me feel embarrassed if I was on teams associated to these tools. CI is an obvious use case and we are left entirely to ourselves to tape it together.

[bootstraponline]

https://github.com/TestArmada/flank#android-code-coverage

This is the documentation I created after lots of trial and error. Hope it's useful. 😄

[nikitha2]

I wrote these posts

https://medium.com/mobile-testing/generating-code-coverage-from-espresso-tests-running-in-firebase-test-lab-e98edcea7bdf
https://stackoverflow.com/questions/40726208/how-to-get-code-coverage-reports-from-google-firebase-for-android-espresso-tests

Did they no work? @bootstraponline how is your setup different?

I wrote these posts

https://medium.com/mobile-testing/generating-code-coverage-from-espresso-tests-running-in-firebase-test-lab-e98edcea7bdf
https://stackoverflow.com/questions/40726208/how-to-get-code-coverage-reports-from-google-firebase-for-android-espresso-tests

Did they no work? @bootstraponline how is your setup different?

@prolificcoder
I got he coverage.ec file. However this is a raw file and we cannot open it. Can you help me on how do we get a readable version of the file? I am using firebase testlab to get the coverage report

[prolificcoder]

@nikitha2 you can't open it but I recall that coverage programs can read the binary/raw format.

[Aidan128]

I've written an article at https://proandroiddev.com/android-code-coverage-on-firebase-test-lab-part-1-the-basics-9e1492ec5399 that might be helpful, especially if you're trying to get this working for API 30. The corresponding sample project is at https://github.com/Aidan128/CoverageExample1/

@nikitha2 it also covers how to generate reports from a downloaded .ec file.