Parameterized Instrumented tests mangle test output

[ktkopone]

E.g. Android Studio displays them as:

Note that the parameterized test method and its cases seem to have been put under new mangled class headings ("String)") instead of all grouped under the test class (StupidConnectedTest). Additionally, the parameter cases ("", "abc", "123") have been mangled with the method name appended.

Checking the console output reveals the parameterized test has been similarly
mangled there too, so this is not or not solely an IDE issue:

gradle_instrumentation_param_tests_output.txt

Sample instrumentation test file (src/androidTest/java/stupidSampleGradleProject/):

StupidConnectedTest.java.txt

Sample build.gradle:
build.gradle.txt

[ktkopone]

Also, amusingly the parameterized test method is erroneously getting its own row in addition to the parameterized cases, and this appears to be receiving a negative test duration:

[mannodermaus]

This will be addressed in the upcoming 1.2.0 release of the instrumentation test libraries. In the meantime, could you test the latest 1.2.0-SNAPSHOT to see if things are improved?

Here's the output of the sample project's instrumentation tests after the applied change. Much better!

[ktkopone]

It is indeed much more readable now, but it's still not quite at parity with unit tests (stop me if you made it this way as a workaround of some android limitation) - the structure instrumented param tests are reported in still appears to be flat where it's appropriately nested for unit tests. E.g. as visible in your screenshot the param cases are all listed side by side with regular test methods, whereas unit tests currently format them like this:

Looks like this is due to the unit reports having the extra 'method name' vs 'test' field, where the instrumented report only has the one 'test' field for categorizing. Not sure if this is even something you can change at your end.

However the current flat layout is definitely clean enough that we no longer need to painfully avoid param tests in our instrumentation cases, so thanks!

[mannodermaus]

The lack of intermediate nesting is indeed attributable to a limitation with how the Android instrumentation seems to report parameterized tests. Keeping in mind that instrumentation test support is banking on the existing infrastructure, which has to map the new stuff to JUnit 4, it's an unfortunate consequence that nesting can't be done arbitrarily - there's only class nodes and their direct children, the test methods.

As an alternative to the current approach, I considered making each dynamic test group its own entry on the class level (i.e. there would be JavaInstrumentationTests & JavaInstrumentationTests.testParameterized() on the same level in the hierarchy). Ultimately, I decided against that, though.

[mannodermaus]

Released in 1.2.0!