Support sharding with Gradle managed devices

[sergio-sastre]

Describe the bug
Recording screenshots with andoid-testify and gradle managed devices works fine, unless we use sharding, e.g. using -Pandroid.experimental.androidTest.numManagedDeviceShards=2

In that case, only the sceenshots of one of the emulators is pulled out, for 2 shards it would be half of the expected screenshots

This issue relates to:

• The Kotlin library
• The Gradle plugin
• The IntelliJ Platform plugin
• The sample code
• The documentation

To Reproduce

1. configure android-testify to use gradle managed devices, for instance

managedDevices {
   devices {
      pixel3api30(ManagedVirtualDevice) {
         device = "Pixel 3"
         apiLevel = 30
         systemImageSource = "aosp"
      }
   }
}

2. execute it with more than 1 shard, e.g.
   ./gradlew pixel3api30DebugAndroidTest -Pandroid.experimental.androidTest.numManagedDeviceShards=2

3. Only the screenshots of one "shard" can be found in the corresponding gradle managed device folder

Expected behavior
The screenshots of all "shards" are in the corresponding gralde managed device folder

Target Android Device (please complete the following information):

• Device: Any
• Physical or Virtual: Gradle Managed device
• API Level: Any

Additional context
It works if numManagedDeviceShards=1

[sergio-sastre]

Describe the bug Recording screenshots with andoid-testify and gradle managed devices works fine, unless we use sharding, e.g. using -Pandroid.experimental.androidTest.numManagedDeviceShards=2

In that case, only the sceenshots of one of the emulators is pulled out, for 2 shards it would be half of the expected screenshots

This issue relates to:

• The Kotlin library
• The Gradle plugin
• The IntelliJ Platform plugin
• The sample code
• The documentation

To Reproduce

1. configure android-testify to use gradle managed devices, for instance

managedDevices {
   devices {
      pixel3api30(ManagedVirtualDevice) {
         device = "Pixel 3"
         apiLevel = 30
         systemImageSource = "aosp"
      }
   }
}

2. execute it with more than 1 shard, e.g.
   ./gradlew pixel3api30DebugAndroidTest -Pandroid.experimental.androidTest.numManagedDeviceShards=2
3. Only the screenshots of one "shard" can be found in the corresponding gradle managed device folder

Expected behavior The screenshots of all "shards" are in the corresponding gralde managed device folder

Target Android Device (please complete the following information):

• Device: Any
• Physical or Virtual: Gradle Managed device
• API Level: Any

Additional context It works if numManagedDeviceShards=1

I'll tag @olegosipenko, since he might have some suggestion for this ;)

[DanielJette]

Ha! I didn't even know they supported shards. Sounds like we need to read and respect the numManagedDeviceShards flag -- I imagine it's a matter of iterating over multiple directories instead of just one.

[sergio-sastre]

@DanielJette
Could you tell me which classes should be adjusted to support this? I could try to fix it and create the corresponding PR

[DanielJette]

Hi @sergio-sastre
I haven't had a chance to look into this, I'm not entirely sure what might be needed here.

These two PRs added the bulk of the GMD support so they might offer some good places to start looking:

• 07 gradle managed device #172
• 175: Gradle Managed Device outputs to full Testify path #177

#177 is the one where I added the support for properly saving the test storage files in a way that was compatible with Testify. So I'd focus on those changes.

The key file is Library/src/main/java/dev/testify/output/TestStorageDestination.kt

If I had to guess, the issue is with the value returned by getTestStoragePath. I would guess that perhaps it's writing files for each shard to a unique directory that's not being considered by Testify.

TestStorage has this getOutputProperties method ... maybe that has information about where the files are stored when sharded?

If you share what you find, I can try to help out with this.

[sergio-sastre]

Hi @sergio-sastre I haven't had a chance to look into this, I'm not entirely sure what might be needed here.

These two PRs added the bulk of the GMD support so they might offer some good places to start looking:

• 07 gradle managed device #172
• 175: Gradle Managed Device outputs to full Testify path #177

#177 is the one where I added the support for properly saving the test storage files in a way that was compatible with Testify. So I'd focus on those changes.

The key file is Library/src/main/java/dev/testify/output/TestStorageDestination.kt

If I had to guess, the issue is with the value returned by getTestStoragePath. I would guess that perhaps it's writing files for each shard to a unique directory that's not being considered by Testify.

TestStorage has this getOutputProperties method ... maybe that has information about where the files are stored when sharded?

If you share what you find, I can try to help out with this.

After further investigating, it looks like the problem is in Gradle Managed Devices itself. I've already reported it:
android/testing-samples#502