BottomSheetDialogFragment not included in screenshots?

[DanielJette]

Describe the bug
Trying to record the baseline screenshots for an app with a BottomSheetDialogFragment opened. The screenshot doesn't include the BottomSheetDialogFragment, only the everything that is behind it

This issue relates to:

• The Kotlin library
• The Gradle plugin

To Reproduce
Steps to reproduce the behavior:

1. App with a BottomSheetDialogFragment, open the dialog
2. Create a test and click on 'Record baseline' on Android Studio
3. Check the taken screenshot and the BottomSheetDialogFragment is not there

Expected behavior
The opened BottomSheetDialogFragment should be included in the screenshot

Screenshots
App with BottomSheetDialogFragment opened (disregard bottom nav menu):

Recorded baseline screenshot:

Desktop (please complete the following information):

• OS: MacOS
• Version 11.5.2

Target Android Device (please complete the following information):

• Device: Pixel 4
• Physical or Virtual: Android Virtual Device (Emulator)
• API Level: 29
• Testify Key: 29-1080x2280@440dp-en_US

---

cc @amanzan ported from Shopify/android-testify#243

[DanielJette]

@amanzan Thank you for the bug report.

Testify isn't currently able to capture the bottom sheet fragment on top of another fragment/activity since its internal capture mechanism is based on the view hierarchy. Basically, the bottom sheet isn't in the same view hierarchy, so it's not getting captured.

Supporting this case would be a nice addition to the next version of Testify and so I've tagged this to the 1.2 milestone.

As a workaround for testing bottom sheet fragments, I recommend that you write your test so that you focus on testing the contents of the fragment in isolation, instead of actually as being rendered in a bottom sheet. You can do this by loading the Fragment directly into a host activity and using Testify to capture that, or you can try using setScreenshotViewProvider to instruct Testify to capture a different view hierarchy.

e.g. https://github.com/Shopify/android-testify/blob/master/Sample/src/androidTest/java/com/shopify/testify/sample/ScreenshotRuleExampleTests.kt#L251-L267

[DanielJette]

Copied from @amanzan Shopify/android-testify#243 (comment)

@DanielJette would it be possible to extend somehow ScreenshotRule, so that there's a new assertSame function which accepts currentBitmap as parameter? This way, I could take a screenshot of the entire screen and pass it

[DanielJette]

Hi @amanzan
Unfortunately there's not currently a way to extend the ScreenshotRule in a meaningful way to allow for customization of the currentBitmap.
However, there are plans to significantly expand the extensibility and customizability of the screenshot rule.
This is tentatively scheduled to be included in the 1.2.0 release.

I have a work-in-progress branch here https://github.com/Shopify/android-testify/compare/227-extensible that demonstrates one way that Testify can allow for custom capture methods.

[DanielJette]

Copied from @amanzan Shopify/android-testify#243 (comment)

Cool, looking forward to it 👌 Thanks!

[DanielJette]

Copied from @jennatauro
Shopify/android-testify#243 (comment)

Hi @DanielJette would this issue also be applicable to DropdownMenu from jetpack compose? I'm trying to write a screenshot test for the "expanded" (opened) version of the dropdown and the menu doesn't appear in the screenshot:

expected	actual

I wasn't sure if this was due to a timing issue with when the screenshot is taken or if it's because the dropdown is rendered in a different view hierarchy.

[DanielJette]

@jennatauro I added a DropdownMenu test to the Sample app and I can confirm that I'm seeing the same error as you are.

Here is the Layout Inspector output:

It appears that the Dropdown is rendered in a different layer than the rest of the composition. I suspect this is likely implemented via a new Window by the compose renderer and this Window is not being captured.

You can find my changes to the Sample here Shopify/android-testify#262

As of yet, I don't know how to resolve this issue, but it might be possible to direct Testify to capture the popup Window. Due to the limitations of Testify, it is not likely possible to capture the entire screen without also capturing the System UI. There is an issue requesting full-screen capture here.

[DanielJette]

Copied from @amanzan
Shopify/android-testify#243 (comment)

Hi @DanielJette, I see there is a 1.2.0-beta1 version available. Would this be possible to do in that version then? If so, would you mind guiding me how to approach it?

[DanielJette]

@amanzan Unfortunately the 1.2.0-beta1 version has not been published and there are no plans to publish this version to the Shopify repository. Future updates will be published under the dev.testify namespace (https://search.maven.org/artifact/dev.testify/plugin).

The full-screen functionality you are looking for will be published in the testify-fullscreen library which you can find here: https://github.com/ndtp/android-testify/tree/main/Ext/Fullscreen

We're still working through a lot of details regarding the move to the new repository, so stay tuned for updates. I would anticipate this library being published in Q3 2022.

[DanielJette]

Testify now supports capturing the entire screen, including Dialogs, Menus and Bottom Sheets, through the user of the FullscreenCapture method.

To capture Dialogs. Menus and Bottom Sheets with Testify:

1. Add the testify-fullscreen extension library to your project
   
2. Use the captureFullscreen() modifier on your ScreenshotRule.
3. Exclude the System UI from the screenshot comparison using excludeSystemUi
4. Set the exactness to a value of 0.95f or lower
5. Record & run your tests as usual

For more information, see the example here.