Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Do not run tests with LargeTest annotation in CI #333

Closed
luizgrp opened this issue Jul 6, 2022 · 2 comments · Fixed by #402 or #417
Closed

Do not run tests with LargeTest annotation in CI #333

luizgrp opened this issue Jul 6, 2022 · 2 comments · Fixed by #402 or #417
Assignees

Comments

@luizgrp
Copy link
Member

luizgrp commented Jul 6, 2022

WHAT

Do not run tests with LargeTest annotation in CI.

WHY

It was the intention of #304

However, it looks like they are still running as this recent PR failed with:

com.google.android.horologist.mediasample.playback.PlaybackNotificationTest > testCausesNotification[test(AVD) - 9] FAILED 
	expected to be true
	at com.google.android.horologist.mediasample.playback.PlaybackNotificationTest$testCausesNotification$1$1.invokeSuspend(PlaybackNotificationTest.kt:63)
Tests on test(AVD) - 9 failed: There was 1 failure(s).

And the test that failed is annotated with LargeTest.

luizgrp added a commit to luizgrp/horologist that referenced this issue Jul 19, 2022
@luizgrp luizgrp self-assigned this Jul 19, 2022
luizgrp added a commit to luizgrp/horologist that referenced this issue Jul 19, 2022
@luizgrp
Copy link
Member Author

luizgrp commented Jul 19, 2022

Added a test annotated with LargeTest in #401 and confirmed that they are running in CI.
After cherry-picking commit from #402 it got green, so confirmed that the solution works and large tests will not run in CI after that commit.

@luizgrp
Copy link
Member Author

luizgrp commented Jul 20, 2022

Reopening as this build failed for a test annotated with LargeTest:

com.google.android.horologist.media.ui.screens.PlayerScreenTest > givenCustomControlButtons_thenCustomIsDisplayed[test(AVD) - 11] FAILED 
	java.lang.AssertionError: Activity never becomes requested state "[DESTROYED]" (last lifecycle transition = "RESUMED")
	at androidx.test.core.app.ActivityScenario.waitForActivityToBecomeAnyOf(ActivityScenario.java:338)

Can see in the logs that the IGNORE_LARGE_TESTS is true but SIZE_OPTS is not being set, so the resulting gradlew command does not contain the size filters:

IGNORE_LARGE_TESTS=true

+ SIZE_OPTS=
+ [[ -z true ]]
+ [[ ! -z true ]]

+ ./gradlew --scan --continue --no-configuration-cache --stacktrace --no-parallel runAffectedAndroidTests -Paffected_module_detector.enable -Pandroid.testInstrumentationRunnerArguments.numShards=2 -Pandroid.testInstrumentationRunnerArguments.shardIndex=0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
1 participant