Migrates android semanitcs integration to integration test #127128

chunhtai · 2023-05-18T18:40:13Z

I think the flake is due to setclipboard or semantics update race condition. I migrated the test to use integration test package which relies less on timing

fixes #124636

Pre-launch Checklist

I read the Contributor Guide and followed the process outlined there for submitting PRs.
I read the Tree Hygiene wiki page, which explains my responsibilities.
I read and followed the Flutter Style Guide, including Features we expect every widget to implement.
I signed the CLA.
I listed at least one issue that this PR fixes in the description above.
I updated/added relevant documentation (doc comments with ///).
I added new tests to check the change I am making, or this PR is test-exempt.
All existing and new tests are passing.

If you need help, consider asking for advice on the #hackers-new channel on Discord.

chunhtai · 2023-05-18T18:42:12Z

dev/integration_tests/android_semantics_testing/integration_test/main_test.dart

@@ -0,0 +1,643 @@
+// Copyright 2014 The Flutter Authors. All rights reserved.


This is essentially the same test as test_drive/main_test.dart

chunhtai · 2023-05-18T18:43:18Z

dev/integration_tests/android_semantics_testing/android/project-app.lockfile

@@ -18,11 +18,16 @@ androidx.lifecycle:lifecycle-runtime:2.2.0=debugAndroidTestCompileClasspath,debu
 androidx.lifecycle:lifecycle-viewmodel:2.1.0=debugAndroidTestCompileClasspath,debugCompileClasspath,debugRuntimeClasspath,debugUnitTestCompileClasspath,debugUnitTestRuntimeClasspath,profileCompileClasspath,profileRuntimeClasspath,profileUnitTestCompileClasspath,profileUnitTestRuntimeClasspath,releaseCompileClasspath,releaseRuntimeClasspath,releaseUnitTestCompileClasspath,releaseUnitTestRuntimeClasspath
 androidx.loader:loader:1.0.0=debugAndroidTestCompileClasspath,debugCompileClasspath,debugRuntimeClasspath,debugUnitTestCompileClasspath,debugUnitTestRuntimeClasspath,profileCompileClasspath,profileRuntimeClasspath,profileUnitTestCompileClasspath,profileUnitTestRuntimeClasspath,releaseCompileClasspath,releaseRuntimeClasspath,releaseUnitTestCompileClasspath,releaseUnitTestRuntimeClasspath
 androidx.savedstate:savedstate:1.0.0=debugAndroidTestCompileClasspath,debugCompileClasspath,debugRuntimeClasspath,debugUnitTestCompileClasspath,debugUnitTestRuntimeClasspath,profileCompileClasspath,profileRuntimeClasspath,profileUnitTestCompileClasspath,profileUnitTestRuntimeClasspath,releaseCompileClasspath,releaseRuntimeClasspath,releaseUnitTestCompileClasspath,releaseUnitTestRuntimeClasspath
+androidx.test.espresso:espresso-core:3.2.0=debugAndroidTestCompileClasspath,debugCompileClasspath,debugRuntimeClasspath,debugUnitTestCompileClasspath,debugUnitTestRuntimeClasspath,profileCompileClasspath,profileRuntimeClasspath,profileUnitTestCompileClasspath,profileUnitTestRuntimeClasspath,releaseCompileClasspath,releaseRuntimeClasspath,releaseUnitTestCompileClasspath,releaseUnitTestRuntimeClasspath


not very familiar with these lock file but the test won't run until i update the lock file.

chunhtai · 2023-05-18T18:45:04Z

...tics_testing/android/app/src/main/java/com/yourcompany/platforminteraction/MainActivity.java

@@ -123,11 +122,60 @@ private Map<String, Object> convertSemantics(AccessibilityNodeInfo node, int id)
        if (actionList.size() > 0) {
            ArrayList<Integer> actions = new ArrayList<>();
            for (AccessibilityNodeInfo.AccessibilityAction action : actionList) {
-                actions.add(action.getId());
+                if (kIdByAction.containsKey(action)) {


This make it more stable, in case Android may update the AccessibilityNodeInfo.AccessibilityAction enum.

This is was causing some confusion that the test previously think the paste action to be a copy action. I think Android has probably updated the enum sometimes in the past.

...tics_testing/android/app/src/main/java/com/yourcompany/platforminteraction/MainActivity.java

dev/integration_tests/android_semantics_testing/test_driver/integration_test.dart

dev/integration_tests/android_semantics_testing/lib/main.dart

dev/integration_tests/android_semantics_testing/test_driver/main_test.dart

dev/integration_tests/android_semantics_testing/test_driver/integration_test.dart

chunhtai · 2023-05-19T18:04:49Z

hmm, for some reason the flutter test would get permission denied when sending adb request, but flutter drive would not.

@christopherfujino Do you know why we may have different permission for running flutter test vs flutter drive?

This is the log

E/flutter (30629): [ERROR:flutter/runtime/dart_vm_initializer.cc(41)] Unhandled Exception: ProcessException: Permission denied
E/flutter (30629):   Command: adb shell settings put secure enabled_accessibility_services com.google.android.marvin.talkback/com.google.android.marvin.talkback.TalkBackService
E/flutter (30629): #0      _ProcessImpl._start (dart:io-patch/process_patch.dart:401:33)
E/flutter (30629): #1      Process.start (dart:io-patch/process_patch.dart:38:20)
E/flutter (30629): #2      main (file:///Users/chtai/git/flutter/dev/integration_tests/android_semantics_testing/integration_test/main_test.dart:112:37)
E/flutter (30629): #3      _runMain.<anonymous closure> (dart:ui/hooks.dart:159:23)
E/flutter (30629): #4      _delayEntrypointInvocation.<anonymous closure> (dart:isolate-patch/isolate_patch.dart:296:19)
E/flutter (30629): #5      _RawReceivePort._handleMessage (dart:isolate-patch/isolate_patch.dart:189:12)

chunhtai · 2023-05-19T18:24:25Z

I moved the adb comment to enable talkback to devicelab test runner so I can workaround the issue that i can't call adb in flutter test

chunhtai · 2023-05-23T15:46:47Z

a friendly bump

goderbauer

LGTM

goderbauer · 2023-05-23T17:57:06Z

dev/devicelab/lib/tasks/integration_tests.dart

@@ -237,6 +241,10 @@ class IntegrationTest {
        ]);
      }

+      if (withTalkBack) {
+        await enableTalkBack();


Do we know here what device the test is targeting? Can we assert if someone accidentally sets withTalkBack to true on a device that doesn't support it (e.g. iOS)?

…lutter/flutter#127128)

flutter/flutter@f86c529...216605b 2023-05-24 engine-flutter-autoroll@skia.org Roll Flutter Engine from aba9dc4640c2 to eebcf36cd38f (3 revisions) (flutter/flutter#127458) 2023-05-24 engine-flutter-autoroll@skia.org Roll Flutter Engine from 76afb431a740 to aba9dc4640c2 (1 revision) (flutter/flutter#127453) 2023-05-24 engine-flutter-autoroll@skia.org Roll Flutter Engine from fc6df9512aa2 to 76afb431a740 (3 revisions) (flutter/flutter#127445) 2023-05-23 49699333+dependabot[bot]@users.noreply.github.com Bump actions/labeler from 6b107e7a7ee5e054e0bcce60de5181d21e2f00fb to 2713f7303c96cb1e69627957ec16eea0fd7f94a4 (flutter/flutter#127438) 2023-05-23 engine-flutter-autoroll@skia.org Roll Flutter Engine from 3535e0d301dd to fc6df9512aa2 (1 revision) (flutter/flutter#127440) 2023-05-23 47866232+chunhtai@users.noreply.github.com Migrates android semanitcs integration to integration test (flutter/flutter#127128) 2023-05-23 engine-flutter-autoroll@skia.org Roll Flutter Engine from 23b04314d5d2 to 3535e0d301dd (3 revisions) (flutter/flutter#127428) 2023-05-23 engine-flutter-autoroll@skia.org Roll Flutter Engine from ccf50f344d5d to 23b04314d5d2 (2 revisions) (flutter/flutter#127421) 2023-05-23 ian@hixie.ch Give channel descriptions in `flutter channel`, use branch instead of upstream for channel name (flutter/flutter#126936) 2023-05-23 72562119+tgucio@users.noreply.github.com Avoid catching errors in TextPainter tests (flutter/flutter#127307) 2023-05-23 christopherfujino@gmail.com [flutter_tools] delete entitlements files after copying to macos build dir (flutter/flutter#127417) 2023-05-23 engine-flutter-autoroll@skia.org Roll Flutter Engine from a6e37bde65fd to ccf50f344d5d (7 revisions) (flutter/flutter#127416) 2023-05-23 113669907+NikolajHarderNota@users.noreply.github.com modal bottom sheet barrier label (flutter/flutter#126431) 2023-05-23 jmccandless@google.com Fix the breaking of multi-code-unit characters in obscure mode (flutter/flutter#123366) 2023-05-23 engine-flutter-autoroll@skia.org Roll Flutter Engine from 168b0bf3f70d to a6e37bde65fd (1 revision) (flutter/flutter#127397) 2023-05-23 vashworth@google.com Revert "Log all output of ios-deploy" (flutter/flutter#127405) 2023-05-23 engine-flutter-autoroll@skia.org Roll Packages from 83959fb to d449a17 (6 revisions) (flutter/flutter#127394) 2023-05-23 vashworth@google.com Log all output of ios-deploy (flutter/flutter#127222) If this roll has caused a breakage, revert this CL and stop the roller using the controls here: https://autoroll.skia.org/r/flutter-packages Please CC bmparr@google.com,rmistry@google.com,stuartmorgan@google.com on the revert to ensure that a human is aware of the problem. To file a bug in Packages: https://github.com/flutter/flutter/issues/new/choose To report a problem with the AutoRoller itself, please file a bug: https://bugs.chromium.org/p/skia/issues/entry?template=Autoroller+Bug Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+doc/main/autoroll/README.md

…27128) I think the flake is due to setclipboard or semantics update race condition. I migrated the test to use integration test package which relies less on timing fixes flutter#124636

…lutter/flutter#127128)

flutter-dashboard bot added a: accessibility Accessibility, e.g. VoiceOver or TalkBack. (aka a11y) team Infra upgrades, team productivity, code health, technical debt. See also team: labels. labels May 18, 2023

chunhtai commented May 18, 2023

View reviewed changes

chunhtai force-pushed the issues/124636 branch from 4f53ee9 to b1fe007 Compare May 18, 2023 18:46

chunhtai requested a review from goderbauer May 18, 2023 18:47

goderbauer reviewed May 18, 2023

View reviewed changes

chunhtai force-pushed the issues/124636 branch from b1fe007 to 0e3092a Compare May 19, 2023 18:22

chunhtai requested a review from goderbauer May 19, 2023 18:24

goderbauer approved these changes May 23, 2023

View reviewed changes

chunhtai added 4 commits May 23, 2023 14:07

Migrates android semanitcs integration to integration test

a5fe6f0

move talkback adb to test runner

e172c67

Add android device check

64afe8b

update

9cffc0c

chunhtai force-pushed the issues/124636 branch from 09d430e to 9cffc0c Compare May 23, 2023 21:07

chunhtai added the autosubmit Merge PR when tree becomes green via auto submit App label May 23, 2023

auto-submit bot merged commit c687dcd into flutter:master May 23, 2023
139 checks passed

engine-flutter-autoroll mentioned this pull request May 24, 2023

Roll Flutter from f86c529b2cd2 to 216605b60f67 (18 revisions) flutter/packages#4071

Merged

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request May 24, 2023

c687dcd56 Migrates android semanitcs integration to integration test (f…

df3ba19

…lutter/flutter#127128)

This was referenced May 25, 2023

Support minifcation for apps that depend on AGP 8 and integration_test #127628

Merged

i127682 android api 33 test support #127689

Merged

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Aug 16, 2023

c687dcd56 Migrates android semanitcs integration to integration test (f…

3f2bf54

…lutter/flutter#127128)

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Aug 17, 2023

c687dcd56 Migrates android semanitcs integration to integration test (f…

283c101

…lutter/flutter#127128)

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Aug 17, 2023

c687dcd56 Migrates android semanitcs integration to integration test (f…

9154f66

…lutter/flutter#127128)

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Aug 17, 2023

c687dcd56 Migrates android semanitcs integration to integration test (f…

4e79704

…lutter/flutter#127128)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Migrates android semanitcs integration to integration test #127128

Migrates android semanitcs integration to integration test #127128

chunhtai commented May 18, 2023

chunhtai May 18, 2023

chunhtai May 18, 2023

chunhtai May 18, 2023

chunhtai commented May 19, 2023

chunhtai commented May 19, 2023 •

edited

chunhtai commented May 23, 2023

goderbauer left a comment

goderbauer May 23, 2023

		@@ -0,0 +1,643 @@
		// Copyright 2014 The Flutter Authors. All rights reserved.

Migrates android semanitcs integration to integration test #127128

Migrates android semanitcs integration to integration test #127128

Conversation

chunhtai commented May 18, 2023

Pre-launch Checklist

chunhtai May 18, 2023

Choose a reason for hiding this comment

chunhtai May 18, 2023

Choose a reason for hiding this comment

chunhtai May 18, 2023

Choose a reason for hiding this comment

chunhtai commented May 19, 2023

chunhtai commented May 19, 2023 • edited

chunhtai commented May 23, 2023

goderbauer left a comment

Choose a reason for hiding this comment

goderbauer May 23, 2023

Choose a reason for hiding this comment

chunhtai commented May 19, 2023 •

edited