Testing Android Accessibility Services #4910

yahman72 · 2015-04-14T14:17:39Z

I'm trying to find a way to test Accessibility Services on Android (4.4) but I don't get it working.

As a proof-of-concept I'm trying to test a simple accessiblity service that logs notifications.

I have been able to narrow down the issue to the UiAutomator. Everything works fine as long as I'm not using the UiAutomator, but as soon as the UiAutomator is involved the Accessibility Services are destroyed (see dumpsys and uiautomator outputs below). The Service are not disabled but simply destroyed i.e. they are still "on" according to the "Settings / Accessibility" i.e. to re-enable them I need to disable them first and then enable them.

The same happens if I'm starting an App via Appium, except that the dumpsys accessiblity fails when executed when the Appium Session is open.

Is this something that could/should be managed by the Appium Bootstrap i.e. keep the accessibility services enabled (or re-enable them)?

outputs with appium
before opening the session

> adb shell dumpsys accessibility
ACCESSIBILITY MANAGER (dumpsys accessibility)

User state[attributes:{id=0, currentUser=true, accessibilityEnabled=true, touchExplorationEnabled=false, displayMagnificationEnabled=false}
           services:{Service[label=Toaster, feedbackType[FEEDBACK_GENERIC], capabilities=0, eventTypes=TYPE_NOTIFICATION_STATE_CHANGED, notificationTimeout=0]}]


[14.04.2015 16:09:29 C:\\]

during open session

> adb shell dumpsys accessibility
ACCESSIBILITY MANAGER (dumpsys accessibility)

User state[attributes:{id=0, currentUser=true, accessibilityEnabled=true, touchExplorationEnabled=false, displayMagnificationEnabled=false,
Exception occurred while dumping:
java.lang.NullPointerException
        at com.android.server.accessibility.AccessibilityManagerService$Service.dump(AccessibilityManagerService.java:2216)
        at com.android.server.accessibility.AccessibilityManagerService.dump(AccessibilityManagerService.java:1510)
        at android.os.Binder.dump(Binder.java:300)
        at android.os.Binder.onTransact(Binder.java:266)
        at android.view.accessibility.IAccessibilityManager$Stub.onTransact(IAccessibilityManager.java:178)
        at android.os.Binder.execTransact(Binder.java:404)
        at dalvik.system.NativeStart.run(Native Method)

[14.04.2015 16:09:47 C:\\]

after closing the session

> adb shell dumpsys accessibility
ACCESSIBILITY MANAGER (dumpsys accessibility)

User state[attributes:{id=0, currentUser=true, accessibilityEnabled=true, touchExplorationEnabled=false, displayMagnificationEnabled=false}
           services:{}]


[14.04.2015 16:10:00 C:\\]

outputs without appium

[14.04.2015 15:52:15 C:\\]
> adb shell dumpsys accessibility
ACCESSIBILITY MANAGER (dumpsys accessibility)

User state[attributes:{id=0, currentUser=true, accessibilityEnabled=true, touchExplorationEnabled=false, displayMagnificationEnabled=false}
           services:{Service[label=Toaster, feedbackType[FEEDBACK_GENERIC], capabilities=0, eventTypes=TYPE_NOTIFICATION_STATE_CHANGED, notificationTimeout=0]}]


[14.04.2015 15:53:11 C:\\]
> adb shell uiautomator dump --compressed
UI hierchary dumped to: /storage/emulated/0/window_dump.xml

[14.04.2015 15:53:35 C:\\]
> adb shell dumpsys accessibility
ACCESSIBILITY MANAGER (dumpsys accessibility)

User state[attributes:{id=0, currentUser=true, accessibilityEnabled=true, touchExplorationEnabled=false, displayMagnificationEnabled=false}
           services:{}]


[14.04.2015 15:53:37 C:\\]
> adb shell dumpsys accessibility
ACCESSIBILITY MANAGER (dumpsys accessibility)

User state[attributes:{id=0, currentUser=true, accessibilityEnabled=true, touchExplorationEnabled=false, displayMagnificationEnabled=false}
           services:{Service[label=Toaster, feedbackType[FEEDBACK_GENERIC], capabilities=0, eventTypes=TYPE_NOTIFICATION_STATE_CHANGED, notificationTimeout=0]}]


[14.04.2015 15:54:17 C:\\]
> adb shell uiautomator dump
UI hierchary dumped to: /storage/emulated/0/window_dump.xml

[14.04.2015 15:54:27 C:\\]
> adb shell dumpsys accessibility
ACCESSIBILITY MANAGER (dumpsys accessibility)

User state[attributes:{id=0, currentUser=true, accessibilityEnabled=true, touchExplorationEnabled=false, displayMagnificationEnabled=false}
           services:{}]

[14.04.2015 15:54:31 C:\\]

If this a Android Platform thing (= UiAutomator bug or "feature") any other ideas how this could be done?

PS.
The same issue seems to be on Android 4.3 (no workarounds in these threads either)
https://groups.google.com/forum/#!msg/adt-dev/-9cyoNlRTh4/vzN0QyUwTVoJ
http://stackoverflow.com/questions/20341440/accessibilityservice-is-destroyed-when-uiautomator-is-run

The text was updated successfully, but these errors were encountered:

jlipps · 2015-04-15T19:50:51Z

this just makes me think that the solution needs to be tightly integrated within our uiautomator code, so that it can share access to the uiautomator objects

v3rde · 2015-10-07T12:08:39Z

I'm stuck with same issue. Any progress on that?

vivek978 · 2016-02-22T10:28:03Z

any update on this issue

Jonahss · 2016-02-22T18:17:42Z

@imurchie

On Mon, Feb 22, 2016 at 2:28 AM, vivek978 notifications@github.com wrote:

any update on this issue

—
Reply to this email directly or view it on GitHub
#4910 (comment).

vivek978 · 2016-03-11T06:31:01Z

@Jonahss
@jlipps
@sebv
any update on this please.

vivek978 · 2016-03-11T06:32:28Z

@imurchie

Sandeep0001 · 2016-04-12T04:50:08Z

Any update???

lwfwind · 2016-05-12T02:52:41Z

I'm stuck with same issue. Any progress on that?

Jonahss · 2016-05-12T18:31:20Z

I'll ping the appium devs.

jlipps · 2016-05-12T19:31:42Z

If the issue is related to UiAutomator I don't see how we can work around that given that Appium runs on UiAutomator...?

nikhil969 · 2016-05-18T06:31:50Z

Is there any temporary workaround

On Friday 13 May 2016, Jonathan Lipps notifications@github.com wrote:

If the issue is related to UiAutomator I don't see how we can work around
that given that Appium runs on UiAutomator...?

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub
#4910 (comment)

yahman72 · 2016-05-18T08:20:11Z

@nikhil969 - there might be a workaround but this depends heavily on what you are testing.
I.e. are you testing the accessibility services in general or just a special scenario (e.g. using accesibility services to catch Toast messages, see #4824)

nikhil969 · 2016-05-18T13:52:18Z

I think if you upgrade uiautomator to uiautomator v2 this issue will be resolved

sravanmedarapu · 2016-12-02T07:42:14Z

It is not possible to test Accessibility Services in parallel with UiAutomator.

Both UiAutomator(including UiAutomator2) and AccessibilityService depends on same resources internally which are singleton in nature and UiAutomator API always holds lock on those resources.

As a result it just not possible to use both UiAutomator and AccessibilityService at a time.

maxblumental · 2018-07-18T15:40:56Z

@sravanmedarapu Could you please provide a little bit more details about the held resources in terms of code? I am confused. We can run simultaneously several AccessibilityServices but running just one UiAutomation in the system causes all AccessibilityServices to suspend.

By the way, I found out that you can run UiAutomation along with AccessibilityServices if you uses a special flag:

int flags = UiAutomation.FLAG_DONT_SUPPRESS_ACCESSIBILITY_SERVICES;
UiAutomation mUiAutomation = 
    InstrumentationRegistry.getInstrumentation().getUiAutomation(flags);

If you obtain UiAutomation in this way, you'll be able to run your tests without suspending AccessibilityServices.

darran-kelinske-fivestars · 2019-04-11T16:07:21Z

@maxblumental had good advice - also found we can do the following:

        val flags = UiAutomation.FLAG_DONT_SUPPRESS_ACCESSIBILITY_SERVICES
        Configurator.getInstance().uiAutomationFlags = flags

        mDevice = UiDevice.getInstance(InstrumentationRegistry.getInstrumentation())

There is a related StackOverflow question:

https://stackoverflow.com/questions/29035320/are-uiautomator-and-accessibilityservices-not-compatible/55636900#55636900

darran-kelinske-fivestars · 2019-04-11T17:52:54Z

@maxblumental was this FLAG_DONT_SUPPRESS_ACCESSIBILITY_SERVICES implemented in appium?

darran-kelinske-fivestars · 2019-04-11T22:24:57Z

@sravanmedarapu why was this closed, when there is a possible fix mentioned in the previous comments?

mrahmadt · 2019-07-14T20:18:29Z

I'm testing with Appium 1.14.0 and this issue still exists.
Unfortunately, this is a core function of my app, I wish if appium team would at least mention this limitation in appium (I spent tremendous time building my test cases then I discovered this limitation :( )

KazuCocoa · 2019-07-15T03:20:04Z

Thanks. I noticed the comment.

I've added Configurator.getInstance().setUiAutomationFlags(UiAutomation.FLAG_DONT_SUPPRESS_ACCESSIBILITY_SERVICES); in UIA2 server.

It is appreciated if someone can try the below steps to test the flag.
The test target should be over API level 24.

git clone git@github.com:kazucocoa/appium-uiautomator2-server.git and run git checkout do-no-suppress-as && npm install && npm build in appium-uiautomator2-server
git clone git@github.com:appium/appium.git and npm install
cp -f path/to/appium-uiautomator2-server/apks/* path/to/appium/node_modules/appium-uiautomator2-server/apks/
- If names of apks in path/to/appium-uiautomator2-server/apks/* differe from path/to/appium/node_modules/appium-uiautomator2-server/apks/, please rename them properly
  - Currently, the name has v3.8.0 version number. It will increment if a new uia2-server will be released.
./build/lib/main.js (Start appium sever)
Run tests

https://appiumpro.com/editions/47

espresso driver probably is also helpful since its almost API does not work via uiautomator

mrahmadt · 2019-07-15T10:15:38Z

Hi
I couldn't find this command
npm checkout do-no-suppress-as
can you check if this is correct?

KazuCocoa · 2019-07-15T13:45:56Z

sorry, that was git checkout

mrahmadt · 2019-07-15T20:45:10Z

Thanks. I noticed the comment.

I've added Configurator.getInstance().setUiAutomationFlags(UiAutomation.FLAG_DONT_SUPPRESS_ACCESSIBILITY_SERVICES); in UIA2 server.

It is appreciated if someone can try the below steps to test the flag.
The test target should be over API level 24.
1. git clone `git@github.com:kazucocoa/appium-uiautomator2-server.git` and run `git checkout do-no-suppress-as && npm install && npm build` in appium-uiautomator2-server

2. git clone `git@github.com:appium/appium.git` and `npm install`

3. `cp -f path/to/appium-uiautomator2-server/apks/* path/to/appium/node_modules/appium-uiautomator2-server/apks/`
   
   * If names of apks in `path/to/appium-uiautomator2-server/apks/*` differe from `path/to/appium/node_modules/appium-uiautomator2-server/apks/`, please rename them properly
     
     * Currently, the name has `v3.8.0` version number. It will increment if a new uia2-server will be released.

4. `./build/lib/main.js` (Start appium sever)

5. Run tests
https://appiumpro.com/editions/47

espresso driver probably is also helpful since its almost API does not work via uiautomator

I assume you will need to run npm build before ./build/lib/main.js ? because the file is not exists

anyway I tried to run npm build, no output showing, so I copied the command from package.json and it's shown below error

~/appium# gulp transpile
[20:43:17] Using gulpfile ~/appium/gulpfile.js
TypeError: Cannot read property 'apply' of undefined
    at /usr/lib/nodejs/gulp/bin/gulp.js:132:20
    at _combinedTickCallback (internal/process/next_tick.js:131:7)
    at process._tickCallback (internal/process/next_tick.js:180:9)
    at Function.Module.runMain (module.js:695:11)
    at startup (bootstrap_node.js:188:16)
    at bootstrap_node.js:609:3

KazuCocoa · 2019-07-16T02:39:59Z

npm build should be done in step 2. So, ./build/lib/main.js should wok after step2, too.
In step 3, it simply replaces apks which will be installed. It does not require re-building by npm.

Can you run appium on your local in step2?
(The step 4 can be node . instead in the appium directory)

KazuCocoa · 2019-08-05T23:43:50Z

Weer you able to try out @mrahmadt ?

mrahmadt · 2019-08-06T04:11:23Z

I used script command to enable accessibility service

On Tue, Aug 6, 2019 at 02:44 Kazuaki Matsuo ***@***.***> wrote: Weer you able to try out @mrahmadt <https://github.com/mrahmadt> ? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#4910?email_source=notifications&email_token=AAHJ3YPIOA5P3CX54757R5LQDC3OVA5CNFSM4BABB7GKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD3TMP3I#issuecomment-518440941>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAHJ3YPGGOBY4QIYOL3RSITQDC3OVANCNFSM4BABB7GA> .

-- echo "Hello World :)"

hari49951 · 2020-03-24T19:56:07Z

@KazuCocoa
I am using Appium 1.16.0 to test the Talkback functionality on my app. When I trigger automation test to do click on button there is no Talkback announcement but when I do same step manually there is a Talkback announcements.
Any idea on this issue?

KazuCocoa · 2020-03-25T00:56:47Z

You can try out appium/appium-uiautomator2-server#289 (comment)

KazuCocoa · 2020-03-30T10:51:31Z

disableSuppressAccessibilityService: true disables the suppress accessibility service with the latest appium@beta

hari49951 · 2020-03-30T16:59:54Z

You can try out appium/appium-uiautomator2-server#289 (comment)

This solution is working for me as expected. Thank you so much @KazuCocoa

shailajaparida · 2021-07-12T16:54:14Z

Hello team
This is working with appium-uiautomator2 with version 18 above and you can integrate with accessibility Insight tool for Android . Thank you very much for giving the fix @KazuCocoa

nnkpiyush · 2022-07-07T09:59:46Z

How can I import UiAUtomation?

triager assigned Jonahss Apr 14, 2015

triager added the Needs Triage bugs which are not yet confirmed label Apr 14, 2015

yahman72 mentioned this issue Apr 14, 2015

Proposal how to support Toast Messages on Android #4824

Closed

jlipps unassigned Jonahss Apr 22, 2015

sebv added this to the Features milestone Jun 3, 2015

sebv added Enhancement feature Android related to Android-native driver(s) and removed Needs Triage bugs which are not yet confirmed labels Jun 3, 2015

jlipps added Enhancement feature and removed Needs Triage bugs which are not yet confirmed Bug a problem that needs fixing Enhancement feature labels Nov 10, 2016

jlipps removed this from the Features milestone Nov 10, 2016

jlipps assigned sravanmedarapu Nov 21, 2016

sravanmedarapu closed this as completed Dec 2, 2016

dpgraham marked this as a duplicate of #6170 Jul 20, 2017

dpgraham mentioned this issue Jul 20, 2017

Accessibility services stops when running appium tests on android device #6170

Closed

This was referenced Jul 15, 2019

feat: do not suppress accessibility services appium/appium-uiautomator2-server#289

Merged

set do not suppress accessibility services for uiautomation appium/appium-espresso-driver#468

Closed

This was referenced Mar 29, 2020

feat: add disableSuppressAccessibilityService appium/appium-espresso-driver#559

Merged

docs: add disableSuppressAccessibilityService for uia2 and espresso #14121

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Testing Android Accessibility Services #4910

Testing Android Accessibility Services #4910

yahman72 commented Apr 14, 2015

jlipps commented Apr 15, 2015

v3rde commented Oct 7, 2015

vivek978 commented Feb 22, 2016

Jonahss commented Feb 22, 2016

vivek978 commented Mar 11, 2016

vivek978 commented Mar 11, 2016

Sandeep0001 commented Apr 12, 2016

lwfwind commented May 12, 2016

Jonahss commented May 12, 2016

jlipps commented May 12, 2016

nikhil969 commented May 18, 2016

yahman72 commented May 18, 2016

nikhil969 commented May 18, 2016 •

edited

sravanmedarapu commented Dec 2, 2016 •

edited

maxblumental commented Jul 18, 2018 •

edited

darran-kelinske-fivestars commented Apr 11, 2019 •

edited

darran-kelinske-fivestars commented Apr 11, 2019

darran-kelinske-fivestars commented Apr 11, 2019

mrahmadt commented Jul 14, 2019

KazuCocoa commented Jul 15, 2019 •

edited

mrahmadt commented Jul 15, 2019

KazuCocoa commented Jul 15, 2019

mrahmadt commented Jul 15, 2019 •

edited

KazuCocoa commented Jul 16, 2019

KazuCocoa commented Aug 5, 2019

mrahmadt commented Aug 6, 2019 via email

hari49951 commented Mar 24, 2020

KazuCocoa commented Mar 25, 2020

KazuCocoa commented Mar 30, 2020

hari49951 commented Mar 30, 2020

shailajaparida commented Jul 12, 2021

nnkpiyush commented Jul 7, 2022

Testing Android Accessibility Services #4910

Testing Android Accessibility Services #4910

Comments

yahman72 commented Apr 14, 2015

jlipps commented Apr 15, 2015

v3rde commented Oct 7, 2015

vivek978 commented Feb 22, 2016

Jonahss commented Feb 22, 2016

vivek978 commented Mar 11, 2016

vivek978 commented Mar 11, 2016

Sandeep0001 commented Apr 12, 2016

lwfwind commented May 12, 2016

Jonahss commented May 12, 2016

jlipps commented May 12, 2016

nikhil969 commented May 18, 2016

yahman72 commented May 18, 2016

nikhil969 commented May 18, 2016 • edited

sravanmedarapu commented Dec 2, 2016 • edited

maxblumental commented Jul 18, 2018 • edited

darran-kelinske-fivestars commented Apr 11, 2019 • edited

darran-kelinske-fivestars commented Apr 11, 2019

darran-kelinske-fivestars commented Apr 11, 2019

mrahmadt commented Jul 14, 2019

KazuCocoa commented Jul 15, 2019 • edited

mrahmadt commented Jul 15, 2019

KazuCocoa commented Jul 15, 2019

mrahmadt commented Jul 15, 2019 • edited

KazuCocoa commented Jul 16, 2019

KazuCocoa commented Aug 5, 2019

mrahmadt commented Aug 6, 2019 via email

hari49951 commented Mar 24, 2020

KazuCocoa commented Mar 25, 2020

KazuCocoa commented Mar 30, 2020

hari49951 commented Mar 30, 2020

shailajaparida commented Jul 12, 2021

nnkpiyush commented Jul 7, 2022

nikhil969 commented May 18, 2016 •

edited

sravanmedarapu commented Dec 2, 2016 •

edited

maxblumental commented Jul 18, 2018 •

edited

darran-kelinske-fivestars commented Apr 11, 2019 •

edited

KazuCocoa commented Jul 15, 2019 •

edited

mrahmadt commented Jul 15, 2019 •

edited