Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Missing accessibility-id for testing purposes #17988

Open
fhuonder opened this issue May 29, 2018 · 18 comments

Comments

Projects
None yet
@fhuonder
Copy link
Contributor

commented May 29, 2018

Hi,

We'd like to test our Flutter-App with Appium. To unambiguously identify the elements (text fields, buttons, ...) it is necessary to provide a "technical" identifier. Searching by name (element's label) is not a stable approach (one problem e.g. ist that the same name could be used by different elements on the view).

Appium offers the selector strategies listed here: http://appium.io/docs/en/commands/element/find-elements.
A stable strategy would be when we could assign an accessibility id to elements (for Android this would be the "content-desc"). Another approach would be to use the native element identifiers (resource-id for android, name for iOS). These can both not be set on the flutter elements.

We found out that when a Semantic Object is created for an element that its label is taken as the accessiblity identifier. But this brings two problems: First, the label is, in our understanding, not the same as the id and it should be possible to treat them differently. Second, the object tree has the semantic node next (as sibling) to the element and this leads to the situation that the actual element to test cannot be identified because there could be further elements.

This situation makes it very hard to test the app with Appium (we think the same problems occur with other testing frameworks).

Do we miss something here?
When not: Is there a workaround?

Thanks for your answer.
Best regards,
Florian

@sdolski

This comment has been minimized.

Copy link

commented Jun 1, 2018

After further investigation we additionally found out that the part with the Semantic Object only works for iOS. In Android, there is not any information available. Means with Appium Desktop you don't see any information. Means it is not possible to select any element.

So in Android it is not possible to test anything.

@zacoid55

This comment has been minimized.

Copy link

commented Jun 26, 2018

Very much wanting this as well. Or if not - any pointers as to where to write the code to expose the content-desc and accessibility ids would be greatly appreciated 🙏

@zoechi zoechi added the framework label Jun 26, 2018

@srix55

This comment has been minimized.

Copy link

commented Aug 17, 2018

Any updates? I would like to use Appium and test flutter on AWS device farm. Currently, I see that flutter is making it hard for ui test automation frameworks to be used. Any workaround?

@zoechi zoechi added the a: tests label Aug 20, 2018

@YAPPDEF

This comment has been minimized.

Copy link

commented Sep 13, 2018

We'd also like to use Appium to test our flutter apps.
Would it be possible to pass the Widget-key to iOS and Android as AccessibilityIdentifier?

Flutter apps have no accessibility identifier and customised resource id. So, it’s quite difficult to implement robust test automation scenario so far.
Flutter app with Appium

@jonahwilliams can you maybe help?

@srix55

This comment has been minimized.

Copy link

commented Sep 15, 2018

@YAPPDEF I guess it's not high priority for the flutter team because not many people are asking for this. I think, that when more software giants start using flutter, this will take the stage. If so, we are looking at a solution that's probably at least a year away.

@superbgv

This comment has been minimized.

Copy link

commented Sep 18, 2018

Even we are looking to use Appium to test our flutter app. But it seems only accessibility id is available in iOS, there are no resource-id or content-desc values which are used as unique identifiers in android. Any work/fix can be expected in near future @jonahwilliams ?

@jonahwilliams

This comment has been minimized.

Copy link
Contributor

commented Sep 18, 2018

We understand that there is some work to do here but we have no concrete plans yet.

@Skyggedans

This comment has been minimized.

Copy link

commented Feb 24, 2019

Any progress on this? I'm planning to use Flutter to develop new application for RealWear HMT-1, it's an industrial Android computer that is worn on the head and all the interaction with it is built on voice commands. It runs a background service, which translates voice commands into button clicks and other gestures. Virtually every application can be controlled by voice, but in order to help the service find some control you need, you should provide it an XML file with some mappings. It supports searching by text, id and content_description attributes. Using text isn't reliable approach, I'd like to use id or content_description, but currently it seems impossible.

@chenyang88

This comment has been minimized.

Copy link

commented Mar 4, 2019

use appium to test flutter app, but there is no content_desc, resource-id or acessibility_id, could you tell me how should we test flutter app, use text to find elements???? are you kidding??? please support content-desc or accessibility-id for the developer, please please please~~ quickly quickly quickly~~

@jmalickenexient

This comment has been minimized.

Copy link

commented Apr 4, 2019

Please support this... this is critical for enterprises wanting to use your framework

@Skyggedans

This comment has been minimized.

Copy link

commented Apr 4, 2019

As I see in master branch, in the upcoming releases the AccessibilityBridge class will become public, so you can inherit it and override as you need to set any attributes you want, and then use it for your application MainActivity via AccessibilityDelegate.

@KaveriMariappan

This comment has been minimized.

Copy link

commented Apr 18, 2019

Thanks Skuggedans, Kindly let me know any sample Project , So that i will check with my automation framework

@Skyggedans

This comment has been minimized.

Copy link

commented Apr 18, 2019

Hi Kaveri, I'd be glad to provide you any info regarding this, but it's only my investigations based on Flutter engine sources, and my considerations, how would I solve such problems, if I run into them in the future. And of course, using an upcoming Flutter release. At this time, I fully satisfied with current implementation for HMT-1 device, and some things I can solve using Semantics wrapper (you can check it also, but it'll not allow you to explicitly set the resource_id or content_description attributes, you can do this only for Text fields and only under certain conditions)

@KaveriMariappan

This comment has been minimized.

Copy link

commented Apr 20, 2019

Thanks Skyaggedans ...

Hi Team,

I hope this discussion started very long back, if anybody find solution and implemented with real time automation, Kindly share your experience and solution .
Regards,
M.Kaveri

@kuyazee

This comment has been minimized.

Copy link

commented May 10, 2019

Our team is also currently looking for this feature, because we want to migrate some of our apps to use Flutter instead of native Swift and Kotlin. It would be awesome if this gets implemented soon.

@srix55

This comment has been minimized.

Copy link

commented May 10, 2019

@jonahwilliams It would be great if you could bring this issue to the flutter team's attention. I do understand that they are super busy after the announcements at Google IO. But, I don't see any way of getting past this issue for serious testing.

@dnfield

This comment has been minimized.

Copy link
Member

commented May 15, 2019

We're working on something like this for both EarlGrey and Espresso. @adazh this sounds like another case that's close to a similar request to use UIAutomator.

@adazh

This comment has been minimized.

Copy link

commented May 16, 2019

As Dan mentioned, we are working on both Espresso and EarlGrey to support Flutter UI testing.

In terms of matchers, we started with matching by text, tooltip, runtime type, value key, and if you guys are familiar with Espresso/EarlGrey's testing APIs, we might consider similar offerings for Flutter - something like onFlutterWidget(withTooltip("foo-tooltip"))
.perform(click())
.check(matches(withText("foo-text")));

@ALL, wondering whether Espresso/EarlGrey is a good fit for your testing needs? Also interested in learning what would be the ideal Flutter matchers? We probably should prioritize these first :). Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.