Flutter support for Appium #12919

truongsinh · 2019-07-16T13:38:12Z

Update

Prototype available at https://www.npmjs.com/package/appium-flutter-driver but we need support from appium declaration first.

The problem

Flutter goes with flutter drive, the e2e test tool, but it does not work with app that has both flutter view and native view (such as embedding Flutter to existing app), or flutter view with web view (open a webview in Flutter).

Having appium flutter driver would solve both problems above, similar to http://appium.io/docs/en/writing-running-appium/web/hybrid/

Related issues:

The text was updated successfully, but these errors were encountered:

umutuzgur · 2019-07-16T13:53:35Z

Hi @truongsinh. That is a really nice idea but I don't think we have that many resources at the moment. Have you contacted the Flutter team to integrate their driver with the Appium project? There are already some drivers which are developed externally from the Appium team but integrated into the Appium project

truongsinh · 2019-07-16T13:56:57Z

@umutuzgur I created flutter/flutter#36259 . Meanwhile, can you show me to some documentation to get started with developing 3rd party Appium driver, and also some examples?

truongsinh · 2019-07-16T13:58:31Z

related threads: google/EarlGrey#778, flutter/flutter#32987

truongsinh · 2019-07-16T13:59:38Z

related threads: flutter/flutter#25465

umutuzgur · 2019-07-16T14:05:02Z

@truongsinh I will ask around where the documentation is for the custom drivers but in the mean time, these are some of the custom drivers appium-youiengine-driver, appium-tizen-driver

mykola-mokhnach · 2019-07-16T19:11:34Z

Linking flutter/flutter#17988

KazuCocoa · 2019-07-17T00:09:40Z

linking flutter/flutter#32062 (comment)
Espresso->Flutter like Espresso->Web

truongsinh · 2019-07-17T13:43:43Z

@umutuzgur one more question, does the driver have to be in JS, or it can be in other languages?

umutuzgur · 2019-07-18T07:43:57Z

@truongsinh It should be JS. I don't know if you can transpile Dart to JS but that would also work I think

truongsinh · 2019-07-21T16:08:43Z

@umutuzgur https://www.npmjs.com/package/appium-flutter-driver first prototype, working with both iOS and Android with limited commands (get text and tap only). But we need to declare this package in appium source code.

KazuCocoa · 2019-07-21T22:58:47Z

Awesome, @truongsinh

It's out of my curiosity.
Do you have any idea about which is better to communicate with DartVM directly or launch an http server for flutter_driver on app like this and communicate with it like appium-uiautomator2-driver/uiautomator2-server, xcuitest-driver/WebDriverAgent and espresso-driver/espresso-server?

Because flutter_driver has knowledge for flutter like https://github.com/flutter/flutter/blob/b712a172f9694745f50505c93340883493b505e5/packages/flutter_driver/lib/src/driver/driver.dart#L770-L821 and getRenderTree()

I considered which way was better for them.

truongsinh · 2019-07-22T01:14:01Z

@KazuCocoa at the moment both approaches are highly experimental thus I don't have a strong idea 😁. Though do you think the http server approach can do context-aware commands (some commands sent to native view, some to web view, some to Flutter view), and on iOS real devices (I understand that http server approach install another http server app on device under test, and it's always painful to install anything on iOS real devices).

On the other hand, Dart VM Service Protocol with extension ext.flutter.driver is undocumented protocol, thus direct communication to Dart VM might have broken changes in the future if the Flutter team refactor their code.

KazuCocoa · 2019-07-22T03:24:25Z

Thanks :)
I was thinking to build it similar to https://github.com/appium/appium-espresso-driver , eventually.
The server communicates with appium-flutter-driver written in JS extends base driver. Clients communicate with the appium-flutter-driver. The appium-flutter-driver handles context, flutter or android/ios in the layer.
I haven't considered iOS real device much yet... As you also addressed highly experimental, mine is also very prototype :)

On the other hand,

yeah...

KazuCocoa · 2019-07-22T10:16:27Z

We can probably catch up with communication protocol with Dart VM between flutter_driver <=> dart vm by their tests => https://github.com/flutter/flutter/tree/180aa0c0b1fc645aa661a9aeedc21ed91c93ad00/packages/flutter_driver/test

here also (for example)
https://github.com/flutter/flutter/blob/4e4b9bd0b1ae094d4c837c8cd6d318d15be7fe40/packages/flutter_driver/lib/src/extension/extension.dart

We should call screenshot via ios/android drivers since flutter 's take screenshot is calling commands outside dart vm. e.g. https://github.com/flutter/flutter/search?p=4&q=screenshot&unscoped_q=screenshot

truongsinh · 2019-07-22T11:14:32Z

Thanks, this really helps, so far I have been doing reverse engineering with FlutterDriver.connectedTo(printCommunication: true) 😂

As discussed in appium#12919 (comment), we need to declare package name and class name for Flutter Driver. In the long run however, I believe this part of code should be refactored so that arbitrary driver can run without code change from `appium` core. Security risk should be minimal because that package must be installed beforehand anyway.

truongsinh · 2019-07-22T11:51:05Z

@KazuCocoa @mykola-mokhnach could you take a look at #12945?

KazuCocoa · 2019-07-22T12:11:35Z

Btw, is there any known limitation for https://github.com/truongsinh/appium-flutter-driver ?

e.g. it can run only debug mode, only on simulator etc...

truongsinh · 2019-07-22T15:34:04Z

Btw, is there any known limitation for https://github.com/truongsinh/appium-flutter-driver ?

e.g. it can run only debug mode, only on simulator etc...

Currently the API is very limited, as documented in https://github.com/truongsinh/appium-flutter-driver#finders and at https://github.com/truongsinh/appium-flutter-driver#finders . I have been able to run the examples on Android emulator, Android real device (Pixel 2), iOS emulator.

I haven't run on iOS real device yet. As for mode, do you mean flutter mode or Android/iOS mode? Does it make sense to run test on non-debug mode :-? Anyway, as long as there's enableFlutterDriverExtension, we can control the Flutter app.

jlipps · 2019-07-22T15:42:39Z

@truongsinh this is a great idea. If you get me your e-mail somehow I will invite you to the appium developer slack chat, for this kind of thing sometimes real time communication is easier. DM @jlipps on twitter might be a good channel.

truongsinh · 2019-07-22T16:17:48Z

@truongsinh this is a great idea. If you get me your e-mail somehow I will invite you to the appium developer slack chat, for this kind of thing sometimes real time communication is easier. DM @jlipps on twitter might be a good channel.

DM'ed

truongsinh · 2019-07-23T15:15:07Z

We should call screenshot via ios/android drivers since flutter 's take screenshot is calling commands outside dart vm. e.g. https://github.com/flutter/flutter/search?p=4&q=screenshot&unscoped_q=screenshot

@KazuCocoa I published v0.0.5 with screenshot functionality. Actually we can have screenshot both from Dart VM and via jwproxy, depends on the needs. Screenshot from Dart VM is pure app, while screenshot via jwproxy may have extraneous element like top and bottom bar, as seen from these screenshots

KazuCocoa · 2019-07-24T00:45:42Z

Thanks 👍 #12919 (comment)

Ah, okay.
Appium has commands like http://appium.io/docs/en/commands/mobile-command/
We can use either way as the command like mobile:viewportScreenshot.

--

btw, I was able to do on iOS real devices (I understand that http server approach install another http server app on device under test, and it's always painful to install anything on iOS real devices).

umutuzgur · 2019-08-01T07:59:56Z

@truongsinh Should we close this issue since now we can communicate over slack?

truongsinh · 2019-08-01T08:15:27Z

it's true we are communicating over slack, but if we keep this issue open until (1) Flutter is actually supported in Appium, or (2) we make a decision that Appium would not support Flutter, we can easily see the status.

I don't have a strong opinion, how about others?

* Support Flutter driver As discussed in #12919 (comment), we need to declare package name and class name for Flutter Driver. In the long run however, I believe this part of code should be refactored so that arbitrary driver can run without code change from `appium` core. Security risk should be minimal because that package must be installed beforehand anyway. * `prepare` supersedes `prepublish` This will also helps with install directly via git, e.g. `npm i -g git://github.com/appium/appium` * add flutter driver in deps

ahashmi24 · 2019-11-25T08:03:14Z

@truongsinh
do we reach any conclusion? whether appium will support flutter based application? if yes please guide me with the initial setup for appium-flutter-driver or suggest to me if I can set-up appium-flutter-driver without bothering my existing Appium setup.

Many Thanks,
Ahmer

stone20130104 · 2019-11-25T09:11:07Z

@ahashmi24 , you can try this,
https://www.npmjs.com/package/appium-flutter-driver
https://github.com/KazuCocoa/proto-flutter-driver-server

stone20130104 · 2019-11-25T09:13:29Z

seems lots of us are waiting for appium-flutter-driver, could you please add flutter support in Appium as soon as possible, we are expecting to use it in Appium. Thanks!!!

vijaydeepak-rezi · 2019-12-09T10:56:38Z

Thanks 👍 #12919 (comment)

Ah, okay.
Appium has commands like http://appium.io/docs/en/commands/mobile-command/
We can use either way as the command like mobile:viewportScreenshot.

--

btw, I was able to do on iOS real devices (I understand that http server approach install another http server app on device under test, and it's always painful to install anything on iOS real devices).

@KazuCocoa How can we Run on IOS Real device using Appium-Flutter-Driver . Do we need to make any specific changes in Desired capability ?

mykola-mokhnach · 2020-01-24T15:24:56Z

Closed as fixed

gimnguyen · 2020-09-30T07:42:27Z

I found out this field in flutter semanticLabel

When I add this attribute to element. I can interact with that element by: MobileBy.AccessibilityId("payment")
It looks like adding an ID to element, right?
However, Text is not fit. Because you can not get text value by this way.
See screenshot below.

supertgtline · 2022-12-11T14:14:23Z

Hi, I have another question, I have multiple navigation bar tab can't add key, so how can I find correct tab I want to interact with them

umutuzgur added the Question label Jul 16, 2019

truongsinh mentioned this issue Jul 16, 2019

Need documentation for flutter driver when used with apps that include native UI flutter/flutter#36259

Open

truongsinh changed the title ~~Flutter support~~ Flutter support for Appium Jul 16, 2019

mykola-mokhnach added the Enhancement feature label Jul 16, 2019

truongsinh mentioned this issue Jul 22, 2019

Support Flutter driver #12945

Merged

9 tasks

mykola-mokhnach closed this as completed Jan 24, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Flutter support for Appium #12919

Flutter support for Appium #12919

truongsinh commented Jul 16, 2019 •

edited

umutuzgur commented Jul 16, 2019 •

edited

truongsinh commented Jul 16, 2019

truongsinh commented Jul 16, 2019

truongsinh commented Jul 16, 2019

umutuzgur commented Jul 16, 2019

mykola-mokhnach commented Jul 16, 2019

KazuCocoa commented Jul 17, 2019

truongsinh commented Jul 17, 2019

umutuzgur commented Jul 18, 2019 •

edited

truongsinh commented Jul 21, 2019

KazuCocoa commented Jul 21, 2019 •

edited

truongsinh commented Jul 22, 2019

KazuCocoa commented Jul 22, 2019

KazuCocoa commented Jul 22, 2019 •

edited

truongsinh commented Jul 22, 2019

truongsinh commented Jul 22, 2019

KazuCocoa commented Jul 22, 2019

truongsinh commented Jul 22, 2019

jlipps commented Jul 22, 2019

truongsinh commented Jul 22, 2019

truongsinh commented Jul 23, 2019 •

edited

KazuCocoa commented Jul 24, 2019

umutuzgur commented Aug 1, 2019

truongsinh commented Aug 1, 2019

ahashmi24 commented Nov 25, 2019

stone20130104 commented Nov 25, 2019

stone20130104 commented Nov 25, 2019

vijaydeepak-rezi commented Dec 9, 2019 •

edited

mykola-mokhnach commented Jan 24, 2020

gimnguyen commented Sep 30, 2020

supertgtline commented Dec 11, 2022

Flutter support for Appium #12919

Flutter support for Appium #12919

Comments

truongsinh commented Jul 16, 2019 • edited

Update

The problem

umutuzgur commented Jul 16, 2019 • edited

truongsinh commented Jul 16, 2019

truongsinh commented Jul 16, 2019

truongsinh commented Jul 16, 2019

umutuzgur commented Jul 16, 2019

mykola-mokhnach commented Jul 16, 2019

KazuCocoa commented Jul 17, 2019

truongsinh commented Jul 17, 2019

umutuzgur commented Jul 18, 2019 • edited

truongsinh commented Jul 21, 2019

KazuCocoa commented Jul 21, 2019 • edited

truongsinh commented Jul 22, 2019

KazuCocoa commented Jul 22, 2019

KazuCocoa commented Jul 22, 2019 • edited

truongsinh commented Jul 22, 2019

truongsinh commented Jul 22, 2019

KazuCocoa commented Jul 22, 2019

truongsinh commented Jul 22, 2019

jlipps commented Jul 22, 2019

truongsinh commented Jul 22, 2019

truongsinh commented Jul 23, 2019 • edited

KazuCocoa commented Jul 24, 2019

umutuzgur commented Aug 1, 2019

truongsinh commented Aug 1, 2019

ahashmi24 commented Nov 25, 2019

stone20130104 commented Nov 25, 2019

stone20130104 commented Nov 25, 2019

vijaydeepak-rezi commented Dec 9, 2019 • edited

mykola-mokhnach commented Jan 24, 2020

gimnguyen commented Sep 30, 2020

supertgtline commented Dec 11, 2022

truongsinh commented Jul 16, 2019 •

edited

umutuzgur commented Jul 16, 2019 •

edited

umutuzgur commented Jul 18, 2019 •

edited

KazuCocoa commented Jul 21, 2019 •

edited

KazuCocoa commented Jul 22, 2019 •

edited

truongsinh commented Jul 23, 2019 •

edited

vijaydeepak-rezi commented Dec 9, 2019 •

edited