[uitests] Work for get appium uitests with CI #14488

rmarinho · 2023-04-10T11:47:54Z

Description of Change

Initial work to write UITests using Appium as backend. This would run against all platforms iOS, Android, Catalyst and Windows.

CI pipeline here: https://devdiv.visualstudio.com/DevDiv/_build?definitionId=17939&_a=summary

Requisites:

For Windows - Make sure DeveloperMode is turned on
Install nodejs
- Windows https://nodejs.org/dist/v18.13.0/node-v18.13.0-x64.msi
- mac https://nodejs.org/dist/v18.16.0/node-v18.16.0.pkg
Validate you have on your path %USERPROFILE%\AppData\Roaming\npm
On Windows machine install Windows App Driver from https://github.com/microsoft/WinAppDriver/releases/tag/v1.2.1
To install Appium run the following as Administrator https://github.com/dotnet/maui/blob/appium/eng/scripts/appium-install.ps1

Running from the command line (you can replace ./build.ps1 by `dotnet cake`)

./build.ps1 --target=dotnet-buildtasks --configuration="Release" --workloads=global

Android

./build.ps1 --target=dotnet-samples --configuration="Release" --workloads=global --android --usenuget=false
./build.ps1 -Script eng/devices/android.cake --target=uitest --project="/Users/ruimarinho/dotnet/maui/src/Controls/tests/UITests/Controls.AppiumTests.csproj" --appproject="/Users/ruimarinho/dotnet/maui/src/Controls/samples/Controls.Sample.UITests/Controls.Sample.UITests.csproj"--device="android-emulator-32_30" --apiversion="30" --configuration="Release"

Windows

./build.ps1 --target=dotnet-samples --configuration="Release" --windows --verbosity=diagnostic --workloads=global --usenuget=false
./build.ps1 -Script eng/devices/windows.cake --target=uitest --project="/Users/ruimarinho/dotnet/maui/src/Controls/tests/UITests/Controls.AppiumTests.csproj" --appproject="/Users/ruimarinho/dotnet/maui/src/Controls/samples/Controls.Sample.UITests/Controls.Sample.UITests.csproj" --device="windows10" --apiversion="10.0.20348" --configuration="Release"

iOS

./build.ps1 --target=dotnet-samples --configuration="Release" --workloads=global --ios --usenuget=false
./build.ps1 -Script  eng/devices/ios.cake --target=uitest --project="/Users/ruimarinho/dotnet/maui/src/Controls/tests/UITests/Controls.AppiumTests.csproj" --appproject="/Users/ruimarinho/dotnet/maui/src/Controls/samples/Controls.Sample.UITests/Controls.Sample.UITests.csproj" --device=ios-simulator-64 --apiversion="16.2" --configuration="Release"

Catalyst

./build.ps1 --target=dotnet-samples --configuration="Release" --catalyst --verbosity=diagnostic --workloads=global --usenuget=false
./build.ps1 -Script eng/devices/catalyst.cake --target=uitest --project="/Users/ruimarinho/dotnet/maui/src/Controls/tests/UITests/Controls.AppiumTests.csproj" --appproject="/Users/ruimarinho/dotnet/maui/src/Controls/samples/Controls.Sample.UITests/Controls.Sample.UITests.csproj" --apiversion="10.13" --configuration="Release" --device=mac

Running from the IDE

Deploy the app Control.Sample.UITests to your simulator, device or on your desktop.
Build the Controls..AppiumTests.csproj so the tests show up on the Test Explorer
Run the tests

fixes #11232

eng/devices/android.cake

PureWeen · 2023-05-05T13:51:58Z

src/Controls/tests/UITests/Tests/AppiumPlatformsTestBase.cs

+				var logDir = (Path.GetDirectoryName(Environment.GetEnvironmentVariable("APPIUM_LOG_FILE")) ?? Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location))!;
+
+				var pageSource = Driver?.PageSource!;
+				File.WriteAllText(Path.Combine(logDir, $"{TestContext.CurrentContext.Test.MethodName}-PageSource.txt"), pageSource);


Currently only tested on windows via the script:

I think the logDir here always evaluates to / because when the script runs it sets the appium_log_dir to

Setting environment variable: APPIUM_LOG_FILE = '/appium_windows.log'

Should we make sure this gets set to an absolute path from the cake script?

We should probably be using BINLOG_DIR instead of BINLOG_ARG when setting that

eng/devices/windows.cake

eng/scripts/appium-install.ps1

Microsoft.Maui.sln

src/Controls/tests/UITests/Tests/AppiumPlatformsTestBase.cs

github-actions · 2023-05-16T10:59:52Z

Thank you for your pull request. We are auto-formatting your source code to follow our code guidelines.

rmarinho · 2023-05-16T16:04:54Z

Created a issue to follow up on the TODOs: #15113

* More stuff with query * Update to Appium 2.0 * Cleanup * Fix tests * Add mac tests * Multiple testcase * Fixes for windows * Deploy as unpackaged app * Fixes for run UITests on windows * Cleanup and move to TestUtils projects for Appium and Xamarin.UITest * Fix references * Add to mac sln * Fix sln * Add uitest yaml * Fix sln * Update yml * more fixes * again powershell * fix ident * Add path and try call cake * Fix * Start background * Start appium on the background * Just run android for now * try install app * try again * if it fails continue * fix paths * appium * try again * try again * fix * again * try again * location of samples * build samples * do not skip xcode * more debug info * Publish test results * nUnit tests for now * try any file * no retry * try vstest * FIX PATH * install more drivers * usenuget when building samples * Move to helpers * Try run iOS tests * try build both * Add ALLTests * Android folder * fixes * try again * Fix path * Cleanup android cake * cleanup iOS cake * add mac cake * fix version * try again * try again * upDATE TOOLS * try again * fix again * again * again * again * rollback xharness * increase timeout * run with dotnet path * Fix dir * Try again * fix * Fix * build and then run tests * fix platformVersion passed to APPIUM * Don t do verbosity * Log for appium * Try 16.2 * Add platform to appium logs * Fix * Start test server * Wait for server to start * Fix typo * Add windows cake * Set uitests windows yaml * provision on macos only * Fix path for app windows * Rename winui * Try again * Make sure to stop and dispose the server * fix template * Install windows driver too * Better appium install drivers * specify device * Fix binlog android * Run device tests in release * Try no path * Try with debug configuration * try build samples just for that platform * Release * rename to match platform * try cleanup * Add mac uitests * Build only for catalyst * Add information * Version mac * Fix folder path * Try build and run windows tests * more stuff on install script * Add retry for ios * run always on xamarin pool * Fix windows * Allow to pass configuration to catalyst * Add bindir info * Try fix windows build * Try set env variables before * [Sample] Clean fields when tapping login button * Take screenshot before querying text element * Enable automationmode-without-authentication * Try using action instead * Use same machine pool as ios/android for mac * Change entry to not get autocorrect pop-up and add delay before click * Disable keyboard completion for catalyst runs * Auto-format source code * Update sln * Fix csproj * Add Sample UITests * Fix yaml * Use the new app uitest sandbox project * Remove old code * Cleanup * Fix Android manifest * Fix build for tizen * Rename projects * Some more renames * Fix ios for PR's * Update MainPageTests.cs * Add to mac slnf * Try more stuff for the mac * Try run before * don t set automation mode * Try Xcode select and move to ventura * Fix script * Dont xcode select * Bool and not string * try old appium * Fix version * Can we skip provisioning * We can't skip provisioning * Dont provision on windows * Update sln and packages * Update node * Configure the windows path to the exe * Auto-format source code --------- Co-authored-by: Scott Banning <scoban@microsoft.com> Co-authored-by: GitHub Actions Autoformatter <autoformat@example.com>

rmarinho added 30 commits January 5, 2023 15:27

More stuff with query

0860d45

Update to Appium 2.0

f81436b

Cleanup

2f10650

Fix tests

dd4508f

Add mac tests

af84aed

Multiple testcase

811d02c

Fixes for windows

923371e

Deploy as unpackaged app

36f1318

Fixes for run UITests on windows

b85f2ce

Cleanup and move to TestUtils projects for Appium and Xamarin.UITest

602b7ad

Fix references

b2490a4

Add to mac sln

b54b6ee

Fix sln

1a62b63

Add uitest yaml

af48c02

Merge branch 'appium' of https://github.com/dotnet/maui into appium

e405310

Fix sln

a93fef6

Update yml

f8241fd

more fixes

2697397

again powershell

d1039c8

fix ident

a378b31

Add path and try call cake

6f9e60f

Fix

c4924bf

Start background

a4140ae

Start appium on the background

1b97bb3

Just run android for now

39345f4

try install app

d0a4dec

try again

f8cb5bf

if it fails continue

bb17f86

fix paths

3eb1fc7

appium

d17eeac

rmarinho added 10 commits May 3, 2023 11:21

Try Xcode select and move to ventura

ce04249

Fix script

05244fb

Dont xcode select

98643e9

Bool and not string

07c05f7

try old appium

3b310da

Fix version

34d0aaa

Can we skip provisioning

b5e108f

We can't skip provisioning

17f9a74

Dont provision on windows

15b7906

Update sln and packages

d0d9a19

PureWeen reviewed May 4, 2023

View reviewed changes

eng/devices/android.cake Show resolved Hide resolved

PureWeen requested changes May 5, 2023

View reviewed changes

eng/scripts/appium-install.ps1 Show resolved Hide resolved

PureWeen reviewed May 5, 2023

View reviewed changes

Microsoft.Maui.sln Show resolved Hide resolved

PureWeen reviewed May 5, 2023

View reviewed changes

src/Controls/tests/UITests/Tests/AppiumPlatformsTestBase.cs Outdated Show resolved Hide resolved

rmarinho and others added 3 commits May 16, 2023 11:40

Update node

e0105a1

Configure the windows path to the exe

277e94f

Auto-format source code

0aee48f

Merge branch 'main' into appium

318c3b5

PureWeen approved these changes May 16, 2023

View reviewed changes

rmarinho enabled auto-merge (squash) May 16, 2023 16:31

Ignore test for now

85f511f

PureWeen approved these changes May 16, 2023

View reviewed changes

rmarinho merged commit dec8b24 into main May 16, 2023
29 checks passed

rmarinho deleted the appium branch May 16, 2023 21:34

NickDarvey mentioned this pull request Jun 6, 2023

.NET MAUI UI Testing Strategy #11236

Open

3 tasks

github-actions bot locked and limited conversation to collaborators Dec 12, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[uitests] Work for get appium uitests with CI #14488

[uitests] Work for get appium uitests with CI #14488

rmarinho commented Apr 10, 2023 •

edited by samhouts

Loading

PureWeen May 5, 2023 •

edited

Loading

sbanni May 8, 2023

github-actions bot commented May 16, 2023

rmarinho commented May 16, 2023

[uitests] Work for get appium uitests with CI #14488

[uitests] Work for get appium uitests with CI #14488

Conversation

rmarinho commented Apr 10, 2023 • edited by samhouts Loading

Description of Change

Running from the command line (you can replace ./build.ps1 by dotnet cake)

Running from the IDE

PureWeen May 5, 2023 • edited Loading

Choose a reason for hiding this comment

sbanni May 8, 2023

Choose a reason for hiding this comment

github-actions bot commented May 16, 2023

rmarinho commented May 16, 2023

rmarinho commented Apr 10, 2023 •

edited by samhouts

Loading

Running from the command line (you can replace ./build.ps1 by `dotnet cake`)

PureWeen May 5, 2023 •

edited

Loading