Run FlaUI tests on CI

[kaafer]

Hi.

UI tests for desktop application(not store app):
Used:
FlaUI.Core v2.0.0-alpha6
FlaUI.UIA3 v2.0.0-alpha6
SpecFlow v.2.3.2
NUnit 3.10.1
NUnit.ConsoleRunner 3.8.0
TeamCity Professional 2017.1.4 (build 47070)

On my local machine works perfectly. On Teamcity test stand pc (win server 2012) when run from console - work perfectly.

But the problem when run test with Teamcity on test stand pc.
Build steps:
get project from git

1. Restore NuGet Packages
2. Build solution
3. Run test with NUnit console runner.

Build is hangs on last step. On debug NUnit mode problem in last string here:
ProcessStartInfo process = new ProcessStartInfo(path);
var app = Application.AttachOrLaunch(process);
var automation = new UIA3Automation();
return app.GetMainWindow(automation);

app initialized success, automation too.
but when in GetWindow method this: _process.MainWindowHandle is hangs.

Any helps will be nice.

[kaafer]

Tryed to run as well your project's UI tests same way. But same hangs in same codeplace.

[BogdanLivadariu]

if the service started that runs the job does not has access to interact with the desktop ... you might get these sort of freezes.

try starting the teamcity instance as an user, and not as a service
and the rerun the tests

for what it worths I'm using flaui for an outlook addin, and the tests we have are running on gitlab-ci
and they run without any issue there
gitlab-ci requires a runner, and that runner is started from a windows box as an user, and not as a service
if we start it as a service we get all sort of issues, but when started as a user then everything start to work as expected

[maxinfet]

We are using TeamCity and Octopus Deploy at my work. Our solution was to run the UI automation as part of Octopus instead of TeamCity. We deploy our UI automation as part of Octopus because TeamCity has no desktop to interact with. The way we handled this is by setting up a VM with a VNC viewer on it. The VNC viewer keeps a desktop alive and the lock screen from coming up. Then TeamCity builds and tells Octopus to deploy. Then the Octopus Tentacle running from a cmd.exe the user launched (as part of a startup script for the user) runs the UI automation. As far as I know it is not possible to run UI automation from a windows service since no desktop will be available.

[Roemer]

Can you try to record what is happening? See https://github.com/Roemer/FlaUI/wiki/Capturing on how to capture videos. If it fails because you have no desktop, then you need to check the other solutions above. UI testing needs a desktop session.

[Roemer]

Is your problem solved? If not, can you check https://github.com/Roemer/FlaUI/wiki/FAQ#how-can-i-run-flaui-tests-on-a-build-serveragent ?

[jakubsuchybio]

Just ran into this problem...
Setup - Teamcity, xUnit, .NET Framework 4.7.1, FlaUI test framework

1. Agent need to be running as console app. To do this just run agent.bat start at startup (create shortcut to shell:startup with start argument) This will start an agent every time after startup in console.
2. Here Documentation how to run xunit 2.0 tests on Team city xunit/xunit#387 I found Teamcity metaruner https://github.com/4Com/team-city-xunit-meta-runner which is really good. BUT by default I had a problem which xunit.runner.console it chooses. By default it specifies all xunit.console.exe files from all directories there. So I had to specify the right framework version to run on. In that meta-runner build step Xunit Runner Executable I used it with net471\xunit.console.exe

And it works! :)

[kaafer]

Thanks everybody, it's solved.
Agent running as console app if in TeamCity.
No any tricks in TFS (also do not needed RDP or VNC).

[jakubsuchybio]

FYI it works.... UNTIL you need to have elevated privileges to run and find the Window via FlaUI on agent...
I spent yesterday figuring out why my ui test wouldn't find any control inside windows installer window and then it hit me... I might not be running with elevated privileges.

So then I spent this day to figure out how to run xUnit tests with elevated privileges on TeamCity and I got it working with http://www.westmesatech.com/misctools.html -> ElevationToolkit1 when I prepend Elevate64.exe before xUnit.Runner.Console.Exe then the UI test runs correctly and it find everyhing as when I run it manually...
However downside to this is, that when you elevate the process it spawns new process to do so and I haven't found a way to get the stdout and stderr out of that new process, so that it would show in TeamCity Build Log. It only shows, that it ran xUnit.Runner.Console.exe, but without any output...
I tried to mimic the Elevate64.exe with my own implementation that would run new process with elevated privileges AND with stdout and stderr redirected, but this is the same problem where i don't have stdout and stderr, and I don't know how to get out of it. :(

Do you guys have any ideas? I might post something about this to TeamCity forums, because I am lost now... :/

[OggyMishra]

if the service started that runs the job does not has access to interact with the desktop ... you might get these sort of freezes.

try starting the teamcity instance as an user, and not as a service
and the rerun the tests

for what it worths I'm using flaui for an outlook addin, and the tests we have are running on gitlab-ci
and they run without any issue there
gitlab-ci requires a runner, and that runner is started from a windows box as an user, and not as a service
if we start it as a service we get all sort of issues, but when started as a user then everything start to work as expected
@BogdanLivadariu what do you mean by starting the services as user, have you added any script in gitlab.yml file to do that. I am also trying to do something similar, in my UI automation I am trying to run the test cases from the gitlab-runner. but it's not able to pick any type of control.

[BogdanLivadariu]

@OggyMishra

try starting the gitlab-runner manually, after you have logged in manually with the Administrator account

just open a new CMD and start the runner

[OggyMishra]

@BogdanLivadariu Thanks it worked after ran gitlab-runner manually.

[Roemer]

Closing this as this seems fixed.