Non interactable elements on remote WinAppDriver

[mreiche]

Setup

• I followed the instruction for remote maschines: https://github.com/microsoft/WinAppDriver/blob/master/Docs/RunningOnRemoteMachine.md
• additionally: I setup autologon for an admistrative user via https://docs.microsoft.com/en-us/sysinternals/downloads/autologon
• WinAppDriver 1.2.1

Current behaviour

• A button is clickable on remote WinAppDriver with active RDP session (I'm watching)
• A button is not clickable on remote WinAppDriver without active RDP session (nobody is watching)

Steps to reproduce

• Step 1: test with active RDP session

// Step 1.1: Find the element
WebElement startBtn = windowsDriver.findElement(By.xpath("//Button[@Name=\"Start\"]"));

// Step 1.2: Check if element is displayed
Assert.assertTrue(startBtn.isDisplayed());

// Step 1.3: Click the element WITH ACTIVE RDP session
startBtn.click();

• WinAppDriver log

POST /session/9E59A52E-369F-43B9-8C6B-B0C8A0EA83B9/element/42.1311784.4.-2147483647/click HTTP/1.1
Accept-Encoding: gzip
Connection: Keep-Alive
Content-Length: 38
Content-Type: application/json; charset=utf-8
Host: 192.168.58.43:4723
User-Agent: selenium/3.141.59 (java windows)

{
  "id": "42.1311784.4.-2147483647"
}
HTTP/1.1 200 OK
Content-Length: 63
Content-Type: application/json

{"sessionId":"9E59A52E-369F-43B9-8C6B-B0C8A0EA83B9","status":0}

• Step 2: Close RDP session and try again

// Step 2.1: Element is still displayed
Assert.assertTrue(startBtn.isDisplayed());

// Step 2.2: But not clickable anymore
startBtn.click();
// Throws: Method threw 'org.openqa.selenium.WebDriverException' exception.

• WinAppDriver log:

POST /session/9E59A52E-369F-43B9-8C6B-B0C8A0EA83B9/element/42.1311784.4.-2147483647/click HTTP/1.1
Accept-Encoding: gzip
Connection: Keep-Alive
Content-Length: 38
Content-Type: application/json; charset=utf-8
Host: 192.168.58.43:4723
User-Agent: selenium/3.141.59 (java windows)

{
  "id": "42.1311784.4.-2147483647"
}
HTTP/1.1 500 Internal Error
Content-Length: 133
Content-Type: application/json

{"status":13,"value":{"error":"unknown error","message":"An unknown error occurred in the remote end while processing the command."}}

• Step 3: Open RDP session again and try again -> Works

Expected behaviour

• The element should be clickable, even without active RDP session

[shoaibmansoor]

Hi @mreiche You need a VNC server running on the VM to be able to access elements when RDP is minimized. I am following these steps to workaround the issue of RDP disconnection/minimized:

**Manual installation of desktop automation tools**

1. Install choco from https://chocolatey.org/docs/installation
2. Install nodejs (skip if already installed): choco install nodejs-lts
3. Install appium from admin terminal: npm i -g appium@1.17.1
4. Add a bat file to be called at startup:
Press WINDOWS+R and then type shell:startup. Create appium-starter.bat in the newly opened folder, and add following command:
a. When AUT doesn’t need admin rights
appium	

OR

b. When AUT needs to be started with admin rights(Be aware of machine password. It should be a static password)
runas /savecred /profile /user:Administrator C:\Users\%USERNAME%\AppData\Roaming\npm\appium.cmd
It will ask for the admin password once and store it for later uses

5. Install ultra vnc from https://filehippo.com/download_ultravnc/
Check the following checkboxes:
a. UltraVNC Server Silent
b. Register UltraVNC Server as a system service
c. Start or restart UltraVNC service
d. Click Next, Next and Finish
e. Open Ultra VNC settings
f. Under Security tab, Check mark Require MS Logon

6. Install AutoLogOn from autologon and enable it.
7. Install WinAppRecorder from UIR-v1.1 and add a shortcut to desktop to access this application(Needed for inspecting elements)
8. Install windows-sdk: choco install windows-sdk-10.0 -y (Needed to have inspect.exe)
9. Enable developer mode following: https://www.howtogeek.com/292914/what-is-developer-mode-in-windows-10/#:~:text=How%20to%20Enable%20Developer%20Mode,be%20put%20into%20Developer%20Mode.
10 Execute command from admin terminal: npm i -g set-vnc-server-resolution && set-vnc-server-resolution
Restart the machine and it's done

[Tree55Topz]

You need to allow GUI access when minimized. You can adjust these settings through the Registry Editor. Make sure to update the HOST machine. So whatever machine is hosting the VMs.

Here is an article that tells you exactly how to do it in very simple terms - this worked for my team. We currently run 6 parallel, all of them are hosted by one machine which has these settings.

https://support.smartbear.com/testcomplete/docs/testing-with/running/via-rdp/in-minimized-window.html

[mreiche]

You need to allow GUI access when minimized. You can adjust these settings through the Registry Editor. Make sure to update the HOST machine. So whatever machine is hosting the VMs.

Unfortunately, I cannot change the VM's HOST so even when this would be my preferend solution, I cannot apply it.

Hi @mreiche You need a VNC server running on the VM to be able to access elements when RDP is minimized. I am following these steps to workaround the issue of RDP disconnection/minimized:

I'm using the UltraVNC solution now, which is not that comfortable like Windows RDP, but it works.

Thank you guys very much. I appreciate a lot.