clicking buttons without a video input

[hmcgowan]

HI Jarmo,

I'm seeing an issue when using rautomation where the automation fails to work properly when there is no monitor or video output (for example, remote desktop into a machine). I'm using automation to close javascript dialogs (via watir) and the script times out in Control#click, wait_until if there is no video output attached at the time. This is happening when it's trying to activate and set focus to the windows. If you are watching it by logging into the machine or you are not locked out of your screen then it runs fine. So it only fails when no-one is watching :). This is a problem, however when you're trying to run against a set of test machines using Jenkins, eg.

I didn't submit a pull request for this because the fix seems like a big change and I'm not sure of the implications or what else might be affected. Also I was not able to figure out how your unit tests needed to be set up - pointing to notepad didn't seem to work for me and I didn't want to install .net to run the bundled app :)....in any case, below is a working solution and steps to reproduce.

You can recreate using the following:

(1) Open IE with this HTML

(2) Run a script that continually opens/closes dialogs (eg from the rautomation/spec dir)

$LOAD_PATH.unshift(File.join(File.dirname(FILE), '..', 'lib'))
require 'rautomation'
require 'watir/ie'

ie = Watir::IE.attach :url, /test/
i = 0
loop do
ie.button(:index,1).click_no_wait
ie.javascript_dialog.button("OK").click
i += 1
puts i
end

(3) Ctrl+Alt+Del while the script is running and wait a bit. You'll notice the ding of the popup stops and if you get out of this screen, the script starts up again. If you wait long enough the script times out.

I was able to get things working by removing the activate and focus calls in Control#click. After this change the script doesn't stall when you disconnect the video. Seems like the activate/focus calls won't work without an active video out.

Work around Jenkins issue where dialogs not clicked

module RAutomation
module Adapter
module WinFfi
class Control
def click
assert_enabled
clicked = false
wait_until do
hwnd = Functions.control_hwnd(@window.hwnd, @locators)

        if hwnd
          Functions.control_click(hwnd) &&
          clicked = true # is clicked at least once
        end

        block_given? ? yield : clicked && !exist?
      end
    end
  end
end

end
end

[jarmo]

Strange, but i cannot see the code in this issue list. I can see it in my e-mail inbox though. I wanted to ask why is that if statement needed there? Does it blow up otherwise if hwnd is not available?

Another question is - as i understand then you're using that solution now everyday. Does it work well enough? I remember that i had to add these focus statements there to make it more robust. I also knew that it wasn't working when machine is locked etc, but i didn't know that it's related with that thing.

I also know that i cannot take screenshots (with my win32screenshot gem) if the machine is locked and have solved the problem by using VNC and not locking the system.

[hmcgowan]

The if statement is there because It blows up :). I think there is a timing issue where postmessage is still working on the click so the window is still active at the bottom of the loop but is gone by the time you request the hwnd.

It seems to work pretty well. All of the tests previously failing are running rock solid now. Also i looked at the old winclicker code to see what it did (because it worked fine in this scenario) and it only sent the click

I think we should probably do something here but i think it's your call on how to solve it :). You know the rautomation code and usage better than I do and maybe this works great for js dialogs but not other kinds of windows. Seems like the choices are monkeypatch watir, make a new method in rautomation to handle this or change the existing method. Making a new method seems to me to have the least chance of breaking existing users of rautomation. Let me know what you'd like to do.

Thanks!
Hugh

Sent from my iPad

On Oct 8, 2011, at 2:57 AM, Jarmo Pertman reply@reply.github.com wrote:

Strange, but i cannot see the code in this issue list. I can see it in my e-mail inbox though. I wanted to ask why is that if statement needed there? Does it blow up otherwise if hwnd is not available?

Another question is - as i understand then you're using that solution now everyday. Does it work well enough? I remember that i had to add these focus statements there to make it more robust. I also knew that it wasn't working when machine is locked etc, but i didn't know that it's related with that thing.

I also know that i cannot take screenshots (with my win32screenshot gem) if the machine is locked and have solved the problem by using VNC and not locking the system.

Reply to this email directly or view it on GitHub:
#28 (comment)

[hmcgowan]

By the way, I'm happy to do the work on implementing this change - I just want to make sure we don't mess up rautomation for non-watir folks :). There's no rush, but it would be good to eventually get a fix for the watir code.

Hugh

---

From: Hugh McGowan hugh_mcgowan@yahoo.com
To: Jarmo Pertman reply@reply.github.com
Sent: Saturday, October 8, 2011 9:18 AM
Subject: Re: [RAutomation] clicking buttons without a video input (#28)

The if statement is there because It blows up :). I think there is a timing issue where postmessage is still working on the click so the window is still active at the bottom of the loop but is gone by the time you request the hwnd.

It seems to work pretty well. All of the tests previously failing are running rock solid now. Also i looked at the old winclicker code to see what it did (because it worked fine in this scenario) and it only sent the click

I think we should probably do something here but i think it's your call on how to solve it :). You know the rautomation code and usage better than I do and maybe this works great for js dialogs but not other kinds of windows. Seems like the choices are monkeypatch watir, make a new method in rautomation to handle this or change the existing method. Making a new method seems to me to have the least chance of breaking existing users of rautomation. Let me know what you'd like to do.

Thanks!
Hugh

Sent from my iPad

On Oct 8, 2011, at 2:57 AM, Jarmo Pertman reply@reply.github.com wrote:

Strange, but i cannot see the code in this issue list. I can see it in my e-mail inbox though. I wanted to ask why is that if statement needed there? Does it blow up otherwise if hwnd is not available?

Another question is - as i understand then you're using that solution now everyday. Does it work well enough? I remember that i had to add these focus statements there to make it more robust. I also knew that it wasn't working when machine is locked etc, but i didn't know that it's related with that thing.

I also know that i cannot take screenshots (with my win32screenshot gem) if the machine is locked and have solved the problem by using VNC and not locking the system.

Reply to this email directly or view it on GitHub:
#28 (comment)

[jarmo]

I don't know if that messes anything up or not. What happens if you try to take the focus away if RAutomation tries to control dialogs - e.g. set text or click on buttons? Have you tried by running RAutomation script and then clicking manually on some other window. Does it fail?

Also, this solution solves only #click... what about #activate, #send_keys and such?

[jarmo]

Still can't see the code in this issue. Can you repost the code here so i could look into it more?

[hmcgowan]

This is the monkey-patch we use to work around the issue:

https://github.com/convio/watirmark/blob/2a788873c0a77e7c7620e83df83ff184b5bc8328/lib/watirmark/extensions/rautomation.rb

[jarmo]

Closing this due to inactivity of 5+ years :) What a nice old-time project it is.