vnc.click() , clicks on the desktop rather than clicking on VNC session window. #206

Closed
sayantan94 opened this Issue May 23, 2016 · 11 comments

Projects

None yet

2 participants

@sayantan94

VNCScreen vnc = new VNCScreen();
vnc.find("image.png").highlight(4).click();
vnc.click(new Pattern("image.png"));

highlight() will show the perfect location of the image, but instead of clicking on the image in vnc window, Sikulix clicks on the active window itself.

@RaiMan
Owner
RaiMan commented May 23, 2016

working on it

@RaiMan
Owner
RaiMan commented May 23, 2016

I think I finally fixed it now (being on the VNCScreen was not propagated to the resulting matches, hence the DeskTopRobot was still used).
Since I cannot test it live (no VNC setup available), I would highly appreciate for some testing and feedback from your side. thanks.
Be aware:

  • findAll() matches might still not work
  • use VNCScreen.newRegion()/newLocation(), to force the resulting objects to the remote screen.
  • the features, that create new Regions/Locations from existing Regions/Locations should propagate the "remote knowledge" (like Region.above(), Location.offset(), ...)
  • using Region/Location.isOtherScreen() shows true for elements on the remote screen
  • in doubt use print/toString() on an object, the screen now shows VNC for a remote
vnc.toString(); // VNC(0)[0,0 1024x768]
vnc.find(img).toString() // M[11,64 130x28]@S(VNC(0)[0,0 1024x768]) S:1,00 C:76,78 [177 msec]
// a match on the desktop: M[11,64 130x28]@S(S(0)[0,0 1024x768]) S:1,00 C:76,78 [177 msec]

example:

Iterator<Match> matches = vnc.findAll(someImage);
while (matches.hasNext()) {
    Match m = matches.next();
    m.hover(); // might still be on desktop
    vnc.newRegion(m).hover(); // should definitely happen on the remote
}

I will trigger a new nightly build later.

@RaiMan RaiMan added the bug label May 23, 2016
@sayantan94

SO, every time i run a code to make a click on the VNC the following log comes up:

Connected[info] runcmd: lsb_release -i -r -s
M[264,155 417x192]@S(VNC(0)[0,0 992x494]) S:0.89 C:472,251 [457 msec]
[log] CLICK on L(472,251)@VNC(0)[0,0 992x494](1152 msec)

The log files shows a click, but in real their isn't any click in the VNC setup.

@RaiMan
Owner
RaiMan commented May 24, 2016

Does the mouse move to the location on the VNC screen?

@sayantan94

NO,it doesn't .

@RaiMan
Owner
RaiMan commented May 24, 2016

... and now the desktop mouse should not move anymore in this case. Correct?

@sayantan94

yes...

@RaiMan
Owner
RaiMan commented May 24, 2016

ok, then this is some problem in the VNCScreen/VNCRobot area or with VNC itself.

I do not have the time currently and no ressources (no VNC setup) to test this in depth.
For now I have to leave you alone with it.

But if you have any new information or suggestions: welcome.

@sayantan94

OK, i will look into it, and let you know if i succeed. :)

@RaiMan
Owner
RaiMan commented May 24, 2016

ok, thanks for your cooperation.

@RaiMan
Owner
RaiMan commented May 28, 2016

For now the VNC handling has got some convenience wrapper (scripting: vncStart(), Java: VNCScreen.start()) to start one ore more VNCScreen sessions. At end of scriptrun or JVM exit all running VNC sessions are auto-stopped. This is tested from Mac towards Windows TightVNC server and as loopback on Windows.
see:
http://sikulix-2014.readthedocs.io/en/latest/screen.html#connecting-to-a-vnc-server-vncscreen

@RaiMan RaiMan closed this May 28, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment