This repository has been archived by the owner. It is now read-only.

vnc.click() , clicks on the desktop rather than clicking on VNC session window. #206

Closed
sayantan94 opened this Issue May 23, 2016 · 11 comments

Comments

2 participants
@sayantan94

sayantan94 commented May 23, 2016

VNCScreen vnc = new VNCScreen();
vnc.find("image.png").highlight(4).click();
vnc.click(new Pattern("image.png"));

highlight() will show the perfect location of the image, but instead of clicking on the image in vnc window, Sikulix clicks on the active window itself.

@RaiMan

This comment has been minimized.

Show comment
Hide comment
@RaiMan

RaiMan May 23, 2016

Owner

working on it

Owner

RaiMan commented May 23, 2016

working on it

@RaiMan

This comment has been minimized.

Show comment
Hide comment
@RaiMan

RaiMan May 23, 2016

Owner

I think I finally fixed it now (being on the VNCScreen was not propagated to the resulting matches, hence the DeskTopRobot was still used).
Since I cannot test it live (no VNC setup available), I would highly appreciate for some testing and feedback from your side. thanks.
Be aware:

  • findAll() matches might still not work
  • use VNCScreen.newRegion()/newLocation(), to force the resulting objects to the remote screen.
  • the features, that create new Regions/Locations from existing Regions/Locations should propagate the "remote knowledge" (like Region.above(), Location.offset(), ...)
  • using Region/Location.isOtherScreen() shows true for elements on the remote screen
  • in doubt use print/toString() on an object, the screen now shows VNC for a remote
vnc.toString(); // VNC(0)[0,0 1024x768]
vnc.find(img).toString() // M[11,64 130x28]@S(VNC(0)[0,0 1024x768]) S:1,00 C:76,78 [177 msec]
// a match on the desktop: M[11,64 130x28]@S(S(0)[0,0 1024x768]) S:1,00 C:76,78 [177 msec]

example:

Iterator<Match> matches = vnc.findAll(someImage);
while (matches.hasNext()) {
    Match m = matches.next();
    m.hover(); // might still be on desktop
    vnc.newRegion(m).hover(); // should definitely happen on the remote
}

I will trigger a new nightly build later.

Owner

RaiMan commented May 23, 2016

I think I finally fixed it now (being on the VNCScreen was not propagated to the resulting matches, hence the DeskTopRobot was still used).
Since I cannot test it live (no VNC setup available), I would highly appreciate for some testing and feedback from your side. thanks.
Be aware:

  • findAll() matches might still not work
  • use VNCScreen.newRegion()/newLocation(), to force the resulting objects to the remote screen.
  • the features, that create new Regions/Locations from existing Regions/Locations should propagate the "remote knowledge" (like Region.above(), Location.offset(), ...)
  • using Region/Location.isOtherScreen() shows true for elements on the remote screen
  • in doubt use print/toString() on an object, the screen now shows VNC for a remote
vnc.toString(); // VNC(0)[0,0 1024x768]
vnc.find(img).toString() // M[11,64 130x28]@S(VNC(0)[0,0 1024x768]) S:1,00 C:76,78 [177 msec]
// a match on the desktop: M[11,64 130x28]@S(S(0)[0,0 1024x768]) S:1,00 C:76,78 [177 msec]

example:

Iterator<Match> matches = vnc.findAll(someImage);
while (matches.hasNext()) {
    Match m = matches.next();
    m.hover(); // might still be on desktop
    vnc.newRegion(m).hover(); // should definitely happen on the remote
}

I will trigger a new nightly build later.

@RaiMan RaiMan added the bug label May 23, 2016

@sayantan94

This comment has been minimized.

Show comment
Hide comment
@sayantan94

sayantan94 May 24, 2016

SO, every time i run a code to make a click on the VNC the following log comes up:

Connected[info] runcmd: lsb_release -i -r -s
M[264,155 417x192]@s(VNC(0)[0,0 992x494]) S:0.89 C:472,251 [457 msec]
[log] CLICK on L(472,251)@vnc(0)[0,0 992x494](1152 msec)

The log files shows a click, but in real their isn't any click in the VNC setup.

sayantan94 commented May 24, 2016

SO, every time i run a code to make a click on the VNC the following log comes up:

Connected[info] runcmd: lsb_release -i -r -s
M[264,155 417x192]@s(VNC(0)[0,0 992x494]) S:0.89 C:472,251 [457 msec]
[log] CLICK on L(472,251)@vnc(0)[0,0 992x494](1152 msec)

The log files shows a click, but in real their isn't any click in the VNC setup.

@RaiMan

This comment has been minimized.

Show comment
Hide comment
@RaiMan

RaiMan May 24, 2016

Owner

Does the mouse move to the location on the VNC screen?

Owner

RaiMan commented May 24, 2016

Does the mouse move to the location on the VNC screen?

@sayantan94

This comment has been minimized.

Show comment
Hide comment
@sayantan94

sayantan94 May 24, 2016

NO,it doesn't .

sayantan94 commented May 24, 2016

NO,it doesn't .

@RaiMan

This comment has been minimized.

Show comment
Hide comment
@RaiMan

RaiMan May 24, 2016

Owner

... and now the desktop mouse should not move anymore in this case. Correct?

Owner

RaiMan commented May 24, 2016

... and now the desktop mouse should not move anymore in this case. Correct?

@sayantan94

This comment has been minimized.

Show comment
Hide comment
@sayantan94

sayantan94 commented May 24, 2016

yes...

@RaiMan

This comment has been minimized.

Show comment
Hide comment
@RaiMan

RaiMan May 24, 2016

Owner

ok, then this is some problem in the VNCScreen/VNCRobot area or with VNC itself.

I do not have the time currently and no ressources (no VNC setup) to test this in depth.
For now I have to leave you alone with it.

But if you have any new information or suggestions: welcome.

Owner

RaiMan commented May 24, 2016

ok, then this is some problem in the VNCScreen/VNCRobot area or with VNC itself.

I do not have the time currently and no ressources (no VNC setup) to test this in depth.
For now I have to leave you alone with it.

But if you have any new information or suggestions: welcome.

@sayantan94

This comment has been minimized.

Show comment
Hide comment
@sayantan94

sayantan94 May 24, 2016

OK, i will look into it, and let you know if i succeed. :)

sayantan94 commented May 24, 2016

OK, i will look into it, and let you know if i succeed. :)

@RaiMan

This comment has been minimized.

Show comment
Hide comment
@RaiMan

RaiMan May 24, 2016

Owner

ok, thanks for your cooperation.

Owner

RaiMan commented May 24, 2016

ok, thanks for your cooperation.

@RaiMan

This comment has been minimized.

Show comment
Hide comment
@RaiMan

RaiMan May 28, 2016

Owner

For now the VNC handling has got some convenience wrapper (scripting: vncStart(), Java: VNCScreen.start()) to start one ore more VNCScreen sessions. At end of scriptrun or JVM exit all running VNC sessions are auto-stopped. This is tested from Mac towards Windows TightVNC server and as loopback on Windows.
see:
http://sikulix-2014.readthedocs.io/en/latest/screen.html#connecting-to-a-vnc-server-vncscreen

Owner

RaiMan commented May 28, 2016

For now the VNC handling has got some convenience wrapper (scripting: vncStart(), Java: VNCScreen.start()) to start one ore more VNCScreen sessions. At end of scriptrun or JVM exit all running VNC sessions are auto-stopped. This is tested from Mac towards Windows TightVNC server and as loopback on Windows.
see:
http://sikulix-2014.readthedocs.io/en/latest/screen.html#connecting-to-a-vnc-server-vncscreen

@RaiMan RaiMan closed this May 28, 2016

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.