Unmapped response

[clay227]

client.getDesktop(); retruns null values.
fallowed the example given on the page and could not make work, at the cleint side i get:

using ScreenRegion screenRegion = desktop.find(imageTarget);

49778 [PoolService-0] ERROR i.s.r.p.c.RemoteObjectMethodInterceptor - Unmapped response
java.lang.IllegalArgumentException: class sun.awt.image.ByteInterleavedRaster declares multiple JSON fields named maxX
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory.getBoundFields(ReflectiveTypeAdapterFactory.java:146) ~[gson-2.3.1.jar:na]
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory.create(ReflectiveTypeAdapterFactory.java:83) ~[gson-2.3.1.jar:na]
at com.google.gson.Gson.getAdapter(Gson.java:359) ~[gson-2.3.1.jar:na]
at com.google.gson.internal.bind.TypeAdapterRuntimeTypeWrapper.write(TypeAdapterRuntimeTypeWrapper.java:55) ~[gson-2.3.1.jar:na]
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.write(ReflectiveTypeAdapterFactory.java:99) ~[gson-2.3.1.jar:na]
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.write(ReflectiveTypeAdapterFactory.java:219) ~[gson-2.3.1.jar:na]
at com.google.gson.internal.bind.TypeAdapterRuntimeTypeWrapper.write(TypeAdapterRuntimeTypeWrapper.java:68) ~[gson-2.3.1.jar:na]
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.write(ReflectiveTypeAdapterFactory.java:99) ~[gson-2.3.1.jar:na]
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.write(ReflectiveTypeAdapterFactory.java:219) ~[gson-2.3.1.jar:na]
at com.google.gson.Gson.toJson(Gson.java:600) ~[gson-2.3.1.jar:na]
at com.google.gson.Gson.toJson(Gson.java:579) ~[gson-2.3.1.jar:na]
at com.google.gson.Gson.toJson(Gson.java:534) ~[gson-2.3.1.jar:na]
at com.google.gson.Gson.toJson(Gson.java:514) ~[gson-2.3.1.jar:na]
at io.sterodium.rmi.protocol.client.RemoteInvoker.toString(RemoteInvoker.java:67) ~[sterodium-rmi-0.1.jar:na]
at io.sterodium.rmi.protocol.client.RemoteInvoker.getMethodInvocation(RemoteInvoker.java:54) ~[sterodium-rmi-0.1.jar:na]
at io.sterodium.rmi.protocol.client.RemoteInvoker.invoke(RemoteInvoker.java:32) ~[sterodium-rmi-0.1.jar:na]
at io.sterodium.rmi.protocol.client.RemoteObjectMethodInterceptor.invoke(RemoteObjectMethodInterceptor.java:58) [sterodium-rmi-0.1.jar:na]
at io.sterodium.rmi.protocol.client.RemoteObjectMethodInterceptor.intercept(RemoteObjectMethodInterceptor.java:40) [sterodium-rmi-0.1.jar:na]
at org.sikuli.api.DesktopScreenRegion$$EnhancerByCGLIB$$c989fea2.find() [cglib-3.1.jar:na]
at com.nexperience.system.uitests.SikuliTestPoc.testScenario(SikuliTestPoc.java:76) [test-classes/:na]
at com.nexperience.system.uitests.BaseTest.scenarioFlow(BaseTest.java:239) [test-classes/:na]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.7.0_10]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) ~[na:1.7.0_10]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.7.0_10]
at java.lang.reflect.Method.invoke(Method.java:601) ~[na:1.7.0_10]
at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:84) [selenium-server-standalone-2.45.0.jar:na]
at org.testng.internal.Invoker.invokeMethod(Invoker.java:714) [selenium-server-standalone-2.45.0.jar:na]
at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:901) [selenium-server-standalone-2.45.0.jar:na]
at org.testng.internal.TestMethodWithDataProviderMethodWorker.call(TestMethodWithDataProviderMethodWorker.java:73) [selenium-server-standalone-2.45.0.jar:na]
at org.testng.internal.TestMethodWithDataProviderMethodWorker.call(TestMethodWithDataProviderMethodWorker.java:14) [selenium-server-standalone-2.45.0.jar:na]
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [na:1.7.0_10]
at java.util.concurrent.FutureTask.run(FutureTask.java:166) [na:1.7.0_10]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [na:1.7.0_10]
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [na:1.7.0_10]
at java.util.concurrent.FutureTask.run(FutureTask.java:166) [na:1.7.0_10]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) [na:1.7.0_10]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) [na:1.7.0_10]
at java.lang.Thread.run(Thread.java:722) [na:1.7.0_10]

[echoAlexey]

Hi @clay227 ,
What version of client do you use?
I see sterodium-rmi-0.1.jar in stacktrace.
Latest release 0.5 uses sterodium-rmi 0.2.

You can find examples of Sikuli tests here - https://github.com/echoAlexey/selenium-examples

We will add example project under Sterodium organisation shortly together with extended Selenium Grid launcher.

[clay227]

Hello echoAlexey thank you for the replay,
at the cleint side i set the maven maven repository to 0.5
at node side i am using all-node-extensions 0.5 jar

[echoAlexey]

Do you use extension-proxy on hub? https://github.com/sterodium/selenium-grid-extensions/wiki/Hub-Configuration

Same 0.5 version.

[clay227]

yes 0.5
note that the upload file part is working, i see it on the node machine file system.
due to vpn issues at my office , i will be able upload all configuration file the day after tommarow.11.11
thanks for now.

[clay227]

hello ,
i check the appdata folder and indeed sterodium-rmi was uisng 01. so i changed my pom like soL with be like so:

<dependencyManagement>
 <dependencies>
    <dependency>
        <groupId>io.sterodium</groupId>
        <artifactId>sterodium-rmi</artifactId>
        <version>0.2</version>
    </dependency>
  </dependencies>
 </dependencyManagement>
<dependencies>

<dependency>
    <groupId>com.sikulix</groupId>
    <artifactId>sikulixapi</artifactId>
    <version>1.1.1-SNAPSHOT</version>
</dependency>
<dependency>
    <groupId>io.sterodium</groupId>
    <artifactId>sikuli-extension-client</artifactId>
    <version>0.5</version>
</dependency>
<dependency>
    <groupId>io.sterodium</groupId>
    <artifactId>file-extension-client</artifactId>
    <version>0.5</version>
</dependency>

This make the sterodium-rmi 02 version work.

however i now get another exception, maybe i should add another dependencyManagement ?
Thanks in advance :)

here is the exception:

7507 [PoolService-0] INFO i.s.r.p.c.RemoteObjectMethodInterceptor - Converting response 'System' to type class java.lang.String or java.lang.Stri
ava.lang.IllegalArgumentException: class sun.awt.image.ByteInterleavedRaster declares multiple JSON fields named maxX
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory.getBoundFields(ReflectiveTypeAdapterFactory.java:146)
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory.create(ReflectiveTypeAdapterFactory.java:83)
at com.google.gson.Gson.getAdapter(Gson.java:359)
at com.google.gson.internal.bind.TypeAdapterRuntimeTypeWrapper.write(TypeAdapterRuntimeTypeWrapper.java:55)
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.write(ReflectiveTypeAdapterFactory.java:99)
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.write(ReflectiveTypeAdapterFactory.java:219)
at com.google.gson.internal.bind.TypeAdapterRuntimeTypeWrapper.write(TypeAdapterRuntimeTypeWrapper.java:68)
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.write(ReflectiveTypeAdapterFactory.java:99)
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.write(ReflectiveTypeAdapterFactory.java:219)
at com.google.gson.Gson.toJson(Gson.java:600)
at com.google.gson.Gson.toJson(Gson.java:579)
at com.google.gson.Gson.toJson(Gson.java:534)
at com.google.gson.Gson.toJson(Gson.java:514)
at io.sterodium.rmi.protocol.client.RemoteInvoker.toString(RemoteInvoker.java:67)
at io.sterodium.rmi.protocol.client.RemoteInvoker.getMethodInvocation(RemoteInvoker.java:54)
at io.sterodium.rmi.protocol.client.RemoteInvoker.invoke(RemoteInvoker.java:32)
at io.sterodium.rmi.protocol.client.RemoteObjectMethodInterceptor.invoke(RemoteObjectMethodInterceptor.java:58)
at io.sterodium.rmi.protocol.client.RemoteObjectMethodInterceptor.intercept(RemoteObjectMethodInterceptor.java:40)
at org.sikuli.api.DesktopScreenRegion$$EnhancerByCGLIB$$36669985.find()
at com.nexperience.system.uitests.SikuliTestPoc.testScenario(SikuliTestPoc.java:76)
at com.nexperience.system.uitests.BaseTest.scenarioFlow(BaseTest.java:239)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:84)
at org.testng.internal.Invoker.invokeMethod(Invoker.java:714)
at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:901)
at org.testng.internal.TestMethodWithDataProviderMethodWorker.call(TestMethodWithDataProviderMethodWorker.java:73)
at org.testng.internal.TestMethodWithDataProviderMethodWorker.call(TestMethodWithDataProviderMethodWorker.java:14)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)

[clay227]

As before the DesktopScreenRegion parameter value seem to be null after getting it's value from client.getDesktop()
{screen=null, x=0, y=0, width=0, height=0, score=0.0}

the node seems to get the request since i see in its logs:
"INFO - *** Calling capture on org.sikuli.api.DesktopScreenRegion"

working with the mouse seems to work well, but nothing that requires DesktopScreenRegion can work.

[echoAlexey]

First, we don't support this version of sikuli.

<dependency>
    <groupId>com.sikulix</groupId>
    <artifactId>sikulixapi</artifactId>
    <version>1.1.1-SNAPSHOT</version>
</dependency>

Node will have

<dependency>
    <groupId>org.sikuli</groupId>
    <artifactId>sikuli-api</artifactId>
    <version>1.2.0</version>
</dependency>

sikuli dependency. On a client it is recommended to use same version, because client API is proxied and method calls are sent via REST to node and then executed on the same version of library.

Can you please try your grid setup against sample tests from example project: https://github.com/echoAlexey/selenium-examples

[clay227]

thanks echoAlexey,
tried your to work with your example (only small changes to work on my selenium grid) and still the same problem.
again the screenRegion seems to be null {screen=null, x=0, y=0, width=0, height=0, score=0.0}
and the same exception.
i even deleted the appdata and made sure the 1.2.0 version is used.

always get :
INFO i.s.r.p.c.RemoteObjectMethodInterceptor - Response: MethodInvocationResultDto{value='null', type='null'}
java.lang.IllegalArgumentException: class sun.awt.image.ByteInterleavedRaster declares multiple JSON fields named maxX
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory.getBoundFields(ReflectiveTypeAdapterFactory.java:146)

[clay227]

also note that the line 45 in SikuliHelper :
" ImageTarget imageTarget = targetFactory.createImageTarget(resource);"
will cause another exception that can be workaround by replacing it with
"ImageTarget imageTarget = new ImageTarget(new File("Capture1.png"));"

14747 [PoolService-0] INFO i.s.r.p.c.RemoteObjectMethodInterceptor - Converting response 'Invocation Exception: java.lang.RuntimeException: Method io.sterodium.extensions.node.rmi.TargetFactory.createImageTarget threw exception' to type class org.sikuli.api.ImageTarget or java.lang.String
java.lang.IllegalArgumentException: Cannot subclass final class class java.lang.String
at net.sf.cglib.proxy.Enhancer.generateClass(Enhancer.java:446)
at net.sf.cglib.core.DefaultGeneratorStrategy.generate(DefaultGeneratorStrategy.java:25)
at net.sf.cglib.core.AbstractClassGenerator.create(AbstractClassGenerator.java:216)
at net.sf.cglib.proxy.Enhancer.createHelper(Enhancer.java:377)
at net.sf.cglib.proxy.Enhancer.createClass(Enhancer.java:317)
at io.sterodium.rmi.protocol.client.RemoteObjectProxyFactory.create(RemoteObjectProxyFactory.java:44)
at io.sterodium.rmi.protocol.client.RemoteObjectMethodInterceptor.convertToType(RemoteObjectMethodInterceptor.java:80)
at io.sterodium.rmi.protocol.client.RemoteObjectMethodInterceptor.invoke(RemoteObjectMethodInterceptor.java:62)
at io.sterodium.rmi.protocol.client.RemoteObjectMethodInterceptor.intercept(RemoteObjectMethodInterceptor.java:40)
at io.sterodium.extensions.node.rmi.TargetFactory$$EnhancerByCGLIB$$7f3f8ed.createImageTarget()
at com.nexperience.system.uitests.ui.SikuliHelper.waitForElement(SikuliHelper.java:45)
at com.nexperience.system.uitests.ui.SikuliHelper.find(SikuliHelper.java:39)
at com.nexperience.system.uitests.SikuliTestPoc.testScenario(SikuliTestPoc.java:60)
at com.nexperience.system.uitests.BaseTest.scenarioFlow(BaseTest.java:239)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:84)
at org.testng.internal.Invoker.invokeMethod(Invoker.java:714)
at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:901)
at org.testng.internal.TestMethodWithDataProviderMethodWorker.call(TestMethodWithDataProviderMethodWorker.java:73)
at org.testng.internal.TestMethodWithDataProviderMethodWorker.call(TestMethodWithDataProviderMethodWorker.java:14)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
15075 [PoolService-0] INFO c.n.system.uitests.BaseTest - end test flow on chrome-46.0.2490.86
Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 16.879 sec <<< FAILURE! - in com.nexperience.system.uitests.SikuliTestPoc
scenarioFlow(com.nexperience.system.uitests.SikuliTestPoc) Time elapsed: 16.279 sec <<< FAILURE!

[echoAlexey]

Please note that SikuliExtensionClient wraps Sikuli API and makes it remote. When you do it yourself - your classes are not proxied and cannot work remotely. Thus when you use sikuli api you should instantiate all classes with SikuliExtensionClient.
Its incorrectly to create new ImageTarget directly in the code, because this should be done remotely on the selenium grid node.

Correct usage is

SikuliExtensionClient sikuliExtensionClient = new SikuliExtensionClient(host, port, sessionId);
sikuliExtensionClient.uploadResourceBundle("folder-in-resources");
sikuliExtensionClient.getTargetFactory().createImageTarget("Capture1.png");

What happens here:

• You create client which will properly construct requests to sikuli extension.
• Upload resources to remote selenium node
• get target factory and create image target (on remote selenium node) and receive back proxy of ImageTarget with ID of real image target on grid.

Can you check logs of your selenium grid (hub and nodes) to determine what may go wrong when you call

ImageTarget imageTarget = targetFactory.createImageTarget(resource);

It seems that image target is not created remotely or some exception is thrown in the middle.

We are working on improving exception handling. =) Next release will have that.

[clay227]

Hello again,
Thanks !
I got it working by comparing to your examples.
The issue is that my resource folder was named "test" after changing it to "test.com" it's working.
any hint on how i could get sikuli to focuse on the browser would be also be helpful.

FYI when i use the "test" folder for images i got the this exception the node side:

11:52:54.083 WARN - Method io.sterodium.extensions.node.rmi.TargetFactory.create
ImageTarget threw exception
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at io.sterodium.rmi.protocol.server.MethodInvoker.invoke(MethodInvoker.j
ava:125)
at io.sterodium.rmi.protocol.server.MethodInvoker.invoke(MethodInvoker.j
ava:44)
at io.sterodium.rmi.protocol.server.RmiFacade.invoke(RmiFacade.java:34)
at io.sterodium.extensions.node.rmi.SikuliApplication.invoke(SikuliAppli
cation.java:40)
at io.sterodium.extensions.node.SikuliExtensionServlet.doPost(SikuliExte
nsionServlet.java:45)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at org.openqa.jetty.jetty.servlet.ServletHolder.handle(ServletHolder.jav
a:428)
at org.openqa.jetty.jetty.servlet.ServletHandler.dispatch(ServletHandler
.java:680)
at org.openqa.jetty.jetty.servlet.ServletHandler.handle(ServletHandler.j
ava:571)
at org.openqa.jetty.http.HttpContext.handle(HttpContext.java:1526)
at org.openqa.jetty.http.HttpContext.handle(HttpContext.java:1479)
at org.openqa.jetty.http.HttpServer.service(HttpServer.java:920)
at org.openqa.jetty.http.HttpConnection.service(HttpConnection.java:820)

    at org.openqa.jetty.http.HttpConnection.handleNext(HttpConnection.java:9

86. at org.openqa.jetty.http.HttpConnection.handle(HttpConnection.java:837)
    at org.openqa.jetty.http.SocketListener.handleConnection(SocketListener.
    

java:243)
at org.openqa.jetty.util.ThreadedServer.handle(ThreadedServer.java:358)
at org.openqa.jetty.util.ThreadPool$PoolThread.run(ThreadPool.java:537)
Caused by: java.lang.RuntimeException: Image file can not be loaded from Capture
1.png
at org.sikuli.api.ImageTarget.(ImageTarget.java:65)
at io.sterodium.extensions.node.rmi.TargetFactory.createImageTarget(Targ
etFactory.java:35)
... 23 more
java.lang.RuntimeException: Method io.sterodium.extensions.node.rmi.TargetFactor
y.createImageTarget threw exception
at io.sterodium.rmi.protocol.server.MethodInvoker.invoke(MethodInvoker.j
ava:133)
at io.sterodium.rmi.protocol.server.MethodInvoker.invoke(MethodInvoker.j
ava:44)
at io.sterodium.rmi.protocol.server.RmiFacade.invoke(RmiFacade.java:34)
at io.sterodium.extensions.node.rmi.SikuliApplication.invoke(SikuliAppli
cation.java:40)
at io.sterodium.extensions.node.SikuliExtensionServlet.doPost(SikuliExte
nsionServlet.java:45)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at org.openqa.jetty.jetty.servlet.ServletHolder.handle(ServletHolder.jav
a:428)
at org.openqa.jetty.jetty.servlet.ServletHandler.dispatch(ServletHandler
.java:680)
at org.openqa.jetty.jetty.servlet.ServletHandler.handle(ServletHandler.j
ava:571)
at org.openqa.jetty.http.HttpContext.handle(HttpContext.java:1526)
at org.openqa.jetty.http.HttpContext.handle(HttpContext.java:1479)
at org.openqa.jetty.http.HttpServer.service(HttpServer.java:920)
at org.openqa.jetty.http.HttpConnection.service(HttpConnection.java:820)

    at org.openqa.jetty.http.HttpConnection.handleNext(HttpConnection.java:9

86. at org.openqa.jetty.http.HttpConnection.handle(HttpConnection.java:837)
    at org.openqa.jetty.http.SocketListener.handleConnection(SocketListener.
    

java:243)
at org.openqa.jetty.util.ThreadedServer.handle(ThreadedServer.java:358)
at org.openqa.jetty.util.ThreadPool$PoolThread.run(ThreadPool.java:537)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at io.sterodium.rmi.protocol.server.MethodInvoker.invoke(MethodInvoker.j
ava:125)
... 18 more
Caused by: java.lang.RuntimeException: Image file can not be loaded from Capture
1.png
at org.sikuli.api.ImageTarget.(ImageTarget.java:65)
at io.sterodium.extensions.node.rmi.TargetFactory.createImageTarget(Targ
etFactory.java:35)
... 23 more
11:52:54.202 INFO - Executing: [take screenshot])

[volkovs]

@clay227 Hello, as @echoAlexey mentioned we will improve exception handling and log messages in next version.

any hint on how i could get sikuli to focuse on the browser would be also be helpful

Do you mean bringing browser window in front? Sikuli doesn't know about windows, it operates with visible screen areas. So, no test/work in parallel in the same desktop (there could be multiple desktops in single node). Switch browser tabs via Selenium API. It might be still possible to switch windows by sending platform specific key strokes via Sikuli.

Please let us know if you have blocking issues.