New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

android and ios treat touchaction coordinates differently (relative vs absolute) #7486

Closed
TikhomirovSergey opened this Issue Dec 9, 2016 · 6 comments

Comments

Projects
None yet
6 participants
@TikhomirovSergey
Member

TikhomirovSergey commented Dec 9, 2016

The problem

The problem is that the swiping on Android still consumes absolute values instead of relative (like iOS).
However the moveTo action consumes relative values on both Android and iOS. So it makes inconvenience on client side.

Also there is some weird behaviour.
When user tries to perform the swiping twise (may be more times) per one TouchAction then it fails with error that says that point is out of screen.

Environment

  • Appium version 1.6.0, 1.6.1 and older:
  • Desktop OS/version used to run Appium: I was trying it on Windows 7/8
  • Node.js version (unless using Appium.app|exe): doesn't matter
  • Mobile platform/version under test: Android
  • Real device or emulator/simulator: Emulator

Details

I think the solution is to make the swiping accept relative coordinate values or to stop support it.

Link to Appium logs

Create a gist which is a paste of your full Appium logs, and link them here. Do not paste your full Appium logs here, as it will make this issue very long and hard to read! If you are reporting a bug, always include Appium logs!

Code To Reproduce Issue [ Good To Have ]

The client code

driver.findElementByAccessibilityId("Views").click();

AndroidElement chronometer = driver.findElementByAccessibilityId("Chronometer");
Point point1 = chronometer.getCenter();
int x1 = point1.getX();
int y1 = point1.getY();

AndroidElement focus = driver.findElementByAccessibilityId("Focus");
Point point2 = focus.getCenter();
int x2 = point2.getX();
int y2 = point2.getY();

TouchAction touchAction = new TouchAction(driver);
touchAction.press(x2, y2).waitAction(2000).moveTo(x1, y1).release()//swipe 1
           .press(x1, y1).waitAction(2000).moveTo(x2, y2).release() //swipe 2, seems it needs for relative values
           .perform();

server logs

However this code works

        driver.findElementByAccessibilityId("Views").click();

        AndroidElement chronometer = driver.findElementByAccessibilityId("Chronometer");
        Point point1 = chronometer.getCenter();
        int x1 = point1.getX();
        int y1 = point1.getY();

        AndroidElement focus = driver.findElementByAccessibilityId("Focus");
        Point point2 = focus.getCenter();
        int x2 = point2.getX();
        int y2 = point2.getY();

        TouchAction touchAction1 = new TouchAction(driver);
        touchAction1.press(x2, y2).waitAction(2000).moveTo(x1, y1).release().perform();

        TouchAction touchAction2 = new TouchAction(driver);
        touchAction2.press(x1, y1).waitAction(2000).moveTo(x2, y2).release()
                .perform();

image

server log

@triager triager added the NeedsTriage label Dec 9, 2016

@SrinivasanTarget

This comment has been minimized.

Show comment
Hide comment
@SrinivasanTarget

SrinivasanTarget Dec 9, 2016

Member

@triager please label:Bug, Android

Member

SrinivasanTarget commented Dec 9, 2016

@triager please label:Bug, Android

@triager triager added Bug and removed NeedsTriage labels Dec 9, 2016

@SrinivasanTarget

This comment has been minimized.

Show comment
Hide comment
@SrinivasanTarget

SrinivasanTarget Dec 9, 2016

Member

@sravanmedarapu @imurchie Can you please look into this?

Member

SrinivasanTarget commented Dec 9, 2016

@sravanmedarapu @imurchie Can you please look into this?

@triager triager added the Android label Dec 9, 2016

@jlipps

This comment has been minimized.

Show comment
Hide comment
@jlipps

jlipps Dec 9, 2016

Member

android and ios should definitely follow the same semantics.

Member

jlipps commented Dec 9, 2016

android and ios should definitely follow the same semantics.

@saikrishna321

This comment has been minimized.

Show comment
Hide comment
@saikrishna321

saikrishna321 Mar 6, 2017

Member

@jlipps i see this issue on iOS side as well, with the below code it works fine - scroll down(legacy)
server logs are shows the right co-ordinates

Dimension dimensions = driver.manage().window().getSize();
Double screenHeightStart = dimensions.getHeight() * 0.5    
int scrollStart = screenHeightStart.intValue();
Double screenHeightEnd = dimensions.getHeight() * 0.30;
int scrollEnd = screenHeightEnd.intValue();
 driver.swipe(0, scrollStart, 0, scrollEnd, 1000);

[debug] [MJSONWP] Calling AppiumDriver.performTouch() with args: [[{"action":"press","options":{"x":0,"y":284}},{"action":"wait","options":{"ms":1000}},{"action":"moveTo","options":{"x":0,"y":-114}},{"action":"release","options":{}}],"9cf39378-bb3b-4720-956e-ee119290d586"]
[debug] [iOS] Executing iOS command 'performTouch'
[debug] [UIAuto] Sending command to instruments: au.dragApp(0, 284, 0, 170, 1)
[debug] [Instruments] [INST] 2017-03-06 05:50:29 +0000 Debug: Got new command 99 from instruments: au.dragApp(0, 284, 0, 170, 1)
[debug] [Instruments] [INST] 2017-03-06 05:50:29 +0000 Debug: evaluating au.dragApp(0, 284, 0, 170, 1)
[debug] [Instruments] [INST] 2017-03-06 05:50:29 +0000 Debug: target.dragFromToForDuration({x:"0", y:"284"}, {x:"0", y:"170"}, "1")
[debug] [Instruments] [INST] 2017-03-06 05:50:30 +0000 Debug: evaluation finished
[debug] [Instruments] [INST] 2017-03-06 05:50:30 +0000 Debug: responding with:
[debug] [Instruments] [INST] 2017-03-06 05:50:30 +0000 Debug: Running system command #100: /usr/local/Cellar/node/7.5.0/bin/node /usr/local/lib/node_modules/appium/node_modules/appium-uiauto/build/lib/bin/command-proxy-client.js /var/folders/yx/q5fqv2j55_ddwj2m5_x8jv7h0000gn/T/instruments_sock 2,{"status":0,"value":""}...
[debug] [UIAuto] Socket data received (25 bytes)
[debug] [UIAuto] Got result from instruments: {"status":0,"value":""}
[debug] [MJSONWP] Responding to client with driver.performTouch() result: ""
[HTTP] <-- POST /wd/hub/session/9cf39378-bb3b-4720-956e-ee119290d586/touch/perform 200 1856 ms - 74
[HTTP] --> POST /wd/hub/session/9cf39378-bb3b-4720-956e-ee119290d586/touch/perform {"actions":[{"action":"press","options":{"x":0,"y":284}},{"action":"moveTo","options":{"x":160,"y":-252}},{"action":"release","options":{}}]}

With TouchActions it does not work on iOS but works on Android

new TouchAction(driver).press(0, scrollStart).
                    moveTo(0,scrollEnd).release().perform();

[debug] [MJSONWP] Calling AppiumDriver.performTouch() with args: [[{"action":"press","options":**{"x":0,"y":284}},{"action":"moveTo","options":{"x":0,"y":170}}**,{"action":"release","options":{}}],"9cf39378-bb3b-4720-956e-ee119290d586"]
[debug] [iOS] Executing iOS command 'performTouch'
[debug] [UIAuto] Sending command to instruments: target.touch([{"touch":**[{"x":0,"y":284}],"time":0.2},{"touch":[{"x":0,"y":454}]**,"time":0.4}])
[debug] [Instruments] [INST] 2017-03-06 05:51:33 +0000 Debug: Got new command 101 from instruments: target.touch([{"touch":[{"x":0,"y":284}],"time":0.2},{"touch":[{"x":0,"y":454}],"time":0.4}])
[debug] [Instruments] [INST] 2017-03-06 05:51:33 +0000 Debug: evaluating target.touch([{"touch":[{"x":0,"y":284}],"time":0.2},{"touch":[{"x":0,"y":454}],"time":0.4}])
[debug] [Instruments] [INST] 2017-03-06 05:51:33 +0000 Debug: target.touch(__NSCFArray)
[debug] [Instruments] [INST] 2017-03-06 05:51:33 +0000 Debug: evaluation finished
[debug] [Instruments] [INST] 2017-03-06 05:51:33 +0000 Debug: responding with:
[debug] [Instruments] [INST] 2017-03-06 05:51:33 +0000 Debug: Running system command #102: /usr/local/Cellar/node/7.5.0/bin/node /usr/local/lib/node_modules/appium/node_modules/appium-uiauto/build/lib/bin/command-proxy-client.js /var/folders/yx/q5fqv2j55_ddwj2m5_x8jv7h0000gn/T/instruments_sock 2,{"status":0,"value":""}...
[debug] [UIAuto] Socket data received (25 bytes)
[debug] [UIAuto] Got result from instruments: {"status":0,"value":""}
[debug] [MJSONWP] Responding to client with driver.performTouch() result: null
[HTTP] <-- POST /wd/hub/session/9cf39378-bb3b-4720-956e-ee119290d586/touch/perform 200 1587 ms - 76

client sends y:170 and target.touch(takes it as 454)

@TikhomirovSergey

Member

saikrishna321 commented Mar 6, 2017

@jlipps i see this issue on iOS side as well, with the below code it works fine - scroll down(legacy)
server logs are shows the right co-ordinates

Dimension dimensions = driver.manage().window().getSize();
Double screenHeightStart = dimensions.getHeight() * 0.5    
int scrollStart = screenHeightStart.intValue();
Double screenHeightEnd = dimensions.getHeight() * 0.30;
int scrollEnd = screenHeightEnd.intValue();
 driver.swipe(0, scrollStart, 0, scrollEnd, 1000);

[debug] [MJSONWP] Calling AppiumDriver.performTouch() with args: [[{"action":"press","options":{"x":0,"y":284}},{"action":"wait","options":{"ms":1000}},{"action":"moveTo","options":{"x":0,"y":-114}},{"action":"release","options":{}}],"9cf39378-bb3b-4720-956e-ee119290d586"]
[debug] [iOS] Executing iOS command 'performTouch'
[debug] [UIAuto] Sending command to instruments: au.dragApp(0, 284, 0, 170, 1)
[debug] [Instruments] [INST] 2017-03-06 05:50:29 +0000 Debug: Got new command 99 from instruments: au.dragApp(0, 284, 0, 170, 1)
[debug] [Instruments] [INST] 2017-03-06 05:50:29 +0000 Debug: evaluating au.dragApp(0, 284, 0, 170, 1)
[debug] [Instruments] [INST] 2017-03-06 05:50:29 +0000 Debug: target.dragFromToForDuration({x:"0", y:"284"}, {x:"0", y:"170"}, "1")
[debug] [Instruments] [INST] 2017-03-06 05:50:30 +0000 Debug: evaluation finished
[debug] [Instruments] [INST] 2017-03-06 05:50:30 +0000 Debug: responding with:
[debug] [Instruments] [INST] 2017-03-06 05:50:30 +0000 Debug: Running system command #100: /usr/local/Cellar/node/7.5.0/bin/node /usr/local/lib/node_modules/appium/node_modules/appium-uiauto/build/lib/bin/command-proxy-client.js /var/folders/yx/q5fqv2j55_ddwj2m5_x8jv7h0000gn/T/instruments_sock 2,{"status":0,"value":""}...
[debug] [UIAuto] Socket data received (25 bytes)
[debug] [UIAuto] Got result from instruments: {"status":0,"value":""}
[debug] [MJSONWP] Responding to client with driver.performTouch() result: ""
[HTTP] <-- POST /wd/hub/session/9cf39378-bb3b-4720-956e-ee119290d586/touch/perform 200 1856 ms - 74
[HTTP] --> POST /wd/hub/session/9cf39378-bb3b-4720-956e-ee119290d586/touch/perform {"actions":[{"action":"press","options":{"x":0,"y":284}},{"action":"moveTo","options":{"x":160,"y":-252}},{"action":"release","options":{}}]}

With TouchActions it does not work on iOS but works on Android

new TouchAction(driver).press(0, scrollStart).
                    moveTo(0,scrollEnd).release().perform();

[debug] [MJSONWP] Calling AppiumDriver.performTouch() with args: [[{"action":"press","options":**{"x":0,"y":284}},{"action":"moveTo","options":{"x":0,"y":170}}**,{"action":"release","options":{}}],"9cf39378-bb3b-4720-956e-ee119290d586"]
[debug] [iOS] Executing iOS command 'performTouch'
[debug] [UIAuto] Sending command to instruments: target.touch([{"touch":**[{"x":0,"y":284}],"time":0.2},{"touch":[{"x":0,"y":454}]**,"time":0.4}])
[debug] [Instruments] [INST] 2017-03-06 05:51:33 +0000 Debug: Got new command 101 from instruments: target.touch([{"touch":[{"x":0,"y":284}],"time":0.2},{"touch":[{"x":0,"y":454}],"time":0.4}])
[debug] [Instruments] [INST] 2017-03-06 05:51:33 +0000 Debug: evaluating target.touch([{"touch":[{"x":0,"y":284}],"time":0.2},{"touch":[{"x":0,"y":454}],"time":0.4}])
[debug] [Instruments] [INST] 2017-03-06 05:51:33 +0000 Debug: target.touch(__NSCFArray)
[debug] [Instruments] [INST] 2017-03-06 05:51:33 +0000 Debug: evaluation finished
[debug] [Instruments] [INST] 2017-03-06 05:51:33 +0000 Debug: responding with:
[debug] [Instruments] [INST] 2017-03-06 05:51:33 +0000 Debug: Running system command #102: /usr/local/Cellar/node/7.5.0/bin/node /usr/local/lib/node_modules/appium/node_modules/appium-uiauto/build/lib/bin/command-proxy-client.js /var/folders/yx/q5fqv2j55_ddwj2m5_x8jv7h0000gn/T/instruments_sock 2,{"status":0,"value":""}...
[debug] [UIAuto] Socket data received (25 bytes)
[debug] [UIAuto] Got result from instruments: {"status":0,"value":""}
[debug] [MJSONWP] Responding to client with driver.performTouch() result: null
[HTTP] <-- POST /wd/hub/session/9cf39378-bb3b-4720-956e-ee119290d586/touch/perform 200 1587 ms - 76

client sends y:170 and target.touch(takes it as 454)

@TikhomirovSergey

@TikhomirovSergey

This comment has been minimized.

Show comment
Hide comment
@TikhomirovSergey

TikhomirovSergey Nov 10, 2017

Member

UPD: The swipe is not supported now. So the problem is moveTo. It seems that it it should accept absolute coordinates for all supported platfoms.

ping @mykola-mokhnach

Member

TikhomirovSergey commented Nov 10, 2017

UPD: The swipe is not supported now. So the problem is moveTo. It seems that it it should accept absolute coordinates for all supported platfoms.

ping @mykola-mokhnach

@mykola-mokhnach

This comment has been minimized.

Show comment
Hide comment
@mykola-mokhnach

mykola-mokhnach Feb 16, 2018

Collaborator

This has been now merged to beta

Collaborator

mykola-mokhnach commented Feb 16, 2018

This has been now merged to beta

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment