swiping an element does not work

[rompic]

Description

I'm experiencing a similar problem as in #350
If I try to swipe an android drawer menu with
menu.swipe(SwipeElementDirection.UP, 100);

I get an error stating that the coordinates are outside of the display bounds:
[AndroidBootstrap] [BOOTSTRAP LOG] [debug] Display bounds: [0,0][1080,1776]
[AndroidBootstrap] [BOOTSTRAP LOG] [debug] Returning result: {"status":13,"value":"Coordinate [x=390.0, y=1893.0] is outside of element rect: [0,0][1080,1776]"}

Expected behaviour:
Swiping up or down is possible.
This or similar code worked before updating to 1.5.2 with java-client 4.0.0

Environment

• java client build version or git revision if you use some shapshot: 4.0.0
• Appium server version or git revision if you use some shapshot: 1.5.2
• Desktop OS/version used to run Appium if necessary: Mac OSX 10.11.5
• Node.js version (unless using Appium.app|exe) or Appium CLI or Appium.app|exe: v6.2.1
• Mobile platform/version under test: Android Emulator nexus 5 / Android 5.0.2
• Real device or emulator/simulator: emulator

Details

Please provide more details, if necessary.
@AndroidFindBy(id="ab_omg_menu_left_layout")
private MobileElement menu;

menu.swipe(SwipeElementDirection.UP, 100);

Code To Reproduce Issue [ Good To Have ]

see above

Ecxeption stacktraces

Link to Appium logs

https://gist.github.com/rompic/a143b5540312254eef0ca8960d7a8028

[SrinivasanTarget]

Thanks @rompic We will look into this ASAP.

[rompic]

on a real device I get saner output, but it still does not work:
[AndroidBootstrap] [BOOTSTRAP LOG] [debug] Got command action: getSize
[AndroidBootstrap] [BOOTSTRAP LOG] [debug] Returning result: {"status":0,"value":{"width":520,"height":1022}}
[debug] [AndroidBootstrap] Received command result from bootstrap
[MJSONWP] Responding to client with driver.getSize() result: {"width":520,"height":1022}
[HTTP] <-- GET /wd/hub/session/772679b7-e536-4cb3-9fec-33b1a5f4021d/element/8/size 200 38 ms - 99
[HTTP] --> POST /wd/hub/session/772679b7-e536-4cb3-9fec-33b1a5f4021d/touch/perform {"actions":[{"action":"press","options":{"x":260,"y":1184}},{"action":"wait","options":{"ms":100}},{"action":"moveTo","options":{"x":260,"y":162}},{"action":"release","options":{}}]}
[MJSONWP] Calling AppiumDriver.performTouch() with args: [[{"action":"press","options":{"x":260,"y":1184}},{"action":"wait","options":{"ms":100}},{"action":"moveTo","options":{"x":260,"y":162}},{"action":...
[debug] [AndroidBootstrap] Sending command to android: {"cmd":"action","action":"swipe","params":{"startX":260,"startY":1184,"endX":260,"endY":162,"steps":3}}
[AndroidBootstrap] [BOOTSTRAP LOG] [debug] Got data from client: {"cmd":"action","action":"swipe","params":{"startX":260,"startY":1184,"endX":260,"endY":162,"steps":3}}
[AndroidBootstrap] [BOOTSTRAP LOG] [debug] Got command of type ACTION
[AndroidBootstrap] [BOOTSTRAP LOG] [debug] Got command action: swipe
[AndroidBootstrap] [BOOTSTRAP LOG] [debug] Display bounds: [0,0][720,1184]
[AndroidBootstrap] [BOOTSTRAP LOG] [debug] Returning result: {"status":13,"value":"Coordinate [x=260.0, y=1184.0] is outside of element rect: [0,0][720,1184]"}
[debug] [AndroidBootstrap] Received command result from bootstrap

[TikhomirovSergey]

@rompic @SrinivasanTarget
It is interesting. I'm suspecting the situation when the bottom border is outside of the screen.
This string crearly says that
https://gist.github.com/rompic/a143b5540312254eef0ca8960d7a8028#file-swipe_android-L585

Have you tried that something like

menu.swipe(SwipeElementDirection.UP, 200, //this is the offset from the start board,
100, //this is the offset from the end board
 100);

?

[rompic]

tried your example on a real device:
[MJSONWP] Responding to client with driver.getSize() result: {"width":520,"height":1022}
[HTTP] <-- GET /wd/hub/session/8aef6e8c-a989-46af-ad1f-445f84113802/element/8/size 200 35 ms - 99
[HTTP] --> POST /wd/hub/session/8aef6e8c-a989-46af-ad1f-445f84113802/touch/perform {"actions":[{"action":"press","options":{"x":260,"y":984}},{"action":"wait","options":{"ms":100}},{"action":"moveTo","options":{"x":260,"y":262}},{"action":"release","options":{}}]}
[MJSONWP] Calling AppiumDriver.performTouch() with args: [[{"action":"press","options":{"x":260,"y":984}},{"action":"wait","options":{"ms":100}},{"action":"moveTo","options":{"x":260,"y":262}},{"action":"...
[debug] [AndroidBootstrap] Sending command to android: {"cmd":"action","action":"swipe","params":{"startX":260,"startY":984,"endX":260,"endY":262,"steps":3}}
[AndroidBootstrap] [BOOTSTRAP LOG] [debug] Got data from client: {"cmd":"action","action":"swipe","params":{"startX":260,"startY":984,"endX":260,"endY":262,"steps":3}}
[AndroidBootstrap] [BOOTSTRAP LOG] [debug] Got command of type ACTION
[AndroidBootstrap] [BOOTSTRAP LOG] [debug] Got command action: swipe
[AndroidBootstrap] [BOOTSTRAP LOG] [debug] Display bounds: [0,0][720,1184]
[AndroidBootstrap] [BOOTSTRAP LOG] [debug] Display bounds: [0,0][720,1184]
[AndroidBootstrap] [BOOTSTRAP LOG] [debug] Swiping from [x=260.0, y=984.0] to [x=260.0, y=262.0] with steps: 3
[AndroidBootstrap] [BOOTSTRAP LOG] [debug] Returning result: {"status":0,"value":true}
[debug] [AndroidBootstrap] Received command result from bootstrap
[MJSONWP] Responding to client with driver.performTouch() result: true
[HTTP] <-- POST /wd/hub/session/8aef6e8c-a989-46af-ad1f-445f84113802/touch/perform 200 88 ms - 76
-> works

on emulator:
[MJSONWP] Responding to client with driver.getSize() result: {"width":780,"height":1650}
[HTTP] <-- GET /wd/hub/session/54846725-fe0f-4173-8c61-565a799d964a/element/8/size 200 13 ms - 99
[HTTP] --> POST /wd/hub/session/54846725-fe0f-4173-8c61-565a799d964a/touch/perform {"actions":[{"action":"press","options":{"x":390,"y":1693}},{"action":"wait","options":{"ms":100}},{"action":"moveTo","options":{"x":390,"y":343}},{"action":"release","options":{}}]}
[MJSONWP] Calling AppiumDriver.performTouch() with args: [[{"action":"press","options":{"x":390,"y":1693}},{"action":"wait","options":{"ms":100}},{"action":"moveTo","options":{"x":390,"y":343}},{"action":...
[debug] [AndroidBootstrap] Sending command to android: {"cmd":"action","action":"swipe","params":{"startX":390,"startY":1693,"endX":390,"endY":343,"steps":3}}
[AndroidBootstrap] [BOOTSTRAP LOG] [debug] Got data from client: {"cmd":"action","action":"swipe","params":{"startX":390,"startY":1693,"endX":390,"endY":343,"steps":3}}
[AndroidBootstrap] [BOOTSTRAP LOG] [debug] Got command of type ACTION
[AndroidBootstrap] [BOOTSTRAP LOG] [debug] Got command action: swipe
[AndroidBootstrap] [BOOTSTRAP LOG] [debug] Display bounds: [0,0][1080,1776]
[AndroidBootstrap] [BOOTSTRAP LOG] [debug] Display bounds: [0,0][1080,1776]
[AndroidBootstrap] [BOOTSTRAP LOG] [debug] Swiping from [x=390.0, y=1693.0] to [x=390.0, y=343.0] with steps: 3
[AndroidBootstrap] [BOOTSTRAP LOG] [debug] Returning result: {"status":0,"value":true}
[debug] [AndroidBootstrap] Received command result from bootstrap
[MJSONWP] Responding to client with driver.performTouch() result: true
[HTTP] <-- POST /wd/hub/session/54846725-fe0f-4173-8c61-565a799d964a/touch/perform 200 99 ms - 76
-> works

still this looks like a bug. I actually used a similar call to yours to swipe one "page" of a menu to collect all entries. This at least seemed to work before updateing (note that either the method signature of swipe changed or I got it wrong in this case and it worked by chance).

//assumes that we are at the beginning of the screen
    public List<String> getMenuItems(){ //todo solution with mobileelements instead of strings?
        //todo don't use set as there may be duplicate entries and we also want to check the order
        LinkedHashSet<String> menuItemsComplete= menuItems.stream().map(MobileElement::getText)
                .collect(Collectors.toCollection(LinkedHashSet::new));
        //on android only elements within the viewport are included in the elementTree
        if (SetupHelper.isTargetAndroid()){
            boolean searchElements=true;
            while(searchElements)
            {
                menu.swipe(SwipeElementDirection.UP, 1, 1, GestureHelper.getScreenHeight()-1); //swipe one page
                LinkedHashSet<String> menuItemsTemp=menuItems.stream().map(MobileElement::getText)
                        .collect(Collectors.toCollection(LinkedHashSet::new));
                if (menuItemsComplete.containsAll(menuItemsTemp)){
                    searchElements=false;
                }
                else {
                    menuItemsComplete.addAll(menuItemsTemp);
                }

            }
        }
        return new ArrayList<>(menuItemsComplete);
    }

btw. would be great if you can point to a better way of doing this.

[SrinivasanTarget]

@rompic You can find the samples here: https://github.com/appium/java-client/blob/master/src/test/java/io/appium/java_client/android/AndroidGestureTest.java#L111

[SrinivasanTarget]

@TikhomirovSergey I suspect this seems to be a bug only if offset is 0 i.e only if user tries this way to swipe: menu.swipe(SwipeElementDirection.UP, 100) but below one should be working fine

menu.swipe(SwipeElementDirection.UP, 200, //this is the offset from the start board,
100, //this is the offset from the end board
 100);

[TikhomirovSergey]

@SrinivasanTarget There may be a problem when an element is partially visible on a screen and other parts are outside of screen borders. May be it is the server problem (wrong coordinates are returned).

It looks wrong.

menu.swipe(SwipeElementDirection.UP, 1, 1, GestureHelper.getScreenHeight()-1)

The last parameter is a time.
https://github.com/appium/java-client/blob/master/src/main/java/io/appium/java_client/MobileElement.java#L62

@rompic What happens if you try to do something like that

        Dimension size = menu.getSize();
        Point location =    menu.getLocation();

        driver.tap(2, location.x + size.getWidth(), location.y + size.getHeight(), 2000)

or perform another touch action with same coordinates?

[rompic]

@TikhomirovSergey I get

   [debug] [AndroidBootstrap] Received command result from bootstrap
    [MJSONWP] Responding to client with driver.findElements() result: [{"ELEMENT":"9"}]
    [HTTP] <-- POST /wd/hub/session/cbf2b7f6-efee-4245-928c-95a5694e0cf8/elements 200 57 ms - 89 
    [HTTP] --> POST /wd/hub/session/cbf2b7f6-efee-4245-928c-95a5694e0cf8/timeouts {"type":"implicit","ms":1000}
    [MJSONWP] Calling AppiumDriver.timeouts() with args: ["implicit",1000,"cbf2b7f6-efee-4245-928c-95a5694e0cf8"]
    [debug] [BaseDriver] Set implicit wait to 1000ms
    [MJSONWP] Responding to client with driver.timeouts() result: null
    [HTTP] <-- POST /wd/hub/session/cbf2b7f6-efee-4245-928c-95a5694e0cf8/timeouts 200 5 ms - 76 
    [HTTP] --> GET /wd/hub/session/cbf2b7f6-efee-4245-928c-95a5694e0cf8/element/9/location {}
    [MJSONWP] Calling AppiumDriver.getLocation() with args: ["9","cbf2b7f6-efee-4245-928c-95a5694e0cf8"]
    [debug] [AndroidBootstrap] Sending command to android: {"cmd":"action","action":"element:getLocation","params":{"elementId":"9"}}
    [AndroidBootstrap] [BOOTSTRAP LOG] [debug] Got data from client: {"cmd":"action","action":"element:getLocation","params":{"elementId":"9"}}
    [AndroidBootstrap] [BOOTSTRAP LOG] [debug] Got command of type ACTION
    [AndroidBootstrap] [BOOTSTRAP LOG] [debug] Got command action: getLocation
    [AndroidBootstrap] [BOOTSTRAP LOG] [debug] Returning result: {"status":0,"value":{"x":0,"y":243}}
    [debug] [AndroidBootstrap] Received command result from bootstrap
    [MJSONWP] Responding to client with driver.getLocation() result: {"x":0,"y":243}
    [HTTP] <-- GET /wd/hub/session/cbf2b7f6-efee-4245-928c-95a5694e0cf8/element/9/location 200 33 ms - 87 
    [HTTP] --> POST /wd/hub/session/cbf2b7f6-efee-4245-928c-95a5694e0cf8/touch/multi/perform {"actions":[[{"action":"press","options":{"x":**780**,"y":**1893**}},{"action":"wait","options":{"ms":2000}},{"action":"release","options":{}}],[{"action":"press","options":{"x":**780**,"y":**1893**}},{"action":"wait","options":{"ms":2000}},{"action":"release","options":{}}]]}
    [MJSONWP] Calling AppiumDriver.performMultiAction() with args: [[[{"action":"press","options":{"x":780,"y":1893}},{"action":"wait","options":{"ms":2000}},{"action":"release","options":{}}],[{"action":"press","o...
    [debug] [AndroidBootstrap] Sending command to android: {"cmd":"action","action":"performMultiPointerGesture","params":{"actions":[[{"action":"press","time":0.005,"touch":{"x":**780**,"y":**1893**}},{"action":"wait","time":2.005,"touch":{"ms":2000,"x":780,"y":1893}}],[{"action":"press","time":0.005,"touch":{"x":**780**,"y":**1893**}},{"action":"wait","time":2.005,"touch":{"ms":2000,"x":**780**,"y":**1893**}}]]}}
    [AndroidBootstrap] [BOOTSTRAP LOG] [debug] Got data from client: {"cmd":"action","action":"performMultiPointerGesture","params":{"actions":[[{"action":"press","time":0.005,"touch":{"x":**780**,"y":**1893**}},{"action":"wait","time":2.005,"touch":{"ms":2000,"x":780,"y":1893}}],[{"action":"press","time":0.005,"touch":{"x":**780**,"y":**1893**}},{"action":"wait","time":2.005,"touch":{"ms":2000,"x":**780**,"y":**1893**}}]]}}
    [AndroidBootstrap] [BOOTSTRAP LOG] [debug] Got command of type ACTION
    [AndroidBootstrap] [BOOTSTRAP LOG] [debug] Got command action: performMultiPointerGesture
    [AndroidBootstrap] [BOOTSTRAP LOG] [debug] Returning result: {"status":0,"value":"OK"}
    [debug] [AndroidBootstrap] Received command result from bootstrap
    [MJSONWP] Responding to client with driver.performMultiAction() result: "OK"
    [HTTP] <-- POST /wd/hub/session/cbf2b7f6-efee-4245-928c-95a5694e0cf8/touch/multi/perform 200 19516 ms - 76 

but it taps on the android multitask button:

[shynkevich-alex]

when will it be fixed?

[TikhomirovSergey]

@shynkevich-alex These methods (swipe/othere gestures) are going to be deprecated. Probably it won;t be supported anymore.

[saikrishna321]

@TikhomirovSergey @SrinivasanTarget so all these would move to TouchActions ?

[TikhomirovSergey]

@saikrishna321 No. There will be no predefined gesture methods. They are flacky and don't work in many user cases. We will allow users to construct any gesture on their own.

But I think we should provide some feature instead to simplify user's work