New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

iOS WebView scroll up #3974

Closed
wilcollins opened this Issue Nov 4, 2014 · 29 comments

Comments

Projects
None yet
8 participants
@wilcollins

wilcollins commented Nov 4, 2014

We are currently able to scroll down in our webview using

jse.executeScript("arguments[0].scrollIntoView(false);", element);

but it is only able to scroll DOWN to elements, not up. I have used numerous methods to try to work around this issue, but none of them have been successful so far.

I have tried using IOSDriver.scrollTo but for some reason it is unable to locate the element with the provided text even though I use driver.findElement("//someXpathLocator").getText()

I also tried to use a similar approach I use for some Android tests where I utilize HasTouchScreen .down(), .move(), and .up() methods, but this is not yet implemented.

I even used "mobile: scroll" on a native element but the entire webview was scrolled, not just the list view, leaving the top third of the screen black/empty (without moving the listview) until the next UI interaction occurred.

I don't think you guys would have shipped without full scrolling functionality within iOS webview, but I can't find a method that works and it is blocking our testing so any help would be greatly appreciated!

@0x1mason

This comment has been minimized.

Show comment
Hide comment
@0x1mason

0x1mason commented Nov 7, 2014

Can you try scrollTo using the element name? Looks like it doesn't accept xpaths.

https://github.com/appium/java-client/blob/dbb443a30fbc5ea9ccbb9f0060b8ff8bfd0c6bd2/src/main/java/io/appium/java_client/ios/IOSElement.java#L39

@Jonahss Any suggestions?

@0x1mason

This comment has been minimized.

Show comment
Hide comment
@0x1mason

0x1mason Nov 7, 2014

Looks like this may be related: #3851

0x1mason commented Nov 7, 2014

Looks like this may be related: #3851

@0x1mason 0x1mason added iOS and removed NeedsTriage labels Nov 7, 2014

@0x1mason 0x1mason added this to the Appium 1.3.4 milestone Nov 7, 2014

@wilcollins

This comment has been minimized.

Show comment
Hide comment
@wilcollins

wilcollins Nov 7, 2014

From the documentation:
/**

  • Scroll to the element whose 'text' attribute contains the input text.
  • Scrolling happens within this element
  • @param text input text contained in text attribute
    */

I don't want to use the element name because it is searching for the 'text' attribute which is why I thought element.getText() was the correct value to pass to the function call.

wilcollins commented Nov 7, 2014

From the documentation:
/**

  • Scroll to the element whose 'text' attribute contains the input text.
  • Scrolling happens within this element
  • @param text input text contained in text attribute
    */

I don't want to use the element name because it is searching for the 'text' attribute which is why I thought element.getText() was the correct value to pass to the function call.

@0x1mason

This comment has been minimized.

Show comment
Hide comment
@0x1mason

0x1mason Nov 13, 2014

I looked into what the text argument in scrollTo will actually locate. The java doc isn't so much misleading as it's only a small portion of what will gets searched. There's a lot of voodoo going on under the covers.

The scrollTo method wraps a call to scrollToElementWithPredicate that searches for the name of the element. According to Apple, the name attribute on an iOS element

is derived from the accessibility attribute of the underlying view. If an identifier attribute string is specified, that string is used as the name; otherwise, the label attribute string is used as the name.

It's my understand that we also check the element's text if the other methods return null. All that is to say, if you know the accessibility id or label, that will work too.

0x1mason commented Nov 13, 2014

I looked into what the text argument in scrollTo will actually locate. The java doc isn't so much misleading as it's only a small portion of what will gets searched. There's a lot of voodoo going on under the covers.

The scrollTo method wraps a call to scrollToElementWithPredicate that searches for the name of the element. According to Apple, the name attribute on an iOS element

is derived from the accessibility attribute of the underlying view. If an identifier attribute string is specified, that string is used as the name; otherwise, the label attribute string is used as the name.

It's my understand that we also check the element's text if the other methods return null. All that is to say, if you know the accessibility id or label, that will work too.

@0x1mason

This comment has been minimized.

Show comment
Hide comment
@0x1mason

0x1mason Nov 13, 2014

We definitely need to document this better among other things.

0x1mason commented Nov 13, 2014

We definitely need to document this better among other things.

@wilcollins

This comment has been minimized.

Show comment
Hide comment
@wilcollins

wilcollins Nov 26, 2014

((IOSElement)element).scrollTo(((RemoteWebElement)element).getId());
and
((IOSElement)element).scrollTo(element.getAttribute("id"));

Throw the WebDriverException: Invalid locator strategy: -ios uiautomation

Appium v1.3.3 via npm
java-client 2.1.0
Selenium 2.44.0
iOS 7.1

wilcollins commented Nov 26, 2014

((IOSElement)element).scrollTo(((RemoteWebElement)element).getId());
and
((IOSElement)element).scrollTo(element.getAttribute("id"));

Throw the WebDriverException: Invalid locator strategy: -ios uiautomation

Appium v1.3.3 via npm
java-client 2.1.0
Selenium 2.44.0
iOS 7.1

@wilcollins

This comment has been minimized.

Show comment
Hide comment
@wilcollins

wilcollins Nov 26, 2014

Also, I am still getting "unimplemented" for "mobile: scroll" and "mobile: scrollTo" even though I thought they were supposed to be implemented.

wilcollins commented Nov 26, 2014

Also, I am still getting "unimplemented" for "mobile: scroll" and "mobile: scrollTo" even though I thought they were supposed to be implemented.

@wilcollins

This comment has been minimized.

Show comment
Hide comment
@wilcollins

wilcollins Dec 2, 2014

is "WebDriverException: Invalid locator strategy: -ios uiautomation" caused by Appium's internal code or improper usage of the scrollTo function?

wilcollins commented Dec 2, 2014

is "WebDriverException: Invalid locator strategy: -ios uiautomation" caused by Appium's internal code or improper usage of the scrollTo function?

@jlipps

This comment has been minimized.

Show comment
Hide comment
@jlipps

jlipps Dec 2, 2014

Member

scrollTo is only implemented in the native context

Member

jlipps commented Dec 2, 2014

scrollTo is only implemented in the native context

@wilcollins

This comment has been minimized.

Show comment
Hide comment
@wilcollins

wilcollins Dec 2, 2014

What is the proper method for scrolling in iOS webviews?

wilcollins commented Dec 2, 2014

What is the proper method for scrolling in iOS webviews?

@Jonahss

This comment has been minimized.

Show comment
Hide comment
@Jonahss

Jonahss Dec 2, 2014

Contributor

Good question. However one would normally scroll using Selenium I suppose.
Or you can always use a TouchAction swipe over the element.

On Tue, Dec 2, 2014 at 10:01 AM, Wil Collins notifications@github.com
wrote:

What is the proper method for scrolling in iOS webviews?


Reply to this email directly or view it on GitHub
#3974 (comment).

Contributor

Jonahss commented Dec 2, 2014

Good question. However one would normally scroll using Selenium I suppose.
Or you can always use a TouchAction swipe over the element.

On Tue, Dec 2, 2014 at 10:01 AM, Wil Collins notifications@github.com
wrote:

What is the proper method for scrolling in iOS webviews?


Reply to this email directly or view it on GitHub
#3974 (comment).

@jlipps

This comment has been minimized.

Show comment
Hide comment
@jlipps

jlipps Dec 2, 2014

Member

I usually just scroll using javascript, either taking advantage of jQuery or just modifying scrollTop and scrollLeft until the condition I want is satisfied.

Member

jlipps commented Dec 2, 2014

I usually just scroll using javascript, either taking advantage of jQuery or just modifying scrollTop and scrollLeft until the condition I want is satisfied.

@Jonahss

This comment has been minimized.

Show comment
Hide comment
@Jonahss

Jonahss Dec 4, 2014

Contributor

@wilcollins Still having trouble with scrolling? What have you tried? What works for you?
Any advice for improvements?

Contributor

Jonahss commented Dec 4, 2014

@wilcollins Still having trouble with scrolling? What have you tried? What works for you?
Any advice for improvements?

@wilcollins

This comment has been minimized.

Show comment
Hide comment
@wilcollins

wilcollins Jan 13, 2015

((JavascriptExecutor) (MobileUtil.getDriver())).executeScript("window.scrollTop=1000");
returns null and performs no actions on the webview

and the element's scrollTop property is 0 before executing this
((JavascriptExecutor) (MobileUtil.getDriver())).executeScript("arguments[0].scrollTop=0", element);
so there is no change after this line is executed

wilcollins commented Jan 13, 2015

((JavascriptExecutor) (MobileUtil.getDriver())).executeScript("window.scrollTop=1000");
returns null and performs no actions on the webview

and the element's scrollTop property is 0 before executing this
((JavascriptExecutor) (MobileUtil.getDriver())).executeScript("arguments[0].scrollTop=0", element);
so there is no change after this line is executed

@wilcollins

This comment has been minimized.

Show comment
Hide comment
@wilcollins

wilcollins Jan 13, 2015

jse.executeScript("arguments[0].scrollIntoView(false);", element);
works but only for down-scrolling

wilcollins commented Jan 13, 2015

jse.executeScript("arguments[0].scrollIntoView(false);", element);
works but only for down-scrolling

@wilcollins

This comment has been minimized.

Show comment
Hide comment
@wilcollins

wilcollins Jan 13, 2015

Advice: Any function that works in one direction should work for both. Being able to scroll directly to an element is a good API endpoint so the user doesn't have to determine the coordinates and then pass that to the function. And if you could encapsulate all of the scrolling functions into a universal "scrollTo" that picks the appropriate method depending on the current state (OS, context, etc), that would be great.

wilcollins commented Jan 13, 2015

Advice: Any function that works in one direction should work for both. Being able to scroll directly to an element is a good API endpoint so the user doesn't have to determine the coordinates and then pass that to the function. And if you could encapsulate all of the scrolling functions into a universal "scrollTo" that picks the appropriate method depending on the current state (OS, context, etc), that would be great.

@wilcollins

This comment has been minimized.

Show comment
Hide comment
@wilcollins

wilcollins Jan 13, 2015

I have also tried TouchActions in the native context with no success:

public static void scrollTo(WebElement we) {
        Point p = we.getLocation();
        while (!we.isDisplayed()) {
            try {
                MobileUtil.switchToNativeView();
                AppiumDriver driver = (AppiumDriver) MobileUtil.getDriver();
                Point appLoc = driver.findElement(By.xpath("//UIAWebView[1]")).getLocation();

                TouchAction action = new TouchAction(driver);
                action.press(1, 1);
                action.moveTo(1, 100);
                action.release();
                ((MobileDriver) MobileUtil.getDriver()).performTouchAction(action); << does nothing

                driver.swipe(1, 1, 1, 100, 1); << "An error occurred while executing user supplied JavaScript."
            }
            catch (Exception e) {
                e.printStackTrace();
            }
            finally {
                MobileUtil.switchToWebView();
            }
        }
    }

MobileUtil.getDriver() returns type WebDriver, but is instantiated as a SwipeableIOSDriver that extends IOSDriver and implements HasTouchScreen

wilcollins commented Jan 13, 2015

I have also tried TouchActions in the native context with no success:

public static void scrollTo(WebElement we) {
        Point p = we.getLocation();
        while (!we.isDisplayed()) {
            try {
                MobileUtil.switchToNativeView();
                AppiumDriver driver = (AppiumDriver) MobileUtil.getDriver();
                Point appLoc = driver.findElement(By.xpath("//UIAWebView[1]")).getLocation();

                TouchAction action = new TouchAction(driver);
                action.press(1, 1);
                action.moveTo(1, 100);
                action.release();
                ((MobileDriver) MobileUtil.getDriver()).performTouchAction(action); << does nothing

                driver.swipe(1, 1, 1, 100, 1); << "An error occurred while executing user supplied JavaScript."
            }
            catch (Exception e) {
                e.printStackTrace();
            }
            finally {
                MobileUtil.switchToWebView();
            }
        }
    }

MobileUtil.getDriver() returns type WebDriver, but is instantiated as a SwipeableIOSDriver that extends IOSDriver and implements HasTouchScreen

@wilcollins

This comment has been minimized.

Show comment
Hide comment
@wilcollins

wilcollins Jan 13, 2015

Update: "mobile: swipe" works on Appium 1.3.0 in both the Native and WebView contexts (both UP and DOWN scroll work). However, in 1.3.4, this function returns "Not Yet Implemented" in both contexts. So, for now, I have reverted to 1.3.0 -- please fix iOS scrolling so we can use the latest features :-)

wilcollins commented Jan 13, 2015

Update: "mobile: swipe" works on Appium 1.3.0 in both the Native and WebView contexts (both UP and DOWN scroll work). However, in 1.3.4, this function returns "Not Yet Implemented" in both contexts. So, for now, I have reverted to 1.3.0 -- please fix iOS scrolling so we can use the latest features :-)

@wilcollins

This comment has been minimized.

Show comment
Hide comment
@wilcollins

wilcollins Jan 13, 2015

/**
     * This method scrolls the page
     * 
     * @param pageToScroll
     *            - need to specify the page to perform the scroll
     * @param speed
     *            - speed to scroll in pixels/second
     * @return boolean
     *         return if the scroll was successful
     * @throws InterruptedException
     */
    @SuppressWarnings("serial")
    private static boolean scroll(WebElement viewToScroll, int speed, boolean scrollDown) {

        RemoteWebDriver remoteWebDriver = (RemoteWebDriver) MobileUtil.getDriver();

        int offSize = remoteWebDriver.manage().window().getSize().getHeight() / 3;
        int screenSize = remoteWebDriver.manage().window().getSize().getHeight();
        Double initialYPos = 0.0;
        Double finalYPos = 0.0;
        // final Double scrollSpeed = (double) speed / speedControl;
        final Double scrollSpeed = 30.0 / speed;
        final Double xPosition = remoteWebDriver.manage().window().getSize().getWidth() - 15.0;

        if (scrollDown) {
            // Code to scroll down
            initialYPos = (double) screenSize - offSize;
            finalYPos = initialYPos - offSize;
        }
        else {
            // Code to scroll up
            finalYPos = (double) screenSize - offSize;
            initialYPos = finalYPos - offSize;

        }
        final Double initialY = initialYPos;
        final Double finalY = finalYPos;

        System.out.println("Start Y: " + initialY + "; Final Y: " + finalY + "; Scroll duration: " + scrollSpeed);

        remoteWebDriver.executeScript("mobile: swipe", new HashMap<String, Double>() {
            {
                put("touchCount", 1.0);
                put("startX", xPosition);
                put("startY", initialY);
                put("endX", xPosition);
                put("endY", finalY);
                put("duration", scrollSpeed);
            }
        });

        return true;
    }

wilcollins commented Jan 13, 2015

/**
     * This method scrolls the page
     * 
     * @param pageToScroll
     *            - need to specify the page to perform the scroll
     * @param speed
     *            - speed to scroll in pixels/second
     * @return boolean
     *         return if the scroll was successful
     * @throws InterruptedException
     */
    @SuppressWarnings("serial")
    private static boolean scroll(WebElement viewToScroll, int speed, boolean scrollDown) {

        RemoteWebDriver remoteWebDriver = (RemoteWebDriver) MobileUtil.getDriver();

        int offSize = remoteWebDriver.manage().window().getSize().getHeight() / 3;
        int screenSize = remoteWebDriver.manage().window().getSize().getHeight();
        Double initialYPos = 0.0;
        Double finalYPos = 0.0;
        // final Double scrollSpeed = (double) speed / speedControl;
        final Double scrollSpeed = 30.0 / speed;
        final Double xPosition = remoteWebDriver.manage().window().getSize().getWidth() - 15.0;

        if (scrollDown) {
            // Code to scroll down
            initialYPos = (double) screenSize - offSize;
            finalYPos = initialYPos - offSize;
        }
        else {
            // Code to scroll up
            finalYPos = (double) screenSize - offSize;
            initialYPos = finalYPos - offSize;

        }
        final Double initialY = initialYPos;
        final Double finalY = finalYPos;

        System.out.println("Start Y: " + initialY + "; Final Y: " + finalY + "; Scroll duration: " + scrollSpeed);

        remoteWebDriver.executeScript("mobile: swipe", new HashMap<String, Double>() {
            {
                put("touchCount", 1.0);
                put("startX", xPosition);
                put("startY", initialY);
                put("endX", xPosition);
                put("endY", finalY);
                put("duration", scrollSpeed);
            }
        });

        return true;
    }
@jlipps

This comment has been minimized.

Show comment
Hide comment
@jlipps

jlipps Feb 12, 2015

Member

what about "mobile: scroll"?

Member

jlipps commented Feb 12, 2015

what about "mobile: scroll"?

@wilcollins

This comment has been minimized.

Show comment
Hide comment
@wilcollins

wilcollins Feb 12, 2015

We have currently reverted to 1.3.0 and are using "mobile: swipe".

What are the parameters for "mobile: scroll"? And which versions should that be supported in?

wilcollins commented Feb 12, 2015

We have currently reverted to 1.3.0 and are using "mobile: swipe".

What are the parameters for "mobile: scroll"? And which versions should that be supported in?

@jlipps

This comment has been minimized.

Show comment
Hide comment
@jlipps

jlipps Feb 12, 2015

Member

it's supported in all recent versions. something like this in Java, maybe?

        remoteWebDriver.executeScript("mobile: scroll", new HashMap<String, Double>() {
            {
                put("direction", "down");
            }
        });
Member

jlipps commented Feb 12, 2015

it's supported in all recent versions. something like this in Java, maybe?

        remoteWebDriver.executeScript("mobile: scroll", new HashMap<String, Double>() {
            {
                put("direction", "down");
            }
        });
@wilcollins

This comment has been minimized.

Show comment
Hide comment
@wilcollins

wilcollins Feb 12, 2015

That should be HashMap<String, String> but I'll give that a shot when I get some time and I'll let you know how it goes. I had tried it before with no success but hopefully that won't be the case again.

Also, is it supported for WebViews? I have noticed that certain functionality was limited (at least at some point) within webviews

wilcollins commented Feb 12, 2015

That should be HashMap<String, String> but I'll give that a shot when I get some time and I'll let you know how it goes. I had tried it before with no success but hopefully that won't be the case again.

Also, is it supported for WebViews? I have noticed that certain functionality was limited (at least at some point) within webviews

@jlipps

This comment has been minimized.

Show comment
Hide comment
@jlipps

jlipps Feb 12, 2015

Member

you can always go back to the native context to use that command, so I would say it doesn't matter whether it works specifically within webviews, no?

Member

jlipps commented Feb 12, 2015

you can always go back to the native context to use that command, so I would say it doesn't matter whether it works specifically within webviews, no?

@wilcollins

This comment has been minimized.

Show comment
Hide comment
@wilcollins

wilcollins Feb 12, 2015

Read my first post in this thread. I mentioned my attempt to use it in
native context
On Feb 12, 2015 2:48 PM, "Jonathan Lipps" notifications@github.com wrote:

you can always go back to the native context to use that command, so I
would say it doesn't matter whether it works specifically within webviews,
no?


Reply to this email directly or view it on GitHub
#3974 (comment).

wilcollins commented Feb 12, 2015

Read my first post in this thread. I mentioned my attempt to use it in
native context
On Feb 12, 2015 2:48 PM, "Jonathan Lipps" notifications@github.com wrote:

you can always go back to the native context to use that command, so I
would say it doesn't matter whether it works specifically within webviews,
no?


Reply to this email directly or view it on GitHub
#3974 (comment).

@jlipps

This comment has been minimized.

Show comment
Hide comment
@jlipps

jlipps Feb 12, 2015

Member

@wilcollins I see. You can also pass in an "element" parameter to mobile: scroll. This has to be the ID of a native element, of course.

Member

jlipps commented Feb 12, 2015

@wilcollins I see. You can also pass in an "element" parameter to mobile: scroll. This has to be the ID of a native element, of course.

@BhaskarVj

This comment has been minimized.

Show comment
Hide comment
@BhaskarVj

BhaskarVj Feb 14, 2015

Could you please put some code for scroll to an element(not scrollTo("text")), using in java.

@0x1mason @jlipps @Jonahss

BhaskarVj commented Feb 14, 2015

Could you please put some code for scroll to an element(not scrollTo("text")), using in java.

@0x1mason @jlipps @Jonahss

@jlipps

This comment has been minimized.

Show comment
Hide comment
@jlipps

jlipps Apr 29, 2015

Member

the documentation has been updated appropriately, so I'm closing this issue. @bhaskar-appium the correct way to achieve what you want is to use mobile: scroll and then check that the element in question is within the bounds of the viewport.

Member

jlipps commented Apr 29, 2015

the documentation has been updated appropriately, so I'm closing this issue. @bhaskar-appium the correct way to achieve what you want is to use mobile: scroll and then check that the element in question is within the bounds of the viewport.

@jlipps jlipps closed this Apr 29, 2015

@VenkiDeveloper

This comment has been minimized.

Show comment
Hide comment
@VenkiDeveloper

VenkiDeveloper Aug 23, 2016

Try this code, its worked for me...to swipe up/down without using any element

Map<String, Object> params = new HashMap<String, Object>();
params.put("direction", "up/down");
appiumIOSDriver.executeScript("mobile: scroll", params);

VenkiDeveloper commented Aug 23, 2016

Try this code, its worked for me...to swipe up/down without using any element

Map<String, Object> params = new HashMap<String, Object>();
params.put("direction", "up/down");
appiumIOSDriver.executeScript("mobile: scroll", params);

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment