Elements get invalidated in UIATableView

[bernii]

Tables seem to be kinda specific in iOS apps. They get invalidated very fast making Appium element-interaction process fail. This is due the fact that for example 'tap on element' takes two requests to perform it:

1. Find an element
2. Tap on the element

This two-step process of clicking is just not working (cause elements get wiped between two requests: getElement and click). To make it work, there should be a feature available to execute instructions in one step. Same goes with getting attributes of elements... (they might be not there when you request for attribute of an element).

The only workaround (for tapping) it performing tap on screen coordinates. But there are no workarounds for other scenarios (like getting an attribute of an element).

[gaige]

Can you provide an example driving any of the example apps in appium? I don't think I've encountered this brittleness testing our apps, including ones that have a lot of tables in them. We are able to find an element and click on it without trouble.

For example, we do the following (python) in a number of places:
# add field
fieldTable = self.driver.find_element_by_tag_name("tableView")
createButton = fieldTable.find_elements_by_tag_name("tableCell")[0]
self.assertEqual( createButton.get_attribute("name"),"Add Field")
createButton.click()

Note that despite the fact that I use the variable "createButton", it's actually just a cell, not a button within the cell, in this case.

What kind of errors are you seeing?

[bootstraponline]

there should be a feature available to execute instructions in one step.

That was mentioned on #174.

"constructing a chain of commands to be executed all at one time, rather than executing each one along the way"

[jlipps]

Along with @gaige I'm not seeing this happening in the UICatalog app, maybe because the table contents are not dynamically generated? Any way @bernii or others that I can get an example app that has this problem so I can write tests against it as I'm coming up with a fix?

[penguinho]

Any developments on this? This one is breaking my stuff really bad.

[bootstraponline]

I think we're waiting on a way to reproduce the problem.

[gaige]

@penguinho If you can get it to reproduce in a way that we can reproduce it, we might be able to help. Best case would be to reproduce with UICatalog app.

What exactly are you doing that exhibits the behavior?
Is it 100% consistent? 90%? 50%?

Looking at UICatalog vs. what you're doing in your app, do you see any obvious differences? Things that might perturb the element ids

Note: I'm not doubting that it happens. However, I can't even start looking into it with any reliability unless I have some reasonable expectation that I can make it happen (since that'd be required to try and verify a fix).

[penguinho]

I have a 100% repro, on a ton of different elements in our edit profile control.

UITableViewCell.contentView with 2 subviews (UILabel, and UITextField) I am trying to sendkeys to the UITextField

[penguinho]

If someone's in the Bay I can drop by and show them our app (unfortunately its pretty difficult to rip out the source)

[bootstraponline]

Couldn't you submit a simulator build to Sauce Labs? They might sign a NDA.

[penguinho]

Can I get conf from Sauce on the NDA?

[penguinho]

I'd have to check with iOS team, but I think I'd be able to drop a build under those circumstances.

[jlipps]

OK, I'm checking on it.

On Mar 4, 2013, at 12:42 PM, Dan Cuellar notifications@github.com wrote:

I'd have to check with iOS team, but I think I'd be able to drop a build under those circumstances.

—
Reply to this email directly or view it on GitHub.

[penguinho]

vetoed on my end, but team offerered to try to build a demo app for me. I'll let you know when it's done. They said they try to tackle it tonight

[penguinho]

Got a dummy app with a repro. https://dl.dropbox.com/u/46918906/tableViewTest.zip

wrapped C# code to repro (convert to your native tongue)

ERROR:
Client.SetValue(By.Name("TextFieldLabelForSection3Row3"), "Testing");

NO ERROR:
Client.SetValue(By.Name("TextFieldLabelForSection0Row0"), "Testing");

[gaige]

Interesting. I loaded up the app and pointed python at it. I can do the following:

driver = webdriver.Remote('http://127.0.0.1:4723/wd/hub', desired_capabilities={ 'browserName':'ios', 'platform':'Mac', 'app':app})
elems = driver.find_elements_by_tag_name("textField")
elems[2].send_keys('hi')
elems[3].send_keys('hi')
elems[10].send_keys('hi')
elems[12].send_keys('hi') 
elems[15].send_keys('hi')
elems[18].send_keys('hi')
elems[30].send_keys('hi')

and all but the [30] works. If I then invoke one of the previous commands (elems[2].send_keys('hi')), it still works, but any reference above 20 fails.

Hmmm, I have an idea....

[gaige]

OK, take a look at your cell construction in the example code:

• dequeue a cell if it's available
• if it is available, then use it (including all of the sub-elements, such as the cell and textField) without resetting the accessibility label
• if it is not available, create a new item and set the accessibility label.

If you don't reset the accessibility label, the reused cells are going to have the previous accessibility labels. It'll work fine for anything displayed on the screen originally, but won't work at all for anything after the first cell is reused, such as the 3rd segment.

By moving the accessibility setters below the cell reuse, it makes things a little bit better. However, whenever the cells are reused, it appears that we get an inconsistency, which requires things like requesting the element more than once.

For example: if I find "TextFieldLabelForSection5Row3" and then send a click to it (tap), the screen scrolls the item into view and then aborts the tap. I'm thinking this is because the element itself was only temporarily created while it was offscreen for evaluation of the accessibility label. If I then find the element again (exact same name), and the next click (or send keys, or whatever) seems to work correctly.

It would appear that if we could hold a reference a little differently that we might be able to find the accessibility element again without having to "tap twice" like this (and without the exception on the first one that brings it into view). Similarly, if we had a separate "scroll into view" function, we could check for is_displayed(), then scroll the element into view, re-find it, and use that.

I had reasonable success (once I made the change above for the accessibility elements) by trapping the exception and scrolling the item into view and then re-finding it.

[penguinho]

With those changes I am able to setValue now, but I have to issue an ExecuteScript with "target.frontMostApp().mainWindow().tableViews()["Empty list"].cells()["CellLabelForSection3Row3"].scrollToVisible()" first

Can I scroll through appium over JSON-wire instead? What's the command.

However, this doesn't fix the problem on the app we actually have as the elements are onscreen when the error occurs

[jlipps]

you can use a swipe command to scroll

On Mar 7, 2013, at 9:58 AM, Dan Cuellar notifications@github.com wrote:

With those changes I am able to setValue now, but I have to issue an ExecuteScript with "target.frontMostApp().mainWindow().tableViews()["Empty list"].cells()["CellLabelForSection3Row3"].scrollToVisible()" first

Can I scroll through appium over JSON-wire instead? What's the command.

However, this doesn't fix the problem on the app we actually have as the elements are onscreen when the error occurs

—
Reply to this email directly or view it on GitHub.

[jlipps]

and scrollToElement shouldn't be hard to implement on the appium side either. is there a jsonwp equivalent command?

On Mar 7, 2013, at 9:58 AM, Dan Cuellar notifications@github.com wrote:

With those changes I am able to setValue now, but I have to issue an ExecuteScript with "target.frontMostApp().mainWindow().tableViews()["Empty list"].cells()["CellLabelForSection3Row3"].scrollToVisible()" first

Can I scroll through appium over JSON-wire instead? What's the command.

However, this doesn't fix the problem on the app we actually have as the elements are onscreen when the error occurs

—
Reply to this email directly or view it on GitHub.

[gaige]

I couldn't find a jsopwp equivalent for scrollToElement, but it would definitely be useful, especially with the iOS penchant for reuse of cells.

[gaige]

@penguinho Can you provide an example of this?

However, this doesn't fix the problem on the app we actually have as the elements are onscreen when the error occurs

I was unable to replicate the problem with any on-screen element as long as they were not being reused by iOS (i.e. scrolled in or out of the view).

If you can point me in the direction, I'll be happy to look at it further.

[penguinho]

The problem is not in the dummy app, it's in the actual app the dummy app was trying to mimic.

[gaige]

OK, good luck. If they haven't already, they should take a close look at the lifespan of the accessibility labels. One way to take a look at this closely can be to run a GUI probe on the mac running the simulator (UIBrowser.app or similar) and see what the labels are.

If you can reproduce this further, I'll be happy to take a look at it.

[jlipps]

Everyone this affects, please check out #270 and look at the tests to see how this functionality works. Please tell me if it addresses the issue here so I can close this ticket! Looking at you @penguinho

[jlipps]

@penguinho any update on whether using findAndAct resolves your issue?

[penguinho]

haven't tried yet, I'll let you know.

[jlipps]

since findAndAct is the best I think we can do and since it is seemingly well-tested, I'm going to go ahead and close things. feel free to re-open this if you find it doesn't work.

[npenkov]

Still the problem with invalidated cells exists, but the method findAndAct is removed from controller with commit 50c3dbd. Is there a reason why this method is first deprecated and then removed? Can you provide an alternative way of executing operation together with find? We have an application that is very dynamic in creating the UI, respectively we intensively use Tables with cells.

[jlipps]

findAndAct never really worked that well, I got a lot of the invalid cell failures even with that. I'd recommend using execute() with UIAutomation javascript directly.

[npenkov]

Thanks @jlipps , with UIAutomation works like charm (and much faster).

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.