Checkbox sometimes not checked after check …

[henrik]

I realise this may be hard to do anything about without a repro, but I thought I'd mention it on the off-chance that the maintainer(s) can think of an explanation, or at least in case anyone else runs into it in the future.

This happens on a fairly complex page in our app, and I can't currrently take the time to try to make a minimal repro, but the nut of it is that if I do

# Other code…

check "My checkbox"
expect(find_field("My checkbox").checked?).to be true

Then sometimes the expectation fails.

When it fails, screenshots are completely white. When it passes, I can see the checked checkbox in the screenshot.

Adding a sleep 0.1 before checking the box seems to consistently fix it. Adding a sleep 0.01 fixes it some of the time. With no sleep, it fails almost every time. So clearly there's a timing thing going on.

The preceding code in the test changes two other forms on the same page and submits them with Ajax. The page is not rewritten much by those submissions – it only toggles some classes on those other forms. If I remove either one (or both) of the two Ajax submissions, it starts passing without the sleep.

Happy to screen share with a maintainer, but again, afraid I won't be able to take the time to make a distributable or minimal repro out of this :/

[twalpole]

From the description it sounds like your page is dynamically modifying/adding behavior to the checkbox (possibly react/angular?). If the behavior hasn't fully been added before you tell Capybara to check the box then the box may get reset by the JS and/or the expected behavior may not trigger. It sounds like you need to have an expectation for whatever visible change on your page would indicate the page is actually ready for use before calling check. Other than that without a repo there's not much I can do.

Additionally the fact the screenshot is white could imply you've set Capybara.ignore_hidden_elements = false - if you have then stop doing that.

[henrik]

@twalpole Thank you again!

The page is just server-rendered with very light JS to add/remove CSS classes. Those CSS classes don't style the checkbox other than changing the opacity of its container. And I still get the issue if I stop doing those styling changes.

We do have ignore_hidden_elements = false for some reason. Will look into getting rid of that (agree we shouldn't have it). Could you say more about why that could cause white screenshots? Hidden elements pushing visible content outside the viewport?

Wonder if it could be a timing issue where content loads/shifts so the click … call thinks it has clicked something, but the checkbox had just moved out of the way. Is that theoretically possible? Don't know if Apparition triggers click events on the specific element or on a coordinate or what.

[henrik]

I tried setting ignore_hidden_elements to true, removed the sleep and re-ran the test. Saw it fail again, and with an all-white screenshot. So that doesn't seem to be the culprit in this case.

Just in case it has any bearing on things, I noticed that even when the test passes, the screenshot has the top part of the page occluded by a box with the same colour as the body background. I made the background red just to make sure it was the body bg. This image shows the HTML page on left and the screenshot on right:

[twalpole]

Having ignore_hidden_elements = false can end up with white screenshots if the page is keeping everything hidden to prevent flash of styles. When ignore_hidden_elements = true then Capybara waits for the elements to be non-hidden when finding them which would imply the page has loaded everything and removed the hiding. It definitely could be an issue of objects moving (generally best to disable animation when testing if possible) since clicks are at a location. From that image, do you have an overlay which is sliding up off the page? If so set an expectation on it's style (have_style) or for it to not exist (have_no_css/xpath/etc) in the page depending on what happens to it.

[henrik]

Thank you again! I was thinking elements moving not so much from animations, as web fonts or images loading and shifting the page.

We don't have an overlay that I'm aware of, and the weird thing is that this is the body background colour. Changing the body bg colour (e.g. to red above) changes the colour of the "overlay". So it doesn't seem to actually be an overlay but more that the top part of the page is somehow unrendered and the bg colour shows through.

[twalpole]

@henrik Or you do have an overlay that is dynamically using the background color (or are animating the form into place)?? It's really not possible that the browser is drawing the page from the bottom up and hasn't rendered the top yet, so for that image to exist you have an overlay /animation happening (99% sure).

[henrik]

@twalpole Hm, that's an idea. If I find the time I'll grep around and/or try removing code to see if it is some kind of overlay/animation after all. Thank you! Really appreciate you taking the time.

[jordoh]

I've been running into a similar issue with an image-heavy page, where click_link on a link with javascript-based effects intermittently results in an unexpected state. Adding console.error calls to the page showed the failing cases either clicking on a different link or not clicking on any link at all.

The skip_image_loading option resulted in fewer test failures, but the page also has webfonts that can cause it to shift around. Observing document.readyState around the time of the click_link call showed that the page was in fact changing from "interactive" to "complete" right around the time of the click_link call, though not with a direct correlation to the failures.

I've eventually worked around this by adding a method to wait for the document to be complete. This is essentially a more sophisticated version of the sleep 0.01 you describe, but is patterned after Capybara's synchronize and apparition's wait_for_loaded:

def wait_for_document_load_complete
  timer = Capybara::Helpers.timer(expire_in: Capybara.default_max_wait_time)

  begin
    return if page.driver.evaluate_script("document.readyState") == "complete"
    sleep 0.01
  end until timer.expired?

  raise TimeoutError.new("wait_for_document_load_complete")
end

Calling this immediately before the first click_link after a visit call has completely eliminated the intermittent mis-click failures for me.