Chrome - Link('<link_text>') works in non-headless mode, but doesn't work in headless mode

[tddschn]

I'm using start_chrome() on macOS, Google Chrome, chromedriver and helium are upgraded to the latest (stable) versions.

I have a script with a line

helium.click(helium.Link('<link_text>'))

to click on a hyperlink on a web page, which works perfectly except when headless=True is passed to start_chrome().

I'm not very familiar with the underlying selenium-python framework, so I'm not sure if I ask this question in the right issue tracker. :(

Is this a known problem of helium? Is there any way to make helium work with headless Chrome? Thanks!

Versions:

• Google Chrome: 87.0.4280.88
• chromedriver: 87.0.4280.20
• selenium: 3.141.0
• helium: 3.0.5
• python: 3.8.5

[shravanasati]

I encountered the same issue, in headless mode helium fails to identify elements.

[ttamg]

Hi. Here is code to replicate the same sort of issue. In this case it works for Chrome but not Firefox. I am using Ubuntu 20.04 LTS.

The code below looks for links on the Microsoft home page at the top and bottom of the page:

• All the objects can be found on the page. The helium.find_all() does find the object. So that's good.
• The helium.click() does work when the object is on the page but not when it needs to scroll down. I have found this in Chrome headless mode too. The code below raises a Lookup error in Firefox Headless and non-headless modes only.

# Code to replicate the click issue (on Firefox - Ubuntu)
import helium

HEADLESS = False
URL = "https://www.microsoft.com/en-gb"

driver = helium.start_firefox(URL, headless=HEADLESS)
# driver = helium.start_chrome(URL, headless=HEADLESS)

# Object near top of page so does not require any scroll
obj1 = helium.find_all(helium.Link("Deals"))
print(f"Links found on page - {obj1}")
helium.click(obj1[0])
print(f"Link clicked - now on page URL = {driver.current_url}")

# Reset
driver.get(URL)

# Object near foot of the page so requires a scroll down to click
obj2 = helium.find_all(helium.Link("Investors"))
print(f"Links found on page - {obj2}")
helium.click(obj2[0])
print(f"Link clicked - now on page URL = {driver.current_url}")

helium.kill_browser()

When you run this on Firefox we get a lookup error only at the very end when trying to click the link at bottom of the page.

I had a look at the source and the click method does use a scroll to object but for some reason it doesn't work on Firefox. Line 205 in https://github.com/mherrmann/selenium-python-helium/blob/master/helium/_impl/__init__.py.

I was struggling to debug much beyond this but hopefully it helps someone who understands Selenium to dig a bit deeper.

[danbaghilovici]

This may be a problem with selenium . I remember that they had problems identifying the elements when wrapped around css pseudo selectors. From @ttamg example, the footer navigation links from microsoft seem to be inside an ::after and ::before.

I will try to test to see if that's the issue.

[myii]

Hitting this issue on Firefox as well and groped around for any way to workaround it. This is an ugly hack but it gets the job done both headless or otherwise:

    if use_search_to_scroll:
        press('/')
        write(link_text)
    click(link_text)

Where:

• use_search_to_scroll is used to run the hack only for the links that may need to be scrolled to.
• link_text is the link text that needs to be clicked.

Look forward to a proper solution so that I can eventually remove this nasty hack!

[mherrmann]

Look forward to a proper solution so that I can eventually remove this nasty hack!

I am always happy to receive Pull Requests :-)

[myii]

I am always happy to receive Pull Requests :-)

@mherrmann I've found a cleaner workaround, I'm wondering whether it's something that could be used as a solution.

So this instead of the snippet above:

    scroll_down(Link(link_text).web_element.location['y'])

• Single-liner, no need for the if block.

In fact, because my site has a navigation bar and breadcrumbs, I had to adjust for those (simplifying my own solution a bit):

    scroll_down(Link(link_text).web_element.location['y']) - 100

• I suppose it would be better to scroll enough so that the link is in the centre of the window but I haven't figured out how to reference the window height yet.
• Link(link_text) can be adjusted to deal with the type of element that needs to be clicked on, of course.

In terms of the solution, is .web_element.location['y'] reliable enough?