[🚀 Feature]: [py] Validate URL's before navigation

[cgoldberg]

Description

When you call driver.get() or driver.browsing_context.navigate()), it attempts to navigate to a URL, even if the URL is malformed.

Browsers don't handle this very well. For example...

if you do:

driver.get("example.com")

or

driver.get("http//example.com")

Chrome will just not navigate and not return any error (Firefox returns an error).

Proposed change:

If we validate the URL before attempting navigation, we can raise a useful exception: raise InvalidArgumentException("Invalid URL").

Here is some example code for validation:

from urllib.parse import urlparse

def is_valid_url(url):
    try:
        result = urlparse(url)
        return bool(result.scheme)
    except AttributeError:
        return False

This validates it can be parsed as a URL and contains a scheme.

Have you considered any alternatives or workarounds?

No response

Does this apply to specific language bindings?

Python

What part(s) of Selenium does this relate to?

No response

[selenium-ci]

@cgoldberg, thank you for creating this issue. We will troubleshoot it as soon as we can.

Selenium Triage Team: remember to follow the Triage Guide

[titusfortner]

This is what the spec says:

If URL is not an absolute URL or is not an absolute URL with fragment or not a local scheme, return error with error code invalid argument.

Technically users can create valid local schemes of their own that aren't in VALID_URL_SCHEMES so I don't think we should prevent those. Verify fragments (https://www.example.com/documentation.html#installation) and queries (https://www.example.com/documentation.html?foo=bar) pass the parse (I suspect they do)

[cgoldberg]

users can create valid local schemes of their own that aren't in VALID_URL_SCHEMES

So I guess we can't validate the URL scheme.. maybe it should just try to parse the URL and verify it has a scheme and netloc and let everything else through.

The validation would just be:

def is_valid_url(url):
    try:
        result = urlparse(url)
        return all([result.scheme, result.netloc])
    except AttributeError:
        return False

[emanlove]

about:blank is a valid url within Chrome but fails the check above.

[cgoldberg]

@emanlove thanks.. you're right.

We could use:

def is_valid_url(url):
    try:
        result = urlparse(url)
        return bool(result.scheme)
    except AttributeError:
        return False

... that's not a lot of validation, but it would save users from being confused when driver.get(example.com) doesn't navigate or raise an exception in Chrome/Edge.

I'm not sure this is even worth doing though.

[rpallavisharma]

I think it would be better to leave that decision to end user implementing selenium.
But this could make a good blog or knowledge article for community.

[shbenzer]

Now that we’re moving to bidi default in Selenium 5 we could possibly check for error codes in driver .get()