Deprecate schemeless URLs?

[kenballus]

Context

Schemeless URLs can cause a lot of parsing confusion because they aren't standardized in the RFC. Examples like the following are especially weird, and could cause problems when interoperability with other parsers matters:

"evil.com://good.com"

Here are some popular URL parsers' interpretations:
urllib3:

Scheme:   (nil)
Userinfo: (nil)
Host:     evil.com
Port:     (nil)
Path:     //good.com
Query:    (nil)
Fragment: (nil)

cpython urllib:

Scheme:   evil.com
Userinfo: (nil)
Host:     good.com
Port:     (nil)
Path:     (nil)
Query:    (nil)
Fragment: (nil)

furl:

Scheme:   evil.com
Userinfo: (nil)
Host:     good.com
Port:     (nil)
Path:     (nil)
Query:    (nil)
Fragment: (nil)

hyperlink:

Scheme:   evil.com
Userinfo: (nil)
Host:     good.com
Port:     (nil)
Path:     /
Query:    (nil)
Fragment: (nil)

rfc3986:

Scheme:   evil.com
Userinfo: (nil)
Host:     good.com
Port:     (nil)
Path:     (nil)
Query:    (nil)
Fragment: (nil)

yarl:

Scheme:   evil.com
Userinfo: (nil)
Host:     good.com
Port:     (nil)
Path:     /
Query:    (nil)
Fragment: (nil)

As you can see, we're the outlier here.

In my opinion, this is something worth fixing, but I imagine that schemeless URLs are in pretty widespread use with urllib3. Thus, we might consider adding a DeprecationWarning that encourages people to explicitly state their schemes.

[sethmlarson]

Totally agree that this is something worth fixing, thank you for doing all this research and opening this. My hope is that schemeless URLs aren't as widespread as we think and that we've been supporting this mostly for backwards compatibility and not because a large swath of the ecosystem relies on it.

I think adding a DeprecationWarning as you mentioned that links to this issue (we can edit the issue body to provide actionable feedback for users) and says that a removal will occur in a future version. We can then get that released in 1.26.x and 2.x to start warning users and gathering feedback if necessary.

[sigmavirus24]

In my experience these often occur in the Location header and require resolution per the specifications. I don't remember how urllib3 does redirect handling but this may be required to support that

[DarkPhily]

Hi,
I'm interested in contributing.
I would like to give this issue a try, but I'm not sure what should be accomplished at the end?
Raise an DeprecationWarning?
Change the parsing to be more inline with other libs?

Some guidance would be much appreciated.

[Mr-Sunglasses]

Hey @kenballus, @sethmlarson Is this issue available for work on?

[JohnJamesUtley]

@kenballus and I are working on this right now

[JohnJamesUtley]

I removed support for schemeless URLs in a branch from my fork:
https://github.com/JohnJamesUtley/urllib3/tree/remove_schemeless_urls

Let me know if you want me to make a pull request

Edit: Based off of and checked that it conforms with RFC 3986 specifications

[sethmlarson]

Closed in #2950

[sethmlarson]

Thanks @Ousret! I've approved your expense in OpenCollective for $300