Bug with merging base_url with relative path if colon present in path #1498
Comments
|
@gtors Thanks for reporting :-) Indeed, we've got an issue with interpreting colons when merging URLs: >>> import httpx
>>> httpx.Request("GET", "https://example.org/sub:path").url.raw
(b'https', b'example.org', None, b'/sub:path') # ✅
>>> httpx.Client(base_url="https://example.org").build_request("GET", "/sub:path").url.raw
(b'sub', b'', None, b'path') # ❌
>>> httpx.Client(base_url="https://example.org").build_request("GET", "sub:path").url.raw
(b'sub', b'', None, b'path') # ✅ (Surprising, but in line with RFC 3986: `http:path` is treated as an authority-less HTTP URL)
>>> httpx.Client(base_url="https://example.org").build_request("GET", "/sub%3Apath").url.raw
(b'https', b'example.org', None, b'/sub%3Apath') # ✅
>>> httpx.Client(base_url="https://example.org").build_request("GET", "/ok").url.raw
(b'https', b'example.org', None, b'/ok') # ✅In 0.16.1 we get: >>> import httpx
>>> httpx.Request("GET", "https://example.org/sub:path").url.raw
(b'https', b'example.org', None, b'/sub:path') # ✅
>>> httpx.Client(base_url="https://example.org").build_request("GET", "/sub:path").url.raw
(b'https', b'example.org', None, b'/sub%3Apath') # ✅
>>> httpx.Client(base_url="https://example.org").build_request("GET", "sub:path").url.raw
(b'sub', b'', None, b'path') # ✅ (Surprising, but in line with RFC 3986: `http:path` is treated as an authority-less HTTP URL)
>>> httpx.Client(base_url="https://example.org").build_request("GET", "/sub%3Apath").url.raw
(b'https', b'example.org', None, b'/sub%3Apath') # ✅
>>> httpx.Client(base_url="https://example.org").build_request("GET", "/ok").url.raw
(b'https', b'example.org', None, b'/ok') # ✅So indeed, it looks like something like In test form: client = httpx.Client(base_url="https://www.example.com")
request = client.build_request("GET", "/sub:path")
assert request.url == "https://www.example.com/sub%3Apath"Raises: E AssertionError: assert URL('sub:path') == 'https://www.example.com/sub%3Apath'
E + where URL('sub:path') = <Request('GET', 'sub:path')>.urlSeems to come from this operation, where we strip Lines 328 to 331 in 0f280af Leading to joining Lines 399 to 403 in 8696405 Refs #1407, which is new for 0.17.0. Not sure yet how to resolve, or if we don't just want to disallow |
florimondmanca
added
the
user-experience
|
cc @tomchristie — in case you've got any thoughts on this. |
|
I guess the first step here is a failing test case. I think the following change would work based on a bit of testing in the console. merge_url = URL(url)
if merge_url.is_relative_url:
# When merging against a base URL, we append the path.
# In order to do so we always ensure the base_url paths include the trailing '/',
# and always strip any leading '/' from the merge URL.
merged_raw_path = self.base_url.raw_path + merge_url.raw_path.lstrip(b"/")
return self.base_url.copy_with(raw_path=merged_raw_path)
return merge_url |
Checklist
master.Describe the bug
After upgrading to
httpx 0.17.0fromhttpx 0.16.1, thebase_urlmerge with relative paths containing a colon was broken.To reproduce
also, calling the
getmethod leads to the following exception:Expected behavior
client.build_request("POST", "/sub:path")returns<Request('POST', 'https://example.org/some/path/sub%3Apath')>client.get("/sub:path")not raises UnsupportedProtocol exceptionActual behavior
client.build_request("POST", "/sub:path")returns<Request('POST', 'sub:path')>client.get("/sub:path")raises UnsupportedProtocol exceptionEnvironment
The text was updated successfully, but these errors were encountered: