xmlrpc.client ignores query in URI ("?action=xmlrpc2") since python-3.9

[optak]

BPO	43433
Nosy	@scoder, @vstinner, @kynan, @Rosuav, @serhiy-storchaka, @corona10, @miss-islington, @optak
PRs	bpo-43433: Reattach query parameters after parsing URI #25045 bpo-43433: Preserve query and fragment in the URL of the server in ServerProxy. #25057 [3.9] bpo-43433: Preserve query and fragment in the URL of the server in ServerProxy. (GH-25057) #25068

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/serhiy-storchaka'
closed_at = <Date 2021-03-29.20:35:34.832>
created_at = <Date 2021-03-08.12:29:16.590>
labels = ['expert-XML', 'type-bug', '3.9', '3.10']
title = 'xmlrpc.client ignores query in URI ("?action=xmlrpc2") since python-3.9'
updated_at = <Date 2021-03-29.20:35:34.832>
user = 'https://github.com/optak'

bugs.python.org fields:

activity = <Date 2021-03-29.20:35:34.832>
actor = 'vstinner'
assignee = 'serhiy.storchaka'
closed = True
closed_date = <Date 2021-03-29.20:35:34.832>
closer = 'vstinner'
components = ['XML']
creation = <Date 2021-03-08.12:29:16.590>
creator = 'OndrejPtak'
dependencies = []
files = []
hgrepos = []
issue_num = 43433
keywords = ['patch']
message_count = 8.0
messages = ['388263', '388830', '389621', '389678', '389703', '389705', '389730', '389740']
nosy_count = 9.0
nosy_names = ['scoder', 'vstinner', 'eli.bendersky', 'kynan', 'Rosuav', 'serhiy.storchaka', 'corona10', 'miss-islington', 'OndrejPtak']
pr_nums = ['25045', '25057', '25068']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue43433'
versions = ['Python 3.9', 'Python 3.10']

[optak]

xmlrpc.client proxy behaviour changed and broke tools depending on URI containing query part.

Last working version: https://github.com/python/cpython/blob/3.8/Lib/xmlrpc/client.py#L1417

Changed behaviour here: https://github.com/python/cpython/blob/3.9/Lib/xmlrpc/client.py#L1424

Is this change intended? If so, what is recommended solution for xmlrpc client communicating with URI: http://example.com/path?var=foo ?

[optak]

Reproducer:

• create ServerProxy with UPI containing query part, e.g. http://example.com?foo=bar
• communicate

Old behavior:
-------------
send: b'POST /?foo=bar HTTP/1.1\r\nHost: example.com\r\nAccept-Encoding: gzip\r\nContent-Type: text/xml\r\nUser-Agent: Python-xmlrpc/3.8\r\nContent-Length: 197\r\n\r\n'
send: b"<?xml version='1.0'?>\n<methodCall>\n<methodName>getRecentChanges</methodName>\n<params>\n<param>\n<value><dateTime.iso8601>20210301T00:00:00</dateTime.iso8601></value>\n</param>\n</params>\n</methodCall>\n"

New behaviour:
--------------
send: b'POST / HTTP/1.1\r\nHost: example.com\r\nAccept-Encoding: gzip\r\nContent-Type: text/xml\r\nUser-Agent: Python-xmlrpc/3.9\r\nContent-Length: 197\r\n\r\n'
send: b"<?xml version='1.0'?>\n<methodCall>\n<methodName>getRecentChanges</methodName>\n<params>\n<param>\n<value><dateTime.iso8601>20210301T00:00:00</dateTime.iso8601></value>\n</param>\n</params>\n</methodCall>\n"

[Rosuav]

Can confirm. This changed in bpo-38038 and the fix isn't too difficult.

Adding 3.10 in case it's decided that this shouldn't be patched onto 3.9.

[serhiy-storchaka]

Thank you for your reports OndrejPtak and frathgeber.

Indeed, the behavior change was unintended. PR 25057 tries to to restore the old behavior. Query and fragment are now preserved in the server URL.

But there are still minor differences. Empty query and fragment are stripped from the URL, so "http://example.org/RPC2?", "http://example.org/RPC2#" and "http://example.org/RPC2" refer to the same connection point. I am not sure that it is important. It would be not easy to preserve the old behavior in these corner cases.

[corona10]

New changeset c1b073a by Serhiy Storchaka in branch 'master':
bpo-43433: Preserve query and fragment in the URL of the server in ServerProxy. (GH-25057)
c1b073a

[miss-islington]

New changeset 5334605 by Miss Islington (bot) in branch '3.9':
bpo-43433: Preserve query and fragment in the URL of the server in ServerProxy. (GH-25057)
5334605

[kynan]

Thanks Serhiy for the very quick fix! I believe handling these corner cases is not worth the effort and I really hope no one relies on these being differentiated. This change will go into a future 3.9 release?

[vstinner]

It seems like the issue is now fixed. Thanks Serhiy for the fix and thanks OndrejPtak for the report!

This change will go into a future 3.9 release?

Yes, sure. Python 3.9 Release Schedule:
https://www.python.org/dev/peps/pep-0596/

Next expected release: Python 3.9.3: Monday, 2021-05-03.