Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

aiohttp.ClientSession incorrectly decoding special characters (parentheses) before making request #7341

Closed
1 task done
piotr-gregrowski opened this issue Jul 6, 2023 · 2 comments
Labels

Comments

@piotr-gregrowski
Copy link

piotr-gregrowski commented Jul 6, 2023

Describe the bug

The url that I pass into the .get() function is not being handled correctly when there are parentheses in the path. This is causing issues when trying to download documents that use the standard naming convention for copies:
my_document (1).pdf

The expected url for that document download would look something like:
https://python.org/my_document%20%281%29.pdf
with the following encodings:

  • space -> %20
  • left parenthesis ( -> %28
  • right parenthesis ) -> %29

Instead it is being decoded at some point and ends up looking like this:
https://www.python.org/document%20(1).pdf

To Reproduce

async with aiohttp.ClientSession() as session:
        async with session.get('http://python.org/my_document%20%281%29') as response:
                print(response.request_info)

> RequestInfo(url=URL('https://www.python.org/my_document%20(1).pdf'), method='GET', headers=<CIMultiDictProxy('Host': 'www.python.org', ...)>, real_url=URL('https://www.python.org/document%20(1).pdf'))

Expected behavior

async with aiohttp.ClientSession() as session:
        async with session.get('http://python.org/my_document%20%281%29') as response:
                print(response.request_info)

> RequestInfo(url=URL('https://www.python.org/my_document%20%281%29.pdf'), method='GET', headers=<CIMultiDictProxy('Host': 'www.python.org', ...)>, real_url=URL('https://www.python.org/document%20%281%29.pdf'))

Logs/tracebacks

See above

Python Version

$ python --version

Python 3.11.1

aiohttp Version

$ python -m pip show aiohttp

Name: aiohttp
Version: 3.8.4
Summary: Async http client/server framework (asyncio)
Home-page: https://github.com/aio-libs/aiohttp
Author: 
Author-email: 
License: Apache 2
Location: [redacted}
Requires: aiosignal, async-timeout, attrs, charset-normalizer, frozenlist, multidict, yarl
Required-by: aiobotocore, aioresponses

multidict Version

$ python -m pip show multidict

Name: multidict
Version: 6.0.2
Summary: multidict implementation
Home-page: https://github.com/aio-libs/multidict
Author: Andrew Svetlov
Author-email: andrew.svetlov@gmail.com
License: Apache 2
Location: [redacted]
Requires: 
Required-by: aiohttp, yarl

yarl Version

$ python -m pip show yarl

Name: yarl
Version: 1.8.1
Summary: Yet another URL library
Home-page: https://github.com/aio-libs/yarl/
Author: Andrew Svetlov
Author-email: andrew.svetlov@gmail.com
License: Apache 2
Location: [redacted]
Requires: idna, multidict
Required-by: aio-pika, aiohttp, aiormq

OS

macOS

Related component

Client

Additional context

No response

Code of Conduct

  • I agree to follow the aio-libs Code of Conduct
@Dreamsorcerer
Copy link
Member

Passing encoded=True parameter prevents URL auto-encoding, user is responsible about URL correctness.
https://yarl.aio-libs.org/en/stable/api.html#yarl.URL

I'm going to update the aiohttp docs with a note, because this gets asked semi-regularly.

@Dreamsorcerer Dreamsorcerer closed this as not planned Won't fix, can't repro, duplicate, stale Jul 7, 2023
@piotr-gregrowski
Copy link
Author

piotr-gregrowski commented Jul 7, 2023

Thanks @Dreamsorcerer! Sorry, I could've been more diligent in reading through the other issues

For anyone else that comes across this, I solved my issue using the link above:

async with http_session.get(yarl.URL(my_url, encoded=True)) as response:

webknjaz pushed a commit that referenced this issue Jul 9, 2023
patchback bot pushed a commit that referenced this issue Jul 9, 2023
Ref #7341

(cherry picked from commit 1805758)
patchback bot pushed a commit that referenced this issue Jul 9, 2023
Ref #7341

(cherry picked from commit 1805758)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants