Skip to content

BaseConnector limit and limit_per_host don't work with tracing enabled #5259

Closed
@bobh66

Description

🐞 Describe the bug
The BaseConnector limit and limit_per_host attributes don't function properly when tracing is enabled and the connection is being reused. BaseConnector.connect() calls _available_connections(), and when it gets back a number > 0, it will await any configured traces before acquiring the connection. That is enough time for another thread to jump in and acquire the same connection, allowing more than the configured number of connections to exist simultaneously. connect() needs to acquire the connection, possibly with a placeholder, before awaiting the traces.

💡 To Reproduce

  1. Configure a limit and/or limit_per_host on a TCPConnection
  2. Configure tracing for on_connection_reuseconn()
  3. Queue up a large number of requests that exceed the limits specified
  4. Observe that more than the configured limit number of queries are run simultaneously

💡 Expected behavior
There should not be more than the configured limit number of connections active at any given time

📋 Logs/tracebacks

📋 Your version of the Python

$ python --version
Python 3.7.3
...

📋 Your version of the aiohttp/yarl/multidict distributions

$ python -m pip show aiohttp
Name: aiohttp
Version: 3.6.2
Summary: Async http client/server framework (asyncio)
Home-page: https://github.com/aio-libs/aiohttp
Author: Nikolay Kim
Author-email: fafhrd91@gmail.com
License: Apache 2
Location: /usr/local/lib/python3.7/site-packages
Requires: chardet, yarl, attrs, async-timeout, multidict
Required-by: spe, faust, bravado-asyncio, aiohttp-cors
$ python -m pip show multidict
Name: multidict
Version: 4.7.5
Summary: multidict implementation
Home-page: https://github.com/aio-libs/multidict
Author: Andrew Svetlov
Author-email: andrew.svetlov@gmail.com
License: Apache 2
Location: /usr/local/lib/python3.7/site-packages
Requires: 
Required-by: yarl, aiohttp
$ python -m pip show yarl

Name: yarl
Version: 1.4.2
Summary: Yet another URL library
Home-page: https://github.com/aio-libs/yarl/
Author: Andrew Svetlov
Author-email: andrew.svetlov@gmail.com
License: Apache 2
Location: /usr/local/lib/python3.7/site-packages
Requires: idna, multidict
Required-by: faust, aiohttp

📋 Additional context

client

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions