Limit requests per timeframe to each host

[mcarans]

Long story short

Currently I can use limit_per_host to limit the number of simultaneous connections to a host. How can I limit the number of requests per timeframe (eg. minute) to each host?

Expected behaviour

A parameter probably on TCPConnector that sets a limit to the number of requests that can be made to a host in a particular timeframe. I could imagine doing what was done for timeouts and introducing a new class to put it all together eg. ConnectorLimit:

limit = aiohttp.ConnectorLimit(total=100, limit_per_host=1, limit_per_time_per_host=5, time=60)
conn = aiohttp.TCPConnector(limit=limit, loop=loop)

In the above:

• total is the parameter limit from TCPConnector
• limit_per_host is the parameter limit_per_host from TCPConnector
• limit_per_time_per_host and time are new parameters
• time is in seconds
• so with limit_per_time_per_host=5, it means maximum 5 connections can be made to a host in 60 seconds

Actual behaviour

It's not possible right now.

Steps to reproduce

conn = aiohttp.TCPConnector(limit=100, limit_per_host=1, loop=loop)
timeout = aiohttp.ClientTimeout(total=5*60)
async with aiohttp.ClientSession(connector=conn, timeout=timeout, loop=loop) as session:
....

Your environment

aiohttp 3.3.2 client

[asvetlov]

Perhaps it will never happen in aiohttp.
Your request requires significant complication of connector logic but it is not generic enough.
Rate limiting is just out of the project scope.
You can build it on top of aiohttp.

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a [new issue] for related bugs.
If you feel like there's important points made in this discussion, please include those exceprts into that [new issue].
[new issue]: https://github.com/aio-libs/aiohttp/issues/new