Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Limit requests per timeframe to each host #3184

Closed
mcarans opened this issue Aug 10, 2018 · 2 comments
Closed

Limit requests per timeframe to each host #3184

mcarans opened this issue Aug 10, 2018 · 2 comments
Labels

Comments

@mcarans
Copy link

@mcarans mcarans commented Aug 10, 2018

Long story short

Currently I can use limit_per_host to limit the number of simultaneous connections to a host. How can I limit the number of requests per timeframe (eg. minute) to each host?

Expected behaviour

A parameter probably on TCPConnector that sets a limit to the number of requests that can be made to a host in a particular timeframe. I could imagine doing what was done for timeouts and introducing a new class to put it all together eg. ConnectorLimit:

limit = aiohttp.ConnectorLimit(total=100, limit_per_host=1, limit_per_time_per_host=5, time=60)
conn = aiohttp.TCPConnector(limit=limit, loop=loop)

In the above:

  • total is the parameter limit from TCPConnector
  • limit_per_host is the parameter limit_per_host from TCPConnector
  • limit_per_time_per_host and time are new parameters
  • time is in seconds
  • so with limit_per_time_per_host=5, it means maximum 5 connections can be made to a host in 60 seconds

Actual behaviour

It's not possible right now.

Steps to reproduce

conn = aiohttp.TCPConnector(limit=100, limit_per_host=1, loop=loop)
timeout = aiohttp.ClientTimeout(total=5*60)
async with aiohttp.ClientSession(connector=conn, timeout=timeout, loop=loop) as session:
....

Your environment

aiohttp 3.3.2 client

@mcarans mcarans changed the title Limit requests per timeframe to each domain Limit requests per timeframe to each host Aug 10, 2018
@asvetlov
Copy link
Member

@asvetlov asvetlov commented Aug 10, 2018

Perhaps it will never happen in aiohttp.
Your request requires significant complication of connector logic but it is not generic enough.
Rate limiting is just out of the project scope.
You can build it on top of aiohttp.

@lock
Copy link

@lock lock bot commented Oct 28, 2019

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a [new issue] for related bugs.
If you feel like there's important points made in this discussion, please include those exceprts into that [new issue].
[new issue]: https://github.com/aio-libs/aiohttp/issues/new

@lock lock bot added the outdated label Oct 28, 2019
@lock lock bot locked as resolved and limited conversation to collaborators Oct 28, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

No branches or pull requests

2 participants