Global or local instance of the Limiter?

[phillipuniverse]

I'm trying to understand how I need to use / re-use Limiter instances and I'm not quite understanding the behavior. I wrote a little sample multithreading script to evaluate the behavior.

I'm using version 3.6.1.

I'm using Django but it shouldn't be that material to what's going on.

Here's my code:

import threading
import unittest

from pyrate_limiter import Duration, Limiter, Rate, RedisBucket, AbstractClock
from redis import Redis


def api_call(attempt):
    redis_connection = Redis(host='localhost', port=6379, db=0)
    p44_imaging_limiter = Limiter(
        RedisBucket.init([Rate(3, Duration.SECOND)], redis_connection, "project44_api.imaging"),
        max_delay=60 * Duration.SECOND,
    )
    print(f"Attempting API call {attempt}")
    p44_imaging_limiter.try_acquire("call-api")
    print(f"Attempted API call {attempt}")


class TestRateLimiting(unittest.TestCase):
    def test_limiter(self):
        i = 1
        threads = []
        while i <= 10:
            threads.append(threading.Thread(target=api_call, kwargs={"attempt": i}))
            i += 1

        [p.start() for p in threads]
        [p.join() for p in threads]

It does not appear that this is rate limited correctly. Here is some of the logging I'm getting:

[2024-05-07T20:48:34.909645+00:00] | INFO     | [pyratelimiter_test] [test_limiter.py:api_call:28] - Attempting API call 1
[2024-05-07T20:48:34.910342+00:00] | INFO     | [pyratelimiter_test] [test_limiter.py:api_call:28] - Attempting API call 3
[2024-05-07T20:48:34.910546+00:00] | INFO     | [pyratelimiter_test] [test_limiter.py:api_call:28] - Attempting API call 10
[2024-05-07T20:48:34.910933+00:00] | INFO     | [pyratelimiter_test] [test_limiter.py:api_call:30] - Attempted API call 3
[2024-05-07T20:48:34.911431+00:00] | INFO     | [pyratelimiter_test] [test_limiter.py:api_call:28] - Attempting API call 7
[2024-05-07T20:48:34.911742+00:00] | INFO     | [pyratelimiter_test] [test_limiter.py:api_call:30] - Attempted API call 1
[2024-05-07T20:48:34.912018+00:00] | INFO     | [pyratelimiter_test] [test_limiter.py:api_call:30] - Attempted API call 10
[2024-05-07T20:48:34.912515+00:00] | INFO     | [pyratelimiter_test] [test_limiter.py:api_call:28] - Attempting API call 5
[2024-05-07T20:48:34.912716+00:00] | INFO     | [pyratelimiter_test] [test_limiter.py:api_call:28] - Attempting API call 2
[2024-05-07T20:48:34.912781+00:00] | INFO     | [pyratelimiter_test] [test_limiter.py:api_call:28] - Attempting API call 9
[2024-05-07T20:48:34.912841+00:00] | INFO     | [pyratelimiter_test] [test_limiter.py:api_call:28] - Attempting API call 4
[2024-05-07T20:48:34.913220+00:00] | INFO     | [pyratelimiter_test] [test_limiter.py:api_call:28] - Attempting API call 6
[2024-05-07T20:48:34.913328+00:00] | INFO     | [pyratelimiter_test] [test_limiter.py:api_call:28] - Attempting API call 8
[2024-05-07T20:48:35.966483+00:00] | INFO     | [pyratelimiter_test] [test_limiter.py:api_call:30] - Attempted API call 2
[2024-05-07T20:48:35.967252+00:00] | INFO     | [pyratelimiter_test] [test_limiter.py:api_call:30] - Attempted API call 8
[2024-05-07T20:48:35.967461+00:00] | ERROR    | [pyrate_limiter] [limiter.py:_handle_reacquire:155] - 
                Re-acquiring with delay expected to be successful,
                if it failed then either clock or bucket is probably unstable
                
[2024-05-07T20:48:35.967789+00:00] | ERROR    | [pyrate_limiter] [limiter.py:_handle_reacquire:155] - 
                Re-acquiring with delay expected to be successful,
                if it failed then either clock or bucket is probably unstable
                
[2024-05-07T20:48:35.967994+00:00] | ERROR    | [pyrate_limiter] [limiter.py:_handle_reacquire:155] - 
                Re-acquiring with delay expected to be successful,
                if it failed then either clock or bucket is probably unstable
                
[2024-05-07T20:48:35.968204+00:00] | INFO     | [pyratelimiter_test] [test_limiter.py:api_call:30] - Attempted API call 6
[2024-05-07T20:48:35.970266+00:00] | ERROR    | [pyrate_limiter] [limiter.py:_handle_reacquire:155] - 
                Re-acquiring with delay expected to be successful,
                if it failed then either clock or bucket is probably unstable

Doing some light debugging it appears that the threads are going to sleep as they get rate limited and then when they wake back up they expect to grab the item from the bucket but can't.

If I set the Limiter as a module level variable then things seem to get rate limited, it's a chunky 3 requests per second:

import threading
import unittest

from pyrate_limiter import Duration, Limiter, Rate, RedisBucket, AbstractClock
from redis import Redis


redis_connection = Redis(host='localhost', port=6379, db=0)
p44_imaging_limiter = Limiter(
    RedisBucket.init([Rate(3, Duration.SECOND)], redis_connection, "project44_api.imaging"),
    max_delay=60 * Duration.SECOND,
)

def api_call(attempt):
    print(f"Attempting API call {attempt}")
    p44_imaging_limiter.try_acquire("call-api")
    print(f"Attempted API call {attempt}")


class TestRateLimiting(unittest.TestCase):
    def test_limiter(self):
        i = 1
        threads = []
        while i <= 10:
            threads.append(threading.Thread(target=api_call, kwargs={"attempt": i}))
            i += 1

        [p.start() for p in threads]
        [p.join() for p in threads]

Output:

[2024-05-07T20:54:58.158751+00:00] | INFO     | [pyratelimiter_test] [test_limiter.py:api_call:18] - Attempting API call 1
[2024-05-07T20:54:58.159001+00:00] | INFO     | [pyratelimiter_test] [test_limiter.py:api_call:18] - Attempting API call 2
[2024-05-07T20:54:58.159094+00:00] | INFO     | [pyratelimiter_test] [test_limiter.py:api_call:18] - Attempting API call 3
[2024-05-07T20:54:58.159174+00:00] | INFO     | [pyratelimiter_test] [test_limiter.py:api_call:18] - Attempting API call 4
[2024-05-07T20:54:58.159271+00:00] | INFO     | [pyratelimiter_test] [test_limiter.py:api_call:18] - Attempting API call 5
[2024-05-07T20:54:58.159360+00:00] | INFO     | [pyratelimiter_test] [test_limiter.py:api_call:18] - Attempting API call 6
[2024-05-07T20:54:58.159424+00:00] | INFO     | [pyratelimiter_test] [test_limiter.py:api_call:18] - Attempting API call 7
[2024-05-07T20:54:58.159494+00:00] | INFO     | [pyratelimiter_test] [test_limiter.py:api_call:20] - Attempted API call 1
[2024-05-07T20:54:58.159608+00:00] | INFO     | [pyratelimiter_test] [test_limiter.py:api_call:18] - Attempting API call 8
[2024-05-07T20:54:58.159687+00:00] | INFO     | [pyratelimiter_test] [test_limiter.py:api_call:18] - Attempting API call 9
[2024-05-07T20:54:58.159759+00:00] | INFO     | [pyratelimiter_test] [test_limiter.py:api_call:18] - Attempting API call 10
[2024-05-07T20:54:58.159954+00:00] | INFO     | [pyratelimiter_test] [test_limiter.py:api_call:20] - Attempted API call 2
[2024-05-07T20:54:58.160357+00:00] | INFO     | [pyratelimiter_test] [test_limiter.py:api_call:20] - Attempted API call 3
[2024-05-07T20:54:59.217019+00:00] | INFO     | [pyratelimiter_test] [test_limiter.py:api_call:20] - Attempted API call 4
[2024-05-07T20:54:59.219066+00:00] | INFO     | [pyratelimiter_test] [test_limiter.py:api_call:20] - Attempted API call 5
[2024-05-07T20:54:59.220607+00:00] | INFO     | [pyratelimiter_test] [test_limiter.py:api_call:20] - Attempted API call 6
[2024-05-07T20:55:00.267008+00:00] | INFO     | [pyratelimiter_test] [test_limiter.py:api_call:20] - Attempted API call 7
[2024-05-07T20:55:00.268272+00:00] | INFO     | [pyratelimiter_test] [test_limiter.py:api_call:20] - Attempted API call 8
[2024-05-07T20:55:00.269378+00:00] | INFO     | [pyratelimiter_test] [test_limiter.py:api_call:20] - Attempted API call 9
[2024-05-07T20:55:01.316729+00:00] | INFO     | [pyratelimiter_test] [test_limiter.py:api_call:20] - Attempted API call 10

---

My goal is to understand how this code will behave across a cluster. My design is to have many containers that are all trying to contend with hitting the same external API, but that external API has a rate limit. I would like workers to sleep when they are rate limited until they can make the API call

[phillipuniverse]

My problem also feels very similar to #128

[vutran1710]

I see your point, but right now i'm quite busy with company stuff so this has to wait.

[phillipuniverse]

Thanks @vutran1710, appreciate the work you put into this library!

My guess is that the resolution of #160 would also fix my issues, seems pretty similar to what I'm seeing.

Also, I updated the test case to make it easier to reproduce by removing all the Django pieces.