Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

When using multiple @retry decorators on a function, inner decorator's state is constantly reset #407

Open
Tomasz-Kluczkowski opened this issue Jul 9, 2023 · 1 comment

Comments

@Tomasz-Kluczkowski
Copy link

Tomasz-Kluczkowski commented Jul 9, 2023

I wanted to implement different wait behaviours depending on which exception was raised when calling a given function.
I wrap the function with @retry twice but I noticed that the inner retry will get its attempt/state reset every time the outer retry catches an exception.
This then means that an already registered failed attempt for exception in the inner decorator constantly starts again.

You can actually see in the logs below that the retry call state object is constantly changing for the inner decorator, but is kept the same for the outer one and thus it can keep counting the attempts.

I assume simply that's how it works in python and the 2 decorators do not share retry state.

Maybe this is desired outcome or maybe I am abusing what tenacity is meant for. Just wanted to check with the rest of the community and if this is considered a bug/feature - let's agree on it :).

Example

    def test_tenacity_retry_if_not_message(self):
        item_number = 0
        too_many_requests_message = '429 too many requests'
        other_message = 'some other exception'

        exceptions = [
            SystemExit(too_many_requests_message),
            SystemExit(other_message),
            SystemExit(too_many_requests_message),
            SystemExit(other_message),
            SystemExit(too_many_requests_message),
            SystemExit(other_message),
            SystemExit(too_many_requests_message),
            SystemExit(other_message),
            SystemExit(too_many_requests_message),
            SystemExit(other_message),
            SystemExit(too_many_requests_message),
            SystemExit(other_message),
            SystemExit(too_many_requests_message),
            SystemExit(other_message),
            SystemExit(too_many_requests_message),
            SystemExit(other_message),
            SystemExit(too_many_requests_message),
        ]

        def mock_get():
            nonlocal item_number
            exception = exceptions[item_number]
            item_number += 1

            raise exception

        @tenacity.retry(wait=tenacity.wait.wait_fixed(5),
                        retry=tenacity.retry_if_exception_message(message=too_many_requests_message),
                        stop=stop_after_attempt(3), before_sleep=log_retry_attempt, reraise=True)
        @tenacity.retry(wait=tenacity.wait.wait_fixed(1),
                        retry=tenacity.retry_if_not_exception_message(message=too_many_requests_message),
                        stop=stop_after_attempt(3), before_sleep=log_retry_attempt, reraise=True)
        def main():
            mock_get()

        main()

Logs:

-------------------------------- live log call ---------------------------------
2023-07-09 16:22:43 [ WARNING] <RetryCallState 4416663328: attempt #1; slept for 5.0; last result: failed (SystemExit 429 too many requests)> (retrying.py:18)
2023-07-09 16:22:48 [ WARNING] <RetryCallState 4416661600: attempt #1; slept for 1.0; last result: failed (SystemExit some other exception)> (retrying.py:18)
2023-07-09 16:22:49 [ WARNING] <RetryCallState 4416663328: attempt #2; slept for 10.0; last result: failed (SystemExit 429 too many requests)> (retrying.py:18)
2023-07-09 16:22:54 [ WARNING] <RetryCallState 4416664240: attempt #1; slept for 1.0; last result: failed (SystemExit some other exception)> (retrying.py:18)
FAILED                                                                   [100%]

end of stack trace:

    def mock_get():
        nonlocal item_number
        exception = exceptions[item_number]
        item_number += 1
    
>       raise exception
E       SystemExit: 429 too many requests
@Tomasz-Kluczkowski Tomasz-Kluczkowski changed the title When using multiple @retry decorators on a function, inner decorators state is constantly reset When using multiple @retry decorators on a function, inner decorator's state is constantly reset Jul 9, 2023
@26038874
Copy link

26038874 commented Aug 8, 2023

I have the same need and look forward to an answer
Attached is my log
image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants