urllib3 Retry and Response Hooks (Duplicates #4514)

[belthaZornv]

Given you have a session, and have successfully attached the auth class and the response hooks accordingly; although you'd like to add the Retry class into the picture from urllib3.

Expected Result

I'd expect the response hooks to be honoured - at the very least on the first response (not retry).

Actual Result

The response hooks are ignored completely.

Reproduction Steps

import requests
from requests.adapters import HTTPAdapter
from requests.packages.urllib3.util.retry import Retry

retry = Retry(total=1, read=1, connect=1, backoff_factor=0.5,
                      status_forcelist=(401, 500))

adapter = HTTPAdapter(max_retries=retry)

session = requests.Session()
session.hooks = {'response': response_hook}

session.mount('http://', adapter)
session.mount('https://', adapter)

def response_hook(response, *args, **kwargs):
    """
    Response hook

    :param response:
    :return:
    """
    if response.status_code == requests.codes.unauthorized:
       print('Your logic here')

System Information

Python version: 3.5
Requests version: 2.11.0
OS: Ubuntu 16.04

[belthaZornv]

Can anyone suggest a work around for this? (except for the standard flagging and re-doing what Retry does)

[sigmavirus24]

@belthaZornv can you share what part of our documentation misled you here?

Retry logic from urllib3 operates at a much lower level than the response hook(s). You have specified 401 as a status code to retry which means that Requests will never see it. It will be retried by urllib3 before we can execute the hook. I thought our documentation was clear that the Retry logic operated at a very low level and caused other parts of Requests to behave differently. If I'm mistaken, please point out where you saw differently.

[belthaZornv]

@sigmavirus24 I'm not saying you or the documentation misled me or someone else, but otherwise I'm more interested in discussing what you'd expect, would you expect the response hook to execute? (Taking into consideration that not everyone knows requests inside out) I did, I expected the hook to execute at least once; but I might have been wrong in thinking so, so my question is, what is the ideal solution when someone wants to Retry but also wants a response hook to execute? More over I believe that unless urllib3 honours the hooks than there isn't much to add.

Although for someone new to requests, one would expect hooks and auth classes to run as by default even in Retry, after knowing the structure beneath it all, yes, I fully agree it shouldn't or even can't.. but people are and will get lost over this.. and would be nice to find the ideal solution if any exist.

[zupo]

Just got bitten by the same thing myself too. I was assuming hooks would work, until I dug deeper.

[stroypet]

For anyone here getting stuck on this, reread the reply from sigmavirus24. If your use case was like mine ( a reauth hook ) if the status code 401 or 403 is in your status_forcelist urllib3 will gobble the response that you may be trying to hook on. You may only want to retry on < 403

...Seems so obvious now.