Python 2.6 POST callbacks get called twice

[spulec]

It looks like POST callbacks get called twice on python 2.6. I believe this is a result of a change in httplib between 2.6 and 2.7, but have not had the time to do a bisect yet.

@httprettified
def test_py26_callback_response():
    ("HTTPretty should call a callback function *once* and set its return value"
    " as the body of the response requests")

    from mock import Mock
    def _request_callback(request, uri, headers):
        return [200, headers,"The {0} response from {1}".format(decode_utf8(request.method), uri)]

    request_callback = Mock()
    request_callback.side_effect = _request_callback

    HTTPretty.register_uri(
        HTTPretty.POST, "https://api.yahoo.com/test_post",
        body=request_callback)

    response = requests.post(
        "https://api.yahoo.com/test_post",
        {"username": "gabrielfalcao"}
    )
    expect(request_callback.call_count).equal(1)

results in

FAIL: HTTPretty should call a callback function and set its return value as the body of the response requests
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/lib/python2.6/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/HTTPretty/httpretty/core.py", line 825, in wrapper
    return test(*args, **kw)
  File "HTTPretty/tests/functional/test_requests.py", line 420, in test_callback_response
    expect(request_callback.call_count).equal(1)
  File "/lib/python2.6/site-packages/sure/__init__.py", line 375, in wrapper
    value = func(self, *args, **kw)
  File "/lib/python2.6/site-packages/sure/__init__.py", line 605, in equal
    raise error
AssertionError: given
X = 2
    and
Y = 1
X is 2 whereas Y is 1

[panovodv]

That's the result of bf39ab2

response called first in send_request/sendall():
File "/usr/lib/python2.6/httplib.py", line 910, in request
self._send_request(method, url, body, headers)
File "/usr/lib/python2.6/httplib.py", line 950, in _send_request
self.send(body)
File "/usr/lib/python2.6/httplib.py", line 755, in send
self.sock.sendall(str)
File "/usr/lib/python2.6/dist-packages/httpretty/core.py", line 256, in sendall
last_entry.callable_body(last_entry.request, last_entry.info.full_url(), request_headers)
File "/usr/lib/python2.6/dist-packages/moto/core/responses.py", line 32, in dispatch
return self.call_action()

and then in getresponse():
File "/usr/lib/python2.6/httplib.py", line 981, in getresponse
method=self._method)
File "/usr/lib/python2.6/dist-packages/boto/connection.py", line 391, in init
httplib.HTTPResponse.init(self, _args, *_kwargs)
File "/usr/lib/python2.6/httplib.py", line 330, in init
self.fp = sock.makefile('rb', 0)
File "/usr/lib/python2.6/dist-packages/httpretty/core.py", line 208, in makefile
self._entry.fill_filekind(self.fd)
File "/usr/lib/python2.6/dist-packages/httpretty/core.py", line 461, in fill_filekind
status, headers, self.body = self.callable_body(self.request,self.info.full_url(),headers)

reverting that commit and removing call to dispatch in sendall fixes the issue. I don't really understand why call to dispatch is there in sendall() at all.

[DreadPirateShawn]

Note: Per @panovodv comment above re: "reverting that commit and removing call to dispatch", the specific workaround that we've implemented locally in "core.py" is:

            if not is_parsing_headers:
                if len(self._sent_data) > 1:
                    headers = utf8(last_requestline(self._sent_data))
                    body = utf8(self._sent_data[-1])

                    # see https://github.com/gabrielfalcao/HTTPretty/issues/100
                    # last_entry = self._entry
                    # last_entry.request.body = body
                    # request_headers = dict(last_entry.request.headers)
                    # If we are receiving more data and the last entry to be processed
                    # was a callback responsed, send the new data to the callback
                    # if last_entry.body_is_callable:
                    #    last_entry.callable_body(last_entry.request, last_entry.info.full_url(), request_headers)
                    self._entry.request.body = body

                    try:
                        return httpretty.historify_request(headers, body, False)
                    except Exception as e:
                        logging.error(traceback.format_exc(e))
                        return self._true_sendall(data, *args, **kw)

Note that "self._entry.request.body = body" is still necessary, after commenting out the "last_entry" logic.

[spulec]

That seems to fix this issue, but I think it would create a regression in #42.

[DreadPirateShawn]

Understood and agreed -- I'm simply being explicit about the workaround we're using to avoid being blocked by the double POST, until the fix for #42 is reworked into a fix that doesn't cause #100.

[kouk]

I ran into this problem too, and basically took @DreadPirateShawn 's approach but with a twist. Instead of:

 self._entry.request.body = body

I did

 self._entry.request.body += body

This ensures that the FakeKey has all the data and not just the last part.

[spulec]

Got it @DreadPirateShawn. Thanks.

[kouk]

Thanks for your work @gabrielfalcao !