-
-
Notifications
You must be signed in to change notification settings - Fork 9.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Error decoding chunked response in _update_chunk_length: ValueError: invalid literal for int() with base 16: b'HTTP/1.1 200 OK\r\n' #4248
Comments
_update_chunk_length
: ValueError: invalid literal for int() with base 16: b'HTTP/1.1 200 OK\r\n'
This works fine with requests 2.10.0 so something regressed between then and 2.13.0. Also tested with 2.18.4 and the bug exists as above. |
I can't spot anything defective or invalid in the response from that server when I look at it manually. One thing I do notice is that it is using both |
I can't reproduce this with Requests 2.18.4 on either Python 3 or Python 2. Have you tried updating urllib3 to check if this resolves the problem? |
Okay sorry, my comment above about 2.10.0 vs. 2.13.0 was incorrect. I did not thoroughly test all the combinations. It turns out this works fine with all versions/combinations of requests and urllib3 that I tested on Fedora 25 but fails with all versions/combinations of requests and urllib3 that I tested on Fedora 26. Which most likely means: it's something specific to Python 3.6? |
Or at least Fedora 36, as it works fine for me with my Python 3.6.2 on macOS. I recommend reporting a bug with Fedora. |
I get this error on Windows with the latest Python 3.7 alpha and latest requests / urllib3. When I use Python 3.5 then it works fine with the same requests / urllib3. I'm suspecting a python version issue as well. |
I haven't dug into this in a while. Back when I was looking at it, I could not find any substantial differences between Fedora 25 and 26 in Python land at all (same Python version, same version of all the relevant Python libraries I could see, same code for urllib3 and requests) but one big difference did stand out: Fedora 25 has openssl 1.0.2 whereas Fedora 26+ has openssl 1.1.0. Not sure if that is related. |
i'm facing same kind of issue. any idea how to bypass this?
|
I solved this problem by adding one line code. python3 on Mac
so, I open the file "......./urllib3/response.py", add
|
It looks that ths line fixes the problem for me. Can you please explain what this line do? |
Shouldn't this then be fixed in urllib3? @Lukasa |
@KonMann yes it should. |
@cosminnastasie I am a little forgotten. It seems like that without this line, there should be occur problem at the last chunk data. when the value of 'line' is Null, the error arise. |
but if we add that line:
problem solved!!! |
@sigmavirus24 ok, as far as I can see there is no open issue about this in the Urllib3 Issue Tracker at the moment => Let's open an issue there? |
@KonMann you seem to have more time than me. Go for it. |
@odboy Do you have some code to reproduce? |
We temporarily repaired it in this way. I wonder if there will be any problem.
|
I'm running into this exact issue as well, with an identical traceback as #4248 (comment). I've tried reverting requests as far back as 2.9.1, and as far forward as the current release (2.20.1), with a few different combinations of urllib3 versions in each case - they all give the exact same traceback. Is there a release which we can use until this is fixed, or do we have to patch our requests with the above corrected def? |
I am having this issue as well, and I agree the problem is at the urllib3 level. |
Can we agree on where we want this issue to be fixed? Here in requests or in urllib3? |
The underlying exception comes from urllib3, but as @dulitz explained in urllib3/urllib3#1516, this isn't necessarily always a "bug" in requests or urllib3. So "fixing" it might not be easy or possible. Regardless, I'm closing this issue here. |
I am able to reproduce the same bug by using this code import requests I think this is a bug in the server side, but maybe urllib3 could do a better job workarounding this bug in the server, as other libraries / applications do for example (same URL works ok in a web browser like chrome or firefox ...) |
As far as I could see, the server (in the example above) sends chunked transfer, but it does not send properly the last 0-length chunk and returns the response code. That's where urllib3 raises exception when trying to decode the length of the chunk from the response status code line. Some insights I could find so far:
r = requests.get('https://www.telecreditobcp.com/tlcnp/index.do', stream=True)
curl: (56) Illegal or missing hexadecimal sequence in chunked-encoding |
- Changed: /qiy_nodes/{}/events New: silenced Empty exception See also: psf/requests#4248 2019-08-08 08:08:41,858: Exception in thread mgd_dev2.events_listener: 2019-08-08 08:08:41,859: Traceback (most recent call last): 2019-08-08 08:08:41,859: File "/home/fdriesenaar/.local/lib/python3.6/site-packages/urllib3/response.py", line 639, in _update_chunk_length 2019-08-08 08:08:41,859: self.chunk_left = int(line, 16) 2019-08-08 08:08:41,859: ValueError: invalid literal for int() with base 16: b'' 2019-08-08 08:08:41,859: 2019-08-08 08:08:41,860: During handling of the above exception, another exception occurred: 2019-08-08 08:08:41,860: 2019-08-08 08:08:41,860: Traceback (most recent call last): 2019-08-08 08:08:41,860: File "/home/fdriesenaar/.local/lib/python3.6/site-packages/urllib3/response.py", line 397, in _error_catcher 2019-08-08 08:08:41,860: yield 2019-08-08 08:08:41,860: File "/home/fdriesenaar/.local/lib/python3.6/site-packages/urllib3/response.py", line 704, in read_chunked 2019-08-08 08:08:41,861: self._update_chunk_length() 2019-08-08 08:08:41,861: File "/home/fdriesenaar/.local/lib/python3.6/site-packages/urllib3/response.py", line 643, in _update_chunk_length 2019-08-08 08:08:41,861: raise httplib.IncompleteRead(line) 2019-08-08 08:08:41,861: http.client.IncompleteRead: IncompleteRead(0 bytes read) 2019-08-08 08:08:41,861: 2019-08-08 08:08:41,861: During handling of the above exception, another exception occurred: 2019-08-08 08:08:41,862: 2019-08-08 08:08:41,862: Traceback (most recent call last): 2019-08-08 08:08:41,862: File "/home/fdriesenaar/.local/lib/python3.6/site-packages/requests/models.py", line 750, in generate 2019-08-08 08:08:41,862: for chunk in self.raw.stream(chunk_size, decode_content=True): 2019-08-08 08:08:41,862: File "/home/fdriesenaar/.local/lib/python3.6/site-packages/urllib3/response.py", line 527, in stream 2019-08-08 08:08:41,862: for line in self.read_chunked(amt, decode_content=decode_content): 2019-08-08 08:08:41,863: File "/home/fdriesenaar/.local/lib/python3.6/site-packages/urllib3/response.py", line 732, in read_chunked 2019-08-08 08:08:41,863: self._original_response.close() 2019-08-08 08:08:41,863: File "/usr/lib/python3.6/contextlib.py", line 100, in __exit__ 2019-08-08 08:08:41,863: self.gen.throw(type, value, traceback) 2019-08-08 08:08:41,863: File "/home/fdriesenaar/.local/lib/python3.6/site-packages/urllib3/response.py", line 415, in _error_catcher 2019-08-08 08:08:41,864: raise ProtocolError('Connection broken: %r' % e, e) 2019-08-08 08:08:41,864: urllib3.exceptions.ProtocolError: ('Connection broken: IncompleteRead(0 bytes read)', IncompleteRead(0 bytes read)) 2019-08-08 08:08:41,864: 2019-08-08 08:08:41,864: During handling of the above exception, another exception occurred: 2019-08-08 08:08:41,864: 2019-08-08 08:08:41,864: Traceback (most recent call last): 2019-08-08 08:08:41,865: File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner 2019-08-08 08:08:41,865: self.run() 2019-08-08 08:08:41,865: File "/usr/lib/python3.6/threading.py", line 864, in run 2019-08-08 08:08:41,865: self._target(*self._args, **self._kwargs) 2019-08-08 08:08:41,865: File "/home/fdriesenaar/QiyTestTool/QiyTestTool/site/flask_app.py", line 196, in node_events_listener 2019-08-08 08:08:41,865: for chunk in r.iter_content(chunk_size=1, decode_unicode=True): 2019-08-08 08:08:41,866: File "/home/fdriesenaar/.local/lib/python3.6/site-packages/requests/utils.py", line 505, in stream_decode_response_unicode 2019-08-08 08:08:41,866: for chunk in iterator: 2019-08-08 08:08:41,866: File "/home/fdriesenaar/.local/lib/python3.6/site-packages/requests/models.py", line 753, in generate 2019-08-08 08:08:41,866: raise ChunkedEncodingError(e) 2019-08-08 08:08:41,866: requests.exceptions.ChunkedEncodingError: ('Connection broken: IncompleteRead(0 bytes read)', IncompleteRead(0 bytes read)) 2019-08-08 08:08:41,866: 2019-08-08 08:09:51,825: Error running WSGI application 2019-08-08 08:09:51,828: queue.Empty 2019-08-08 08:09:51,828: File "/usr/local/lib/python3.6/dist-packages/werkzeug/wsgi.py", line 703, in __next__ 2019-08-08 08:09:51,828: return self._next() 2019-08-08 08:09:51,829: 2019-08-08 08:09:51,829: File "/usr/local/lib/python3.6/dist-packages/werkzeug/wrappers.py", line 81, in _iter_encoded 2019-08-08 08:09:51,829: for item in iterable: 2019-08-08 08:09:51,829: 2019-08-08 08:09:51,829: File "/home/fdriesenaar/QiyTestTool/QiyTestTool/site/flask_app.py", line 913, in gen 2019-08-08 08:09:51,830: event=queue.get(timeout=100) 2019-08-08 08:09:51,830: 2019-08-08 08:09:51,830: File "/usr/lib/python3.6/queue.py", line 172, in get 2019-08-08 08:09:51,830: raise Empty
Solved my problem. Man is a LEGEND. |
Decoding a response with chunked encoding fails because it seems to be reading from the HTTP status line instead of the start of the body.
Expected Result
Successfully return a 200 response object.
Actual Result
Reproduction Steps
System Information
requests 2.13.0
urllib3 1.20
Python 3.6.2
Fedora 26
The text was updated successfully, but these errors were encountered: