Unexpected ConnectionResetError in urllib.request against a valid website

[TymoteuszPaul]

BPO	21896
Nosy	@orsenthil, @ned-deily, @bitdancer, @vadmium

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2015-01-12.01:22:36.270>
created_at = <Date 2014-07-01.12:18:25.726>
labels = ['invalid', 'type-bug', 'library']
title = 'Unexpected ConnectionResetError in urllib.request against a valid website'
updated_at = <Date 2015-01-12.01:22:36.269>
user = 'https://bugs.python.org/TymoteuszPaul'

bugs.python.org fields:

activity = <Date 2015-01-12.01:22:36.269>
actor = 'ned.deily'
assignee = 'none'
closed = True
closed_date = <Date 2015-01-12.01:22:36.270>
closer = 'ned.deily'
components = ['Library (Lib)']
creation = <Date 2014-07-01.12:18:25.726>
creator = 'Tymoteusz.Paul'
dependencies = []
files = []
hgrepos = []
issue_num = 21896
keywords = []
message_count = 4.0
messages = ['222024', '222151', '233867', '233869']
nosy_count = 5.0
nosy_names = ['orsenthil', 'ned.deily', 'r.david.murray', 'martin.panter', 'Tymoteusz.Paul']
pr_nums = []
priority = 'normal'
resolution = 'not a bug'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue21896'
versions = ['Python 2.7', 'Python 3.4', 'Python 3.5']

[TymoteuszPaul]

I've recently ran into a problem with urellib.request.urlopen that it fails against one website (that I've found so far). The website itself is working fine, I can access its content with other libraries like requests, curl and outside of python with telnet, links and so on. But with urllib it fails:

Python 3.4.1 (default, Jul  1 2014, 14:08:25)
[GCC 4.7.3] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import urllib.request
>>> urllib.request.urlopen("http://www.thomsonlocal.com/")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib64/python3.4/urllib/request.py", line 153, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib64/python3.4/urllib/request.py", line 455, in open
    response = self._open(req, data)
  File "/usr/lib64/python3.4/urllib/request.py", line 473, in _open
    '_open', req)
  File "/usr/lib64/python3.4/urllib/request.py", line 433, in _call_chain
    result = func(*args)
  File "/usr/lib64/python3.4/urllib/request.py", line 1215, in http_open
    return self.do_open(http.client.HTTPConnection, req)
  File "/usr/lib64/python3.4/urllib/request.py", line 1194, in do_open
    r = h.getresponse()
  File "/usr/lib64/python3.4/http/client.py", line 1172, in getresponse
    response.begin()
  File "/usr/lib64/python3.4/http/client.py", line 351, in begin
    version, status, reason = self._read_status()
  File "/usr/lib64/python3.4/http/client.py", line 313, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/lib64/python3.4/socket.py", line 371, in readinto
    return self._sock.recv_into(b)
ConnectionResetError: [Errno 104] Connection reset by peer

I've tested it on about 6 different servers, in different parts of the world and all of them seem to be affected. I've tested with with 3.2.5, 3.3.3, 3.4.1 and they are all failed with the same trace.

[ned-deily]

It fails with Python 2's urllib2.urlopen as well.

[vadmium]

Not a Python bug. The web site seems to be doing this based on the user agent; if you change it, it works:

urlopen(Request("http://www.thomsonlocal.com/", headers={"User-Agent": "https://bugs.python.org/issue21896"}))

[ned-deily]

So it does. Thanks, Martin.