Problem with urllib and urllib2 in urlopen?

[ambarish]

BPO	2868
Nosy	@orsenthil, @benjaminp

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2008-05-17.16:50:13.725>
created_at = <Date 2008-05-15.17:34:25.389>
labels = ['extension-modules', 'type-bug', 'invalid']
title = 'Problem with urllib and urllib2 in urlopen?'
updated_at = <Date 2008-05-17.16:50:13.706>
user = 'https://bugs.python.org/ambarish'

bugs.python.org fields:

activity = <Date 2008-05-17.16:50:13.706>
actor = 'benjamin.peterson'
assignee = 'none'
closed = True
closed_date = <Date 2008-05-17.16:50:13.725>
closer = 'benjamin.peterson'
components = ['Extension Modules']
creation = <Date 2008-05-15.17:34:25.389>
creator = 'ambarish'
dependencies = []
files = []
hgrepos = []
issue_num = 2868
keywords = []
message_count = 4.0
messages = ['66872', '66881', '66888', '67003']
nosy_count = 4.0
nosy_names = ['orsenthil', 'benjamin.peterson', 'BitTorment', 'ambarish']
pr_nums = []
priority = 'normal'
resolution = 'not a bug'
stage = None
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue2868'
versions = ['Python 2.5']

[ambarish]

I have the following code:

import urllib
u = 'http://www.mercurynews.com/ci_9216417'
h = urllib.urlopen(u).read()
print h
# Get an empty string
#(can use urllib2 also - get the same behavior)

If I visit the same page with my browser, get the contents of the page
(after some redirects...)

[benjaminp]

This is what happens on the trunk:

>>> import urllib
>>> u = 'http://www.mercurynews.com/ci_9216417'
>>> h = urllib.urlopen(u).read()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/temp/python/trunk/Lib/ssl.py", line 333, in read
data = self._sslobj.read(recv_size)
ssl.SSLError: [Errno 8] _ssl.c:1276: EOF occurred in violation of protocol

[orsenthil]

Here is my analysis:

>> import urllib2
>> url = "http://www.mercurynews.com/ci_9216417"
>> content = urllib2.urlopen(url).read()
>> print content

>>> opened = urllib2.urlopen(url)
>>> print opened.geturl()
https://secure.passport.mnginteractive.com/mngi/servletDispatch/ErightsPassportServlet.dyn?url=http://www.mercurynews.com/ci_9216417?nclick_check=1&forced=true

# This URL Redirection is a 302 Redirection.
# Browser is "unable" to launch the redirected site.
https://secure.passport.mnginteractive.com/mngi/servletDispatch/ErightsPassportServlet.dyn?url=http://www.mercurynews.com/ci_9216417?nclick_check=1&forced=true
# Logically, the urllib /urllib2 is giving a Blank when reading this site.

I would't entire say urllib2's fault before understanding how FF is
handling the redirection of the first site.

1. Open the site mentioned in the Location of 302, you will experience
   the same behaviour as 302.

It seems more of an issue at server end, we have to know how, Firefox is
handling at the first place.

[BitTorment]

I verified the behaviour but this is a problem with that particular
site, not with urllib/urllib2.

Should be closed.