urllib hangs when closing connection

[ChrisBeaumont]

BPO	26960
Nosy	@orsenthil, @giampaolo, @native-api
Files	urllib.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2016-05-10.08:13:07.246>
created_at = <Date 2016-05-05.00:25:54.028>
labels = ['type-bug', 'library']
title = 'urllib hangs when closing connection'
updated_at = <Date 2016-12-10.13:21:31.181>
user = 'https://bugs.python.org/ChrisBeaumont'

bugs.python.org fields:

activity = <Date 2016-12-10.13:21:31.181>
actor = 'Ivan.Pozdeev'
assignee = 'none'
closed = True
closed_date = <Date 2016-05-10.08:13:07.246>
closer = 'python-dev'
components = ['Library (Lib)']
creation = <Date 2016-05-05.00:25:54.028>
creator = 'Chris Beaumont'
dependencies = []
files = ['42728']
hgrepos = []
issue_num = 26960
keywords = ['patch']
message_count = 5.0
messages = ['264868', '265226', '265228', '265229', '282850']
nosy_count = 5.0
nosy_names = ['orsenthil', 'giampaolo.rodola', 'python-dev', 'Ivan.Pozdeev', 'Chris Beaumont']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue26960'
versions = ['Python 2.7']

[ChrisBeaumont]

http://bugs.python.org/issue16270 identified an issue with ftpwrapper.endtransfer that causes ftp fetches to hang in certain situations. A fix was applied to python 3. I see the same issue on python 2.7, and the patch from 16270 fixes it. Is there a reason this fix hasn't been backported to 2.7?

To reproduce:

import urllib
url = "ftp://ftp.fu-berlin.de/pub/misc/movies/database/ratings.list.gz"
fp = urllib.urlopen(url)
fp.close()  # hangs

[python-dev]

New changeset 44d02a5d59fb by Senthil Kumaran in branch '2.7':
Closes bpo-26960.
https://hg.python.org/cpython/rev/44d02a5d59fb

[python-dev]

New changeset 3388e249389d by Senthil Kumaran in branch '2.7':
Add NEWS entry for bpo-26960.
https://hg.python.org/cpython/rev/3388e249389d

[orsenthil]

Thanks for raising this issue, Chris Beaumont.
Fixed in 2.7.x

[native-api]

This fix has caused bpo-25458 to manifest itself in urllib', too. AFAICS, it's impossible to fully fix urllib' to correctly handle end-of-transmission response without fixing `ftplib' first.