urllib.URLopener prematurely deletes files on cleanup

[carlbook]

BPO	12638
Nosy	@orsenthil, @iritkatriel
Files	bug2.py: urllib exception bug3.py

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2021-05-08.11:00:12.728>
created_at = <Date 2011-07-25.15:34:57.929>
labels = ['type-bug', 'library']
title = 'urllib.URLopener prematurely deletes files on cleanup'
updated_at = <Date 2021-05-08.11:00:20.634>
user = 'https://bugs.python.org/carlbook'

bugs.python.org fields:

activity = <Date 2021-05-08.11:00:20.634>
actor = 'iritkatriel'
assignee = 'none'
closed = True
closed_date = <Date 2021-05-08.11:00:12.728>
closer = 'iritkatriel'
components = ['Library (Lib)']
creation = <Date 2011-07-25.15:34:57.929>
creator = 'carlbook'
dependencies = []
files = ['22750', '22752']
hgrepos = []
issue_num = 12638
keywords = []
message_count = 6.0
messages = ['141094', '141095', '141096', '141097', '141102', '389548']
nosy_count = 3.0
nosy_names = ['orsenthil', 'carlbook', 'iritkatriel']
pr_nums = []
priority = 'normal'
resolution = 'out of date'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue12638'
versions = ['Python 2.6', 'Python 2.7', 'Python 3.2']

[carlbook]

urllib.URLopener (or urllib.request.URLopener for Python 3) and user defined classes that inherit from these prematurely delete files upon cleanup. Any temporary files downloaded using the .retrieve() method are deleted when an instance of a URLopener is garbage collected.

I feel this is a violation since the filename is returned to the caller and then silently deleted. It is possible to simply override the .cleanup() method, but I feel this is not a good solution.

[orsenthil]

urlretrieve is a helper function from urllib module. The way to use
it is:

('/tmp/tmpe873xe', <http.client.HTTPMessage object at 0xb72c2f6c>)
>>> import os
>>> os.stat('/tmp/tmpe873xe')
posix.stat_result(st_mode=33152, st_ino=4462517, st_dev=2054,
st_nlink=1, st_uid=1000, st_gid=1000, st_size=33128,
st_atime=1311608669, st_mtime=1311608670, st_ctime=1311608670)

Works properly for me 3.3 and 2.7. Is there any service that is
running on your machine that deleting the tmp files as soon as they
are created?

[orsenthil]

urlretrieve is a helper function from urllib module. The way to use
it is:

('/tmp/tmpe873xe', <http.client.HTTPMessage object at 0xb72c2f6c>)
>>> import os
>>> os.stat('/tmp/tmpe873xe')

Works!

[orsenthil]

Tracker stripped off the code. Here is it.

import urllib.request
urllib.request.urlretrieve('http://bugs.python.org')
('/tmp/tmpe873xe', <http.client.HTTPMessage object at 0xb72c2f6c>)
import os
os.stat('/tmp/tmpe873xe')
posix.stat_result(st_mode=33152, st_ino=4462517, st_dev=2054, st_nlink=1, st_uid=1000, st_gid=1000, st_size=33128, st_atime=1311608669, st_mtime=1311608670, st_ctime=1311608670)

[carlbook]

@orsenthil, that is the correct behavior if you do not want to override any of URLopener's handlers for error codes. In my case, I wanted to override FancyURLopener (a child class of URLopener) to override HTTP 401 behavior. Using urlretrieve is not correct in this case.

Also included python 3.2 code, I didn't test 3.1.

[iritkatriel]

I think this is out of date as this feature was deprecated in bpo-10050.