urllib: Defering open call for file urls

[doerwalter]

BPO	525945
Nosy	@gvanrossum, @doerwalter
Files	diff.txt

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/gvanrossum'
closed_at = <Date 2002-03-07.19:34:31.000>
created_at = <Date 2002-03-05.13:59:29.000>
labels = ['library']
title = 'urllib: Defering open call for file urls'
updated_at = <Date 2002-03-07.19:34:31.000>
user = 'https://github.com/doerwalter'

bugs.python.org fields:

activity = <Date 2002-03-07.19:34:31.000>
actor = 'gvanrossum'
assignee = 'gvanrossum'
closed = True
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2002-03-05.13:59:29.000>
creator = 'doerwalter'
dependencies = []
files = ['4031']
hgrepos = []
issue_num = 525945
keywords = ['patch']
message_count = 4.0
messages = ['39148', '39149', '39150', '39151']
nosy_count = 2.0
nosy_names = ['gvanrossum', 'doerwalter']
pr_nums = []
priority = 'normal'
resolution = 'rejected'
stage = None
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue525945'
versions = []

[doerwalter]

This patch changes the handling of local files in
urllib.urlopen() and urllib2.urlopen(). Opening the
file is deferred until the first time read(), readline
(), readlines() or fileno() is called.

This makes it possible to retrieve the header
information for all URLs via urlopen in a uniform way,
without actually having to open the file.

[gvanrossum]

Logged In: YES
user_id=6380

I don't understand. Can you explain why you care about
this?

[doerwalter]

Logged In: YES
user_id=89016

I'm currently writing a make in Python. This make should be
able to handle not only local files, but remote files
(http, ftp, etc.). One project might have several thousand
targets, and some of them are remote. I want to be able to
handle both types in a uniform way, i.e. via
urllib/urllib2. This means, that I call urllib2.urlopen()
to get the header information about the last modification
date, but I don't want to open the file right away. Only
when the data is required (because the source resource is
newer than the target) should the file be read.

And this might open the door to making streams that are
returned from urlopen() writable (simply by using open
(..., "wb") instead of open(..., "rb") when the first write
is called.

Another possibility might be using urllib.urlretrieve(), but
the API is horrible (one global cleanup function) and not
supported by urllib2.

[gvanrossum]

Logged In: YES
user_id=6380

Hm, I don't understand. I can see how you would want to
defer opening *remote* files, because that takes time. But
the patch defers opening *local* files. I still don't
understand what the point of that is.

I think you're trying to generalize in a direction that just
isn't appropriate for urllib. I'm closing this as Rejected,
if you agree with that you don't have to do anything. :-)