difflib should accept arbitrary line iterators

[techtonik]

BPO	8905
Nosy	@tim-one, @terryjreedy, @giampaolo, @merwok

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2013-03-19.08:20:15.949>
created_at = <Date 2010-06-05.11:45:39.123>
labels = ['type-feature', 'library']
title = 'difflib should accept arbitrary line iterators'
updated_at = <Date 2013-03-19.08:20:15.947>
user = 'https://bugs.python.org/techtonik'

bugs.python.org fields:

activity = <Date 2013-03-19.08:20:15.947>
actor = 'terry.reedy'
assignee = 'none'
closed = True
closed_date = <Date 2013-03-19.08:20:15.949>
closer = 'terry.reedy'
components = ['Library (Lib)']
creation = <Date 2010-06-05.11:45:39.123>
creator = 'techtonik'
dependencies = []
files = []
hgrepos = []
issue_num = 8905
keywords = []
message_count = 4.0
messages = ['107130', '107246', '132872', '184619']
nosy_count = 6.0
nosy_names = ['tim.peters', 'terry.reedy', 'techtonik', 'giampaolo.rodola', 'eric.araujo', 'gruszczy']
pr_nums = []
priority = 'normal'
resolution = 'works for me'
stage = 'test needed'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue8905'
versions = ['Python 3.3']

[techtonik]

difflib operates on the lists, but it should be possible to use arbitrary generators. This will require internal limit on buffer size that has a side advantage of limiting difflib to available memory.

[merwok]

If you allow me to rephrase your feature request to “difflib should allow arbitrary iterators that yield lines”, I’m +1.

Adjusting the version and adding tim_one to nosy as per py3k/Misc/maintainers.rst

[merwok]

A quick look at the code doesn’t immediately tells me that difflib accepts sequences, not only lists. I’m not sure iterators are accepted too. What specific functions or methods have you found too strict?

[terryjreedy]

'difflib' is a module that defines three classes and some functions. It does not do anything in itself. SequenceMatcher, which is the basis for the other functions, operates on sequences of hashable objects. The inputs must be concrete random access indexed sequences because SequenceMatcher scans the inputs and then jumps around finding good matches and the complementary differences. It is completely unlike functions that take iterables as arguments and than call iter to get an iterator. If S-M took iterables as inputs, it would have to either copy all inputs with list (bad) or figure out whether or not to copy. I think users can continue to pass file.readlines() or list(iterable) as needed.

The two functions that mistakenly say inputs are 'lists' instead of 'sequences' will be fixed at part of another issue.