map() and filter() methods for iterators

[dair-targ]

BPO	14961
Nosy	@ncoghlan

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2012-05-30.11:01:16.175>
created_at = <Date 2012-05-30.08:59:54.306>
labels = ['interpreter-core', 'type-feature', 'library']
title = 'map() and filter() methods for iterators'
updated_at = <Date 2012-05-30.11:01:16.164>
user = 'https://bugs.python.org/dair-targ'

bugs.python.org fields:

activity = <Date 2012-05-30.11:01:16.164>
actor = 'ncoghlan'
assignee = 'none'
closed = True
closed_date = <Date 2012-05-30.11:01:16.175>
closer = 'ncoghlan'
components = ['Interpreter Core', 'Library (Lib)']
creation = <Date 2012-05-30.08:59:54.306>
creator = 'dair-targ'
dependencies = []
files = []
hgrepos = []
issue_num = 14961
keywords = []
message_count = 6.0
messages = ['161935', '161937', '161938', '161939', '161940', '161943']
nosy_count = 4.0
nosy_names = ['ncoghlan', 'lehmannro', 'Ramchandra Apte', 'dair-targ']
pr_nums = []
priority = 'normal'
resolution = 'rejected'
stage = None
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue14961'
versions = ['Python 3.4']

[dair-targ]

It might be useful to introduce a new map() and filter() methods to iterators and iterables. Both methods should accept lambda/function which transforms a single argument into value. Both methods should return another iterator.

# proposed methods usage:
range(10).map(abs).filter(lambda x: x % 5 == 0)
# existing equivalent:
filter(lambda x: x % 5 == 0, map(abs, range(-10, 10)))
# result:
[10, 5, 0, 5]

Rough equivalent of implementation:
class iterator:
def map(self, fn):
for v in self:
yield fn(v)

def filter(self, fn):
    for v in self:
        if fn(v):
            yield v
        else:
            continue

Introduction of such methods will allow to transform collections lazy without significant memory consumption (as was mentioned in http://bugs.python.org/issue912738).

[RamchandraApte]

I think this is quite a major change to Python and this needs a PEP BTW bpo-912738 and therefore this bug no longer applies to Python 3.
In Python 3 map returns an iterator.
Even if you wanted to implement this feature in Python 2, it is not possible because in Python 2.7 (the only Python 2 release which is not in bug-fix mode) only minor enhancements are allowed.
-1 for this proposal

[RamchandraApte]

Sorry, small mistake.
Actually all the other Python 2.x releases are in security-fix mode.

[RamchandraApte]

Sorry,
To clarify:
Python 2.7 is in bug-fix mode which means only minor enhancements are allowed.

[lehmannro]

Your proposal seems two-fold: (a) make map/filter lazy and (b) have them as methods instead of functions.

It seems Tim borrowed Guido's time machine and already implemented (a) in Python 3.x, see http://docs.python.org/py3k/library/functions.html#map and http://docs.python.org/py3k/library/functions.html#filter.

Your second proposal-- which is better suited for python-ideas, really --is obstructed by iterators being merely a protocol (the next/next method) which makes it hard to add those methods to one particular type. (This very discussion pops up every so often for str.join too.)

I'd recommend closing this issue.

[ncoghlan]

As Robert noted, the map() and filter() builtins in Python 3 are already lazy and there's no reason to expand the iterator protocol for this functionality.

Map and filter also have dedicated syntax in the form of comprehensions and generator expressions:

    itr = (x for x in map(abs, range(10)) if x % 5 == 0)

Furthermore, the standard library already provides an entire module of tools for creating and working with lazy iterators in both Python 2 and Python 3: http://docs.python.org/library/itertools