Possible performance improvement for itertools.product example on Python Docs

[ataher]

BPO	41120
Nosy	@rhettinger
Files	product example.py: Contains the 3 versions and 3 outputs

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2020-06-26.01:19:47.099>
created_at = <Date 2020-06-25.23:11:22.304>
labels = ['type-feature', 'docs']
title = 'Possible performance improvement for itertools.product example on Python Docs'
updated_at = <Date 2020-06-26.09:39:22.384>
user = 'https://bugs.python.org/ataher'

bugs.python.org fields:

activity = <Date 2020-06-26.09:39:22.384>
actor = 'ataher'
assignee = 'docs@python'
closed = True
closed_date = <Date 2020-06-26.01:19:47.099>
closer = 'rhettinger'
components = ['Documentation']
creation = <Date 2020-06-25.23:11:22.304>
creator = 'ataher'
dependencies = []
files = ['49262']
hgrepos = []
issue_num = 41120
keywords = []
message_count = 3.0
messages = ['372392', '372397', '372418']
nosy_count = 3.0
nosy_names = ['rhettinger', 'docs@python', 'ataher']
pr_nums = []
priority = 'normal'
resolution = 'rejected'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue41120'
versions = []

[ataher]

In the documentation the following example is given:

def product(*args, repeat=1):
    # product('ABCD', 'xy') --> Ax Ay Bx By Cx Cy Dx Dy
    # product(range(2), repeat=3) --> 000 001 010 011 100 101 110 111
    pools = [tuple(pool) for pool in args] * repeat
    result = [[]]
    for pool in pools:
        result = [x+[y] for x in result for y in pool]
    for prod in result:
        yield tuple(prod)

The proposed enhancement uses a nested generator so no intermediate results are created.

def product2(*args, repeat=1):
    def concat(result, pool):
        yield from (x+[y] for x in result for y in pool)
        
    pools = [tuple(pool) for pool in args] * repeat
    result = [[]]
    for pool in pools:
        result = concat(result, pool)
    for prod in result:
        yield (tuple(prod))

[rhettinger]

Thanks for the suggestion, but the purpose of the rough equivalent example is to give a better idea of what product() does. It is not about performance — that job falls to the actual implementation.

Nested generators are an intermediate level Python skill, so using them in the example code makes the docs less accessible.

[ataher]

Good morning Raymond,   Thank you for this quick response.
You are absolutely right on all your points if you only consider that the example given in the documentation as purely a suggestion of "equivalent" version to the one implemented in the library.
However, if you look at the examples given in the documentation as an illustration of an algorithm that can be customized and changed by the end-user if the existing implementation is not suitable then you might consider otherwise.
This is exactly what happened when I wanted to do a Cartesian Product. Starting, I first looked at itertools and
run my routine using itertools.product(). However, it turns out that I needed to flatten the result tuples which prompted me toadjust the given example accordingly and then add the nested generator.

As for your statement:  "Nested generators are an intermediate level Python skill, so using them in the example code makes the docs less accessible.". Let me point to the facts, that someone looking at itertools and trying to do a Cartesian Product
is certainly an intermediate programmer and often likes to understand, learn or peek inside the algorithm used. Additionally, the suggested improvement is a generator similar to the implementation where as the existing example is not.

Thus, I hope I am able to convince you to reconsider your decision especially when you look at the given examples from a wider perspective and more so when the suggested improvement is minor and easy to understand.
Best Regards,
Abbas

Side note: one of the first tutorial videos I used to learn Python were the ones you prepared for Oreilly. Thank you and thank you for all the work you did and still doing for free for the Python language and the Python community over so many many many years.

On Thursday, June 25, 2020, 9:19:55 PM EDT, Raymond Hettinger <report@bugs.python.org> wrote:  

Raymond Hettinger <raymond.hettinger@gmail.com> added the comment:

Thanks for the suggestion, but the purpose of the rough equivalent example is to give a better idea of what product() does.  It is not about performance — that job falls to the actual implementation.

Nested generators are an intermediate level Python skill, so using them in the example code makes the docs less accessible.

----------
resolution:  -> rejected
stage:  -> resolved
status: open -> closed

---

Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue41120\>

---