Add prod() function to the math module #79787
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
assignee = None closed_at = <Date 2019-02-07.07:04:59.745> created_at = <Date 2018-12-28.19:29:12.734> labels = ['3.8', 'type-feature', 'library'] title = 'Add prod() function to the math module' updated_at = <Date 2021-09-28.20:19:11.694> user = 'https://github.com/rhettinger'
activity = <Date 2021-09-28.20:19:11.694> actor = 'lukasz.langa' assignee = 'none' closed = True closed_date = <Date 2019-02-07.07:04:59.745> closer = 'rhettinger' components = ['Library (Lib)'] creation = <Date 2018-12-28.19:29:12.734> creator = 'rhettinger' dependencies =  files =  hgrepos =  issue_num = 35606 keywords = ['patch', 'patch', 'patch'] message_count = 18.0 messages = ['332676', '332762', '332763', '332777', '332824', '333005', '333007', '333034', '333051', '333090', '334995', '335007', '335508', '335510', '335515', '402772', '402778', '402803'] nosy_count = 13.0 nosy_names = ['tim.peters', 'aleax', 'rhettinger', 'mark.dickinson', 'pitrou', 'vstinner', 'lukasz.langa', 'serhiy.storchaka', 'pablogsal', 'miss-islington', 'remi.lapeyre', 'xtreak', 'lschoe'] pr_nums = ['11359', '28595', '28603', '28604'] priority = 'normal' resolution = 'fixed' stage = 'resolved' status = 'closed' superseder = None type = 'enhancement' url = 'https://bugs.python.org/issue35606' versions = ['Python 3.8']
The text was updated successfully, but these errors were encountered:
Back in 2007, a user suggested a built-in prod() function with an API similar to the built-in sum() function. The proposal was rejected because it wasn't needed often enough to justify a builtin function. See https://bugs.python.org/issue1093
Though prod() doesn't meet the threshold for a builtin, it would be reasonable to add this to the math module (or an imath module).
Personally, I've wanted and written this function on several occasions (for applications such as multiplying probabilities). On stack overflow, it has been a popular question with recurring interest. See https://stackoverflow.com/questions/7948291/ and https://stackoverflow.com/questions/595374
@serhiy.storchaka, it should be possible to make it far simpler if we make math_prod_impl more naive by removing the hypothesis made on
A naive implementation would also support user-defined types which would probably be a good thing IMO
FWIW, it is often the one liners that turn out to be the most useful building blocks. In this case the one-liner is inconvenient (two imports), not as fast we would like, and a little opaque: functools.reduce(operator.mul, iterable, 1).
I would be happy with the simplest possible implementation. That said, we do have a history of going gonzo in C internals to get better speed/space performance (look the code for math.factorial() for example), to have better accuracy and avoid overflow/underflow (math.hypot() for example), or to implement particular NaN/Inf handling not present in a naive implementation.
On this subject, some effort has been made in the past to make (almost) all the math module functions behave consistently with respect to things like exceptions, overflow, infinities, nans, signed zeros, etc. (There are unfortunately some exceptions to the general rule, like math.pow.) If possible, I'd like to see any implementation of math.prod do the same; I'd prefer us to get this right initially rather than tweak the definition to make subtle possibly-breaking changes to the implementation later.
That is, the ideal behaviour would include things like:
(a) a product of finite floating-point (or convertible-to-float) numbers should raise an exception on overflow. Probably OverflowError, though ValueError wouldn't be indefensible either.
(b) a product involving infinities but no NaNs or zeros should return an appropriately-signed infinity (where the sign is determined by an xor of all the signs of the inputs)
(c) a product involving both infinities and zeros (but not NaNs) should raise ValueError
(d) a product involving a NaN at any point should just return NaN
The combination of these can get a bit awkward: for example, if a list starts with
BTW, if we wanted to go for IEEE 754 compliance, the operation to implement would be "scaledProd", which takes a vector of inputs and returns a float along with an exponent; this allows taking products of long lists of floats without needing to worry about underflow or overflow. That's probably not what we want here, but the spec of scaledProd might help guide us with the implementation of prod.
Computing the geometric mean of numbers require to compute the product of these numbers:
The geometric mean can be used to summarize benchmark results using different units to get a single number.
When computing the product of floats, is there a smart implementation reducing the error? I'm asking because math.fsum() doesn't use a naive loop but a smart implementation to minimize the error.
Would it make sense to only implement product for an iterable of floats, as math.fsum()?
I don't like the name overlap with itertools.product(). Currently, math and itertools have no overlapping names. Also, I expect that like sum(), the prod() function will be used with generator comprehensions and should best be kept short and not dominating the rest of the expression.
It's true that factorial is spelled-out, but that was probably a mistake -- it has been somewhat awkward in expressions that contain factorial terms. Ideally, we would have comb, perm, fact, and prod.
For the record, I would have to look up the documentation everytime I encounter "comb" and "perm", while the full names are intuitive to me.
"prod" and "fact" feel somewhat less obscure.
I suppose there are two possible audiences here:
I'd like to divorce
Not that floats should suffer benign neglect forever, but heroically complex - and expensive - float implementations should get their own function, like
PR 11359 has the following properties in its current state:
Performance vs naive implementation
./python -m perf timeit -s "import functools;import operator;iterable=list(range(10000))" 'functools.reduce(operator.mul, iterable, 1)' .....................
./python -m perf timeit -s "import functools;import operator;iterable=list(map(float,range(10000)))" 'functools.reduce(operator.mul, iterable, 1)'
./python -m perf timeit -s "import math;iterable=list(map(float,range(10000)))" 'math.prod(iterable)' .....................
./python -m perf timeit -s "import functools;import decimal;import operator;iterable=list(map(decimal.Decimal,range(10000)))" 'functools.reduce(operator.mul, iterable, 1)'
./python -m perf timeit -s "import math;import decimal;iterable=list(map(decimal.Decimal,range(10000)))" 'math.prod(iterable)'
In my humble opinion, any type-specialized implementation should exist in addition to this function (as fprod, iprod, scaledProd) while prod should remain as a general purpose function mirroring sum. Notice that people using numerical suites like numpy are used to the properties described in the previous paragraph and I think this is an advantage.
The main advantage of the function as it exists now in PR11359 is convenience and speed (almost 10x for fast paths and 2x for general types). I think this function will be very useful for scientific/statistical computing without the need for pulling in numpy and friends.
What do people think?
Nice to see the arrival of the prod() function.
Just as for the built-in pow(x, y[, z]) function it would be very useful to have an optional argument z for computing products modulo z. Typical use case in cryptography would be:
prod((pow(x, y, z) for x, y in zip(g, s)), z)
to compute the product of all (potentially many) g[i]**s[i]'s modulo z.
And, just as with the use of pow(), the intermediate values for prod() may in general grow quickly, hence modular reduction is essential to limit time and space usage.
Thanks for the suggestion, Mark.
I was not so sure, and the Nosy List for this issue is already quite extensive, so considered you guys as an appropriate audience for my first comment on this platform.
But I will also look into opening a separate issue later today.