Floor divide should return int

[abalkin]

BPO	22444
Nosy	@tim-one, @rhettinger, @mdickinson, @abalkin, @pitrou, @alex, @encukou, @skrah

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2015-10-02.21:25:03.212>
created_at = <Date 2014-09-19.16:56:55.755>
labels = ['interpreter-core', 'type-bug']
title = 'Floor divide should return int'
updated_at = <Date 2015-10-02.21:25:03.211>
user = 'https://github.com/abalkin'

bugs.python.org fields:

activity = <Date 2015-10-02.21:25:03.211>
actor = 'belopolsky'
assignee = 'none'
closed = True
closed_date = <Date 2015-10-02.21:25:03.212>
closer = 'belopolsky'
components = ['Interpreter Core']
creation = <Date 2014-09-19.16:56:55.755>
creator = 'belopolsky'
dependencies = []
files = []
hgrepos = []
issue_num = 22444
keywords = []
message_count = 24.0
messages = ['227107', '227149', '227151', '227152', '227156', '227158', '227159', '227161', '227162', '227163', '227164', '227165', '227166', '227172', '227180', '227183', '227277', '227279', '227281', '227282', '227301', '227303', '227305', '227306']
nosy_count = 10.0
nosy_names = ['tim.peters', 'rhettinger', 'mark.dickinson', 'belopolsky', 'pitrou', 'casevh', 'Arfrever', 'alex', 'petr.viktorin', 'skrah']
pr_nums = []
priority = 'normal'
resolution = 'rejected'
stage = 'needs patch'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue22444'
versions = ['Python 3.5']

[abalkin]

PEP-3141 defines floor division as floor(x/y) and specifies that floor() should return int type. Builtin float type has been made part of the PEP-3141 numerical tower, but floor division of two floats still results in a float.

See also:

• bpo-1656 - Make math.{floor,ceil}(float) return ints per PEP-3141
• bpo-1623 - Implement PEP-3141 for Decimal
• https://mail.python.org/pipermail/python-ideas/2014-September/029392.html

[pitrou]

Is this change compelling enough to break compatibility, or is it just a matter of purity?

[skrah]

Perhaps it's worth mentioning that several people on Python-ideas took the
opposite view: math.floor() should return a float.

PEP-3141 does not mention Infinities and NaNs:

"The Real ABC indicates that the value is on the real line, and supports
the operations of the float builtin. Real numbers are totally ordered
except for NaNs (which this PEP basically ignores)."

Floats, however, are on the extended real number line, so we have a problem. :)

Other languages
===============

The PEP says that inspiration came from Scheme and Haskell.

However, Scheme returns floats:
-------------------------------

https://mail.python.org/pipermail/python-ideas/2014-September/029432.html

Haskell seems to return the highest representable integer:
----------------------------------------------------------

Prelude> floor (1/0)
179769313486231590772930519078902473361797697894230657273430081157732675805500963132708477322407536021120113879871393357658789768814416622492847430639474124377767893424865485276302219601246094119453082952085005768838150682342462881473913110540827237163350510684586298239947245938479716304835356329624224137216

However, Haskell float support looks sketchy:
---------------------------------------------

Prelude> floor (0/0)
-269653970229347386159395778618353710042696546841345985910145121736599013708251444699062715983611304031680170819807090036488184653221624933739271145959211186566651840137298227914453329401869141179179624428127508653257226023513694322210869665811240855745025766026879447359920868907719574457253034494436336205824

Prelude> let x = 1 / 0
Prelude> x
Infinity
Prelude> x / 0
Infinity

Considering the last two examples, I think Haskell should not provide any
guidance here. ;)

[skrah]

Argh, forget the second Haskell example: inf / 0 is fine.

[abalkin]

Is this change compelling enough to break compatibility,
or is it just a matter of purity?

According to the PEP-3141, Integer is a subtype of Real, so one should be able to substitute an Integer whenever Real is expected. The reverse is not true, for example

>>> [1,2][1.0]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: list indices must be integers, not float

This is one of the common uses of floor division - to find an index of a cell in a regular grid: (x - start) // step. In this situation, it is convenient to have the result ready to be used as an index without a cast.

The substitution principle also suggests that compatibility issues are likely to be small: in most contexts integers behave the same as floats or "better".

Here is an example of a "better" behavior:

>>> x = 1 + 10**50
>>> x * 1 == x
True
>>> x * 1.0 == x
False

The only case I can think of where float result may be preferable is inf // 1 because integers cannot represent infinity. However, this case is arguably already broken.

What are the use-cases for float // float where integer result is not acceptable?

[abalkin]

However, Scheme returns floats

Does Scheme's default integer type support arbitrarily large values?

[casevh]

Does Scheme's default integer type support arbitrarily large values?

Yes, at least is does on the version I tested.

[abalkin]

Perhaps it's worth mentioning that several people on Python-ideas
took the opposite view: math.floor() should return a float.

I sympathize with the idea that math module functions should return floats. I find it unfortunate that math.floor delegates to the __floor__ dunder on non-floats instead of doing math.floor(x.__float__()). It would be more natural to have a floor builtin that *always* delegates to __floor__ and keep math a pure float library.

Note that math module provides the means to compute C-style floor:

>>> x = float('inf')
>>> math.modf(x)[1]
inf
>>> x = -3.4
>>> math.modf(x)[1]
-3.0

Maybe we should add floorf, ceilf, etc. as well. This, however, is a different issue from the one at hand here.

[casevh]

What are the use-cases for float // float where integer result is not acceptable?

It can lead to unexpected memory consumption when dealing with
arbitrary precision values. What should Decimal('1e123456')//1 return?
The result is exactly equal to Decimal('1e123456') but the
corresponding Python integer will consume ~55KB of RAM.

I'm also concerned that returning a very large integer will lead users
to assume that the result is more precise than it really is. Assuming
standard 64-bit double format, only the first 53 bits are significant.
All the remaining bits are 0.

----------

---

Python tracker <report@bugs.python.org>
<http://bugs.python.org/issue22444\>

---

[abalkin]

What should Decimal('1e123456')//1 return?

I think Decimal case should be considered separately. Note that unlike float, they are not part of the numerical tower, so PEP-3141 arguments don't apply:

>>> isinstance(1.0, numbers.Real)
True
>>> isinstance(decimal.Decimal(1), numbers.Real)
False

[casevh]

On Sat, Sep 20, 2014 at 9:38 AM, Alexander Belopolsky
<report@bugs.python.org> wrote:

Alexander Belopolsky added the comment:

> Perhaps it's worth mentioning that several people on Python-ideas
> took the opposite view: math.floor() should return a float.

I sympathize with the idea that math module functions should return floats. I find it unfortunate that math.floor delegates to the __floor__ dunder on non-floats instead of doing math.floor(x.__float__()). It would be more natural to have a floor builtin that *always* delegates to __floor__ and keep math a pure float library.

+1

Note that math module provides the means to compute C-style floor:

>>> x = float('inf')
>>> math.modf(x)[1]
inf
>>> x = -3.4
>>> math.modf(x)[1]
-3.0

That's not immediately obvious...

Maybe we should add floorf, ceilf, etc. as well. This, however, is a different issue from the one at hand here.

i think the issues are related. PEP-3141 defines x//y as
int(floor(x/y)). It also defines divmod(x, y) as (x//y, x % y). These
definitions cannot all be satisfied at the same Python's divmod
function takes extra effort to calculate x//y precisely. Those
corrections are not possible via floor().

I maintain gmpy2 which wraps the GMP, MPFR, and MPC arbitrary
precision libraries. I originally implemented x//y as floor(x/y). That
choice lead to errors in divmod() that I've fixed in the development
version. I still need to fix floor division: do I make it compatible
with divmod() or floor()?

My preference would be to define floor division and divmod in terms of
each other and allow math.ceil()/floor()/trunc() to return floating
point values. The current definitions are impossible to satisfy.

I mentioned my concerns about memory growth in another comment. I'm
not as concerned about the unexpected memory growth in floor division
as I am in floor() etc.

[casevh]

>> What should Decimal('1e123456')//1 return?
>
> I think Decimal case should be considered separately.  Note that unlike float, they are not part of the numerical tower, so PEP 3141 arguments don't apply:
>
>>>> isinstance(1.0, numbers.Real)
> True
>>>> isinstance(decimal.Decimal(1), numbers.Real)
> False
>
I maintain gmpy2 and I've had requests to support the numeric tower.
gmpy2 has integral, rational, real, and complex types so I should be
able to.

[pitrou]

What are the use-cases for float // float where integer result is not acceptable?

I can't think of any. I was mostly making the case for conservatism here.

The indexing use case is interesting, although I suppose enumerate() should eliminate most instances of it.

[rhettinger]

Is this change compelling enough to break compatibility,
or is it just a matter of purity?

I agree with Antoine that making this change is a really bad idea.

1. The current behavior has been around for a long time and is implemented in several modules including decimal and fractions. As core devs, we need to keep focused on a priority of making the language stable (not making changes that truly necessary and invalidating all previously published material) and more importantly not adding yet more obstacles to converting from Python 2 to Python 3 (which Guido has called "death by a thousand cuts").

2. The current behavior can be useful it that it allows floor division operations without unexpected type conversions occurring in the middle of an expression. We really don't want to break those use cases.

# Here is a simple example of a chain of calculations
# where preserving the type matters

from __future__ import print_function
from fractions import Fraction
from decimal import Decimal

def f(x, y):
    return x // 3 * 5 / 7 + y

def g(x, y):
    return int(x // 3) * 5 / 7 + y

for x, y in [
(Fraction(85, 7), Fraction(2, 3)),
(Decimal('12.143'), Decimal('0.667')),
(12.143, 0.667),
]:
print(f(x, y), g(x, y))

In Python 2:
------------
8/3 8/3
3.524142857142857142857142857 2.667
3.52414285714 2.667

In Python 3:
------------

3.5238095238095237 3.5238095238095237
Traceback (most recent call last):
  ...
    return int(x // 3) * 5 / 7 + y
TypeError: unsupported operand type(s) for +: 'float' and 'decimal.Decimal'

I am a strong -1 against breaking code that relies on the floor division being type preserving.

The PEP should be revised to say that floor division is defined to return a value that is *equal* to an Integral but not place any restriction on the return type.

[tim-one]

Floor division on floats is an unattractive nuisance and should be removed, period - so there ;-)

But short of that, I favor leaving it alone. Whose life would be improved by changing it to return an int? Not mine - and doing so anyway is bound to break existing code. +1 on changing the PEP instead (as Raymond suggested).

[alex]

I can't say that I've ever used // on floats, but it seems to me anyone doing so (as opposed to normal division + explicit rounding) *intentionally* might be broken by this change, but anyone doing this incidentally is not really in a "gotcha" situation. Since this is a type-specific behavior, and not a value-specific one, I don't really think there's a win in changing the behavior, and staying backwards compatible is much better.

[abalkin]

[Raymond]

The current behavior has been around for a long time and is implemented in several modules including decimal and fractions.

No, in the fractions module floor division returns an int:

>>> type(Fraction(2) // Fraction(1))
<class 'int'>

It is also implemented in the datetime module where

>>> type(timedelta(2) // timedelta(1))
<class 'int'>

[Raymond]
# Here is a simple example of a chain of calculations
# where preserving the type matters
..

def f(x, y):
    return x // 3 * 5 / 7 + y

def g(x, y):
    return int(x // 3) * 5 / 7 + y
[/Raymond]

I am not sure what is the problem here. In Python 3:

>>> f(12.143, 0.667)
3.5241428571428575
>>> g(12.143, 0.667)
3.5241428571428575

[mdickinson]

-1 from me, too. It's an unnecessary change, and the conversion from float to integer potentially expensive compared to the computation of the floating-point result (especially in extended floating-point implementations that allow a wider exponent range).

If this is about consistency between // and math.floor, I'd much rather see math.floor go back to returning a float instead of an int.

[abalkin]

[Raymond]

The PEP should be revised to say that floor division is defined to
return a value that is *equal* to an Integral but not place any
restriction on the return type.

If we take this route, what float('inf') // 1 and float('nan') // 1 should return?

[mdickinson]

If we take this route, what float('inf') // 1 and float('nan') // 1 should return?

Probably exactly the same as they do right now. I think there's an argument that float('inf') // 1 "should have been" float('inf'). But I'm not sure there's much of a case for changing the current return value.

[abalkin]

Mark,

Raymond suggested that "The PEP-3141 should be revised to say that floor division is defined to return a value that is *equal* to an Integral".

Since nan or inf are not *equal* to any Integral, the current implementation does not comply. In the absence of a recommendation in the PEP, implementers of new numeric types are left with little guidance because existing types are inconsistent:

>>> Decimal('inf') // 1
Decimal('Infinity')
>>> float('inf') // 1
nan

[skrah]

Alexander Belopolsky <report@bugs.python.org> wrote:

Raymond suggested that "The PEP-3141 should be revised to say that floor division is defined to return a value that is *equal* to an Integral".

I guess it should say "equal to an Integral or a special value".

Since nan or inf are not *equal* to any Integral, the current implementation does not comply. In the absence of a recommendation in the PEP, implementers of new numeric types are left with little guidance because existing types are inconsistent:

>>> Decimal('inf') // 1
Decimal('Infinity')
>>> float('inf') // 1
nan

I think both should return inf.

[abalkin]

skrah> I think both should return inf.

What about this case:

>>> Decimal('1') // Decimal('-inf')
Decimal('-0')
>>> 1. // float('-inf')
-1.0

[mdickinson]

I think that one's covered by bpo-22198.