slice.indices raises OverflowError

[PaulUpchurch]

BPO	14794
Nosy	@mdickinson, @ned-deily, @merwok, @hynek, @serhiy-storchaka
Files	issue14794.patch issue14794_v2.patch issue14794_v3.patch issue14794_v4.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/mdickinson'
closed_at = <Date 2012-11-10.14:54:39.068>
created_at = <Date 2012-05-12.22:51:37.984>
labels = ['interpreter-core', 'type-feature']
title = 'slice.indices raises OverflowError'
updated_at = <Date 2012-11-17.19:23:26.519>
user = 'https://bugs.python.org/PaulUpchurch'

bugs.python.org fields:

activity = <Date 2012-11-17.19:23:26.519>
actor = 'mark.dickinson'
assignee = 'mark.dickinson'
closed = True
closed_date = <Date 2012-11-10.14:54:39.068>
closer = 'mark.dickinson'
components = ['Interpreter Core']
creation = <Date 2012-05-12.22:51:37.984>
creator = 'Paul.Upchurch'
dependencies = []
files = ['27862', '27869', '27871', '27878']
hgrepos = []
issue_num = 14794
keywords = ['patch']
message_count = 30.0
messages = ['160500', '160501', '160502', '160506', '160531', '160558', '160583', '160594', '174496', '174544', '174545', '174551', '174552', '174557', '174558', '174675', '174678', '174702', '174715', '174768', '174770', '174771', '174774', '174776', '174778', '174780', '174784', '175279', '175280', '175798']
nosy_count = 7.0
nosy_names = ['mark.dickinson', 'ned.deily', 'eric.araujo', 'python-dev', 'hynek', 'Paul.Upchurch', 'serhiy.storchaka']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'patch review'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue14794'
versions = ['Python 3.4']

[PaulUpchurch]

To reproduce the error:

Python 3.2.2 (default, Sep  5 2011, 22:09:30) 
[GCC 4.6.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> slice(0,90000,None).indices(12600000000)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OverflowError: cannot fit 'int' into an index-sized integer
>>> 

The correct behaviour is to return (0, 90000, 1).

>>> slice(0,90000,None).indices(600000000)
(0, 90000, 1)

This is related to http://bugs.python.org/issue1456470.

[hynek]

This seems to have been fixed as of 3.2.3 (as shipped with Ubuntu Precise):

Python 3.2.3 (default, Apr 12 2012, 19:08:59) 
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> slice(0,90000,None).indices(12600000000)
(0, 90000, 1)

Current tip works fine too:

Python 3.3.0a3+ (default:b32baa5b7626+, May 10 2012, 14:56:20) 
[GCC 4.6.3] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> slice(0,90000,None).indices(12600000000)
(0, 90000, 1)

I'd close this bug unless I'm missing something?

[PaulUpchurch]

Sorry. I didn't realize there was a 3.2.3 out. I'll close it as fixed.

[merwok]

This seems to have been fixed as of 3.2.3 (as shipped with Ubuntu Precise)

Just a note: you can’t really trust the behavior of Python shipped by Debian or derivative systems because doko (the Debian Python maintainer) backports changes to released versions, which means that Debian’s 3.2.3 may not always behave as python.org’s 3.2.3.

[PaulUpchurch]

That's true; it doesn't work with today's downloads from python.org. The version I tested was win32 but I don't think that should matter. Python has always supported large numbers on 32-bit OSs.

My observations:

[1] Debian Wheezy, python3.2, 3.2.3~rc2-1: Fail
[2] Debian Wheezy, python2.7, 2.7.3~rc2-2.1: Fail
[3] python.org, python3.3, 3.3.0a3: Fail
[4] python.org, python3.2, 3.2.3: Fail

I'll compile 64-bit linux from source and try that.

[1] Python 3.2.3rc2 (default, Mar 21 2012, 06:59:51) [GCC 4.6.3] on linux2
[2] Python 2.7.3rc2 (default, Apr 22 2012, 22:35:38) [GCC 4.6.3] on linux2
[3] Python 3.3.0a3 (default, May 1 2012, 16:25:20) [MSC v.1500 32 bit (Intel)] on win32
[4] Python 3.2.3 (default, Apr 11 2012, 07:15:24) [MSC v.1500 32 bit (Intel)] on win32

[hynek]

I did a little compiling party with official releases and all permutations of Linux, OS X x 3.2.2, 3.2.3 worked. Both ran on 64bit (Linux in a VirtualBox).

Python 3.2.2 (default, May 13 2012, 21:24:38) 
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> slice(0,90000,None).indices(12600000000)
(0, 90000, 1)

Python 3.2.2 (default, May 13 2012, 21:33:57) 
[GCC 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2336.1.00)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> slice(0,90000,None).indices(12600000000)
(0, 90000, 1)

Can we narrow it down to 32bit hosts/OS?

[PaulUpchurch]

The pre-built 64-bit Windows binaries from python.org works.

Python 3.3.0a3 (default, May  1 2012, 16:46:00) [MSC v.1500 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> slice(0,90000,None).indices(12600000000)
(0, 90000, 1)

Python 3.2.3 (default, Apr 11 2012, 07:12:16) [MSC v.1500 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> slice(0,90000,None).indices(12600000000)
(0, 90000, 1)

I think this issue is settled. There are several possible actions for people that find this discussion through a web search.

1. Use Ubuntu 12.04 LTS
2. Compile a fresh copy of 3.2 or 3.3 from python.org.
3. Use a pre-built 3.2 or 3.3 64-bit Windows binary from python.org.
4. Wait for your Linux distribution to catch up.

Hynek, Éric: Thanks for your help.

[ned-deily]

The problem you described is definitely still an issue with 32-bit builds.

$ /usr/local/bin/python3.3
Python 3.3.0a3 (v3.3.0a3:0b53b70a40a0, May  1 2012, 11:39:35) 
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys; sys.maxsize
9223372036854775807
>>> slice(0,90000,None).indices(12600000000)
(0, 90000, 1)
$ /usr/local/bin/python3.3-32
Python 3.3.0a3 (v3.3.0a3:0b53b70a40a0, May  1 2012, 11:39:35) 
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys; print(sys.maxsize)
2147483647
>>> slice(0,90000,None).indices(12600000000)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OverflowError: cannot fit 'int' into an index-sized integer

[serhiy-storchaka]

This should be an issue on 64-bit too.

slice(0,1,None).indices(sys.maxsize+1)

[mdickinson]

For 2.7, I don't see any problem with raising OverflowError for a length that's > sys.maxsize, since it's hard to have sequences larger than that anyway.

For 3.x, I'd also see this behaviour as reasonable, and not a bug. If it's raising OverflowError for lengths *smaller* than sys.maxsize, that's a bug. Unless that's the case, I'd call this a feature request for 3.4, rather than a bug that needs fixing in all the maintenance branches.

[mdickinson]

If it's raising OverflowError for lengths *smaller* than sys.maxsize,
that's a bug.

Ah, reading Ned's comment, it looks like that's exactly what it's doing.