String format() has problems parsing numeric indexes

[gosella]

BPO	8985
Nosy	@mdickinson, @ericvsmith, @merwok
Superseder	bpo-7951: Should str.format allow negative indexes when used for getitem access?

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2010-06-12.21:23:18.155>
created_at = <Date 2010-06-12.20:13:59.228>
labels = ['type-bug', 'library', 'expert-unicode']
title = 'String format() has problems parsing numeric indexes'
updated_at = <Date 2010-09-12.23:02:14.845>
user = 'https://bugs.python.org/gosella'

bugs.python.org fields:

activity = <Date 2010-09-12.23:02:14.845>
actor = 'eric.araujo'
assignee = 'none'
closed = True
closed_date = <Date 2010-06-12.21:23:18.155>
closer = 'eric.smith'
components = ['Library (Lib)', 'Unicode']
creation = <Date 2010-06-12.20:13:59.228>
creator = 'gosella'
dependencies = []
files = []
hgrepos = []
issue_num = 8985
keywords = []
message_count = 7.0
messages = ['107691', '107693', '107699', '107705', '107739', '107765', '116243']
nosy_count = 4.0
nosy_names = ['mark.dickinson', 'eric.smith', 'eric.araujo', 'gosella']
pr_nums = []
priority = 'normal'
resolution = 'duplicate'
stage = None
status = 'closed'
superseder = '7951'
type = 'behavior'
url = 'https://bugs.python.org/issue8985'
versions = ['Python 2.6', 'Python 3.1', 'Python 2.7', 'Python 3.2']

[gosella]

The str.format() method allows index lookup on an object that supports __getitem__(). However, negative indexes are not supported.

Examples (using Python 2.6.5):

>>> "{0[0]}".format([0, 1, 2])
'0'

>>> "{0[-1]}".format([0, 1, 2])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: list indices must be integers, not str

>>> u"{0[0]}".format([0, 1, 2])
u'0'
>>> u"{0[-1]}".format([0, 1, 2])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: list indices must be integers, not unicode

Also notice that spaces matter:

>>> "{0[ 0 ]}".format([0, 1, 2])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: list indices must be integers, not str

(The same thing happens on Python 3.1.2)

The problem is that the function get_integer() on Objects/stringlib/string_format.h don't expect spaces or a '-' char, only digits. If the index is not a continuous sequence of digits, it assumes that it is a key for a dict and the index is treated as a string, and that's the cause of the TypeError exception.

This code is the same from 2.6.5 up to trunk.

get_integer() is not very robust to parsing numbers. I'm not familiar with CPython but perhaps the same code used in int(str) can be applied here to take advantage of the better parsing that int() has.

[mdickinson]

The behaviour is by design (I think), though perhaps the error messages could be improved.

Are you asking for negative indices and extra space to be accepted? In that case this should be a feature request rather than a bug report.

[merwok]

FYI, the version field is used to note versions where the bug will be fixed, not versions where it’s found. New features and bug fixes go to the dev branch (3.2), security and documentation fixes go to the stable branches (2.6 and 3.1). (2.7 is in release candidate, so it’s frozen.)

[ericvsmith]

get_integer uses the narrowest possible definition for integer indexes, in order to pass all other strings to mappings.

>>> '{0[ 0 ]} {0[-1]}'.format({' 0 ': 'foo', '-1': 'bar'})
'foo bar'

Remember, it has to guess what type of lookup to do based on whether the value inside [] looks like an integer or not.

From the PEP:
Because keys are not quote-delimited, it is not possible to
specify arbitrary dictionary keys (e.g., the strings "10" or
":-]") from within a format string.

I don't believe this restriction causes any practical problem.

I'm not sure the error could be improved. The code that's being called is essentially:

>>> [0, 1, 2]['-1']
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: list indices must be integers, not str

[gosella]

I now see the rationale behind not accepting ' 10 ' == 10. But what about not accepting '-1' == -1?

I think is odd that negative numbers are not accepted as valid indexes. I'd expect that something like

"First element is {0[0]} and last element is {0[-1]}".format([0,1,2,3])

would work but it didn't.

Should I create a new Issue requesting this as a new feature? Or could be reconsidered as a bug?

I could provide a simple patch that change get_integer() so it would accept negative integers as a valid number.

(This is the first issue I reported and English is not my native language so I apology for any mistake that I made)

[ericvsmith]

I'll consider this a duplicate. bpo-7951 is the existing feature request for this issue. I'll merge the nosy lists.

[merwok]

Just for the record, I said something inexact in my previous message in this thread. New features go to the dev branch (py3k, future 3.2), bug and doc fixes go into py3k and the stable branches (2.7 and 3.1 now), and the previous stable releases (2.5 and 2.6) only get security fixes.