In str.format an incorrect error message for list, tuple, dict, set #57999

py-user · 2012-01-15T05:31:18Z

BPO	13790
Nosy	@terryjreedy, @ericvsmith, @ezio-melotti, @bitdancer, @py-user, @csabella, @miss-islington, @iritkatriel
PRs	bpo-13790: Change 'string' to 'specification' in format doc #18690 [3.7] bpo-13790: Change 'string' to 'specification' in format doc (GH-18690) #18692 [3.8] bpo-13790: Change 'string' to 'specification' in format doc (GH-18690) #18693
Files	i13790.diff i13790b.diff

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/terryjreedy'
closed_at = <Date 2020-09-27.14:25:01.174>
created_at = <Date 2012-01-15.05:31:18.327>
labels = ['easy', 'type-bug', '3.8', '3.9', '3.7', 'docs']
title = 'In str.format an incorrect error message for list, tuple, dict, set'
updated_at = <Date 2020-09-27.14:25:01.173>
user = 'https://github.com/py-user'

bugs.python.org fields:

activity = <Date 2020-09-27.14:25:01.173>
actor = 'eric.smith'
assignee = 'terry.reedy'
closed = True
closed_date = <Date 2020-09-27.14:25:01.174>
closer = 'eric.smith'
components = ['Documentation']
creation = <Date 2012-01-15.05:31:18.327>
creator = 'py.user'
dependencies = []
files = ['24287', '24290']
hgrepos = []
issue_num = 13790
keywords = ['patch', 'easy']
message_count = 34.0
messages = ['151277', '151279', '151300', '151301', '151302', '151306', '151307', '151308', '151316', '151355', '151378', '151709', '151711', '151720', '151722', '151725', '151728', '151730', '151738', '151757', '164343', '164345', '164373', '220399', '220401', '220411', '220412', '312821', '312841', '362910', '362914', '362916', '362917', '377570']
nosy_count = 9.0
nosy_names = ['terry.reedy', 'eric.smith', 'ezio.melotti', 'r.david.murray', 'docs@python', 'py.user', 'cheryl.sabella', 'miss-islington', 'iritkatriel']
pr_nums = ['18690', '18692', '18693']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue13790'
versions = ['Python 3.7', 'Python 3.8', 'Python 3.9']

py-user · 2012-01-15T05:31:18Z

>>> '{0:d}'.format('a')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: Unknown format code 'd' for object of type 'str'
>>> '{0:d}'.format(1+1j)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: Unknown format code 'd' for object of type 'complex'
>>> '{0:d}'.format([])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: Unknown format code 'd' for object of type 'str'
>>>

also strange behavior:
>>> '{0:s}'.format((1, 2, 3))
'(1, 2, 3)'
>>> '{0:10.5s}'.format([1, 2, 3])
'[1, 2     '
>>>

ericvsmith · 2012-01-15T05:57:34Z

I agree it's not the best error message. What's happening is that these types (list, tuple, etc.) do not implement __format__, so object.__format__ is used. It returns str(self). Then the resulting string is formatted with the given format_spec. Since str does not support the 'd' format type, the error you see is raised.

I'm open to suggestions on how to improve this, but I don't see how it's possible given what str.__format__ knows when it generates the error.

py-user · 2012-01-15T22:46:03Z

also strange(unobvious) behavior:
>>> '{0:.3s}'.format((i for i in (1, 2, 3)))
'<ge'
>>> '{0:.3s}'.format(range(10))
'ran'
>>> '{0:.3s}'.format(None)
'Non'
>>>

it would be better to print an error:
ValueError: Unknown format code 's' for object of type 'generator'

like in this:
>>> '{0:d}'.format(4.5)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: Unknown format code 'd' for object of type 'float'
>>>

in the documentation there is nothing about it

bitdancer · 2012-01-15T22:54:16Z

No, it wouldn't. I expect

"{}".format(x)

to produce something for an arbitrary x. Breaking that would break a fundamental Python contract.

Improving the error message for 'd' is more possible. Perhaps "the format code 'd' is not implemented by objects of type <type>"?

bitdancer · 2012-01-15T23:00:49Z

Oh, and when you say there is nothing in the documentation about the 's' case for arbitrary objects, it is made clear in various places that every object has an str, which defaults to its repr if it has no specific __str__. Combine that with the description of what happens when you use a fixed field length for 's', and you get the results you see. There should be nothing surprising about this to anyone who has read the tutorial, I think. (But specific suggestions for improving the docs are always welcome.)

py-user · 2012-01-15T23:26:52Z

R. David Murray wrote:

it is made clear in various places that every object has an str

here:
http://docs.python.org/py3k/library/string.html#format-specification-mini-language

3rd paragraph:
"A general convention is that an empty format string ("") produces the same result as if you had called str() on the value. A non-empty format string typically modifies the result."

"an empty format string ("")" what does it mean ?

"".format(value) or "{}".format(value) or "{0}".format(value) ?

py-user · 2012-01-15T23:29:52Z

also here:
http://docs.python.org/py3k/library/string.html#format-examples

there is no example with list or tuple to know exactly how they are formatted

bitdancer · 2012-01-15T23:49:48Z

"an empty format string" is exactly what I was talking about. Putting nothing between the {}'s is an empty format string. I can't think of any way to make that wording clearer.

The format docs should not contains examples of the repr of all possible python objects. The examples of what tuples and lists and dicts &c look like are shown in the docs for those objects.

py-user · 2012-01-16T04:25:47Z

R. David Murray wrote:

Putting nothing between the {}'s is an empty format string.

this is an empty replacement field

here:
http://docs.python.org/py3k/library/string.html#format-string-syntax

the definition of format string:
"Format strings contain “replacement fields” surrounded by curly braces {}. Anything that is not contained in braces is considered literal text, which is copied unchanged to the output."

"The grammar for a replacement field is as follows:"
replacement_field ::= "{" [field_name] ["!" conversion] [":" format_spec] "}"

bitdancer · 2012-01-16T12:53:05Z

Good point. That should be fixed. It should be "empty format specification".

ericvsmith · 2012-01-16T16:19:26Z

Changing to a documentation issue.

terryjreedy · 2012-01-20T23:44:42Z

Doc patch attached to make sure correct. Should {} be quoted?

Eric, do you want to close off the idea of changing :d errors, or switch back after the doc fix?

ericvsmith · 2012-01-21T00:16:21Z

I don't think "{}" is the correct way to document this. These all have an empty format specifier:

"{}".format(foo)
"{:}".format(foo)
"{0}".format(foo)
"{0:}".format(foo)
"{name}".format(name=foo)
format(foo, "")
format(foo)

That is, they all call foo.__format__(""). If foo.__format__ (well, really type(foo).__format__) doesn't exist, then object.__format__(foo, "") gets called. It's object.__format__ that's checking for the empty format string, and if so it returns str(foo).

What would you suggest changing the ':d' error message to, for objects that don't support a format type of 'd'? This makes sense to me:

>>> format('', 'd')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: Unknown format code 'd' for object of type 'str'

The problem, if there is one, is:
>>> format([], 'd')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: Unknown format code 'd' for object of type 'str'

The problem is that the str that's producing this error doesn't know that it exists because object.__format__ returned str([]).

bitdancer · 2012-01-21T04:48:37Z

Oh, I see. Yes, that is a problem.

object.__format__ knows the type of the object it was called on, right? Couldn't it catch the error and re-raise it with the correct type? (If the type isn't str, of course, we don't want to get into an infinite recursion.)

bitdancer · 2012-01-21T04:50:15Z

Oh, never mind that comment about recursion, I wasn't thinking it through.

terryjreedy · 2012-01-21T05:56:24Z

OK, the example of an empty format spec should be dropped. Let people figure it out ;-).

>>> format([], 'd')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: Unknown format code 'd' for object of type 'str'

One possibility is to give (str of) the object instead of the type:

ValueError: Unknown format code 'd' for object '[]'

The downside is a long message for long strings. It would need to be limited (as is done in test error reports).

ericvsmith · 2012-01-21T09:47:18Z

While looking at object.__format__, I recall that we've already addressed this, sort of. For a different reason, this is already deprecated in 3.3 and will become an error in 3.4. See issues 9856 and 7994.

$ ./python -Wd
Python 3.3.0a0 (default:40e1be1e0707, Jan 15 2012, 00:58:51) 
[GCC 4.6.1] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> format([], 'd')
__main__:1: DeprecationWarning: object.__format__ with a non-empty format string is deprecated
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: Unknown format code 'd' for object of type 'str'
[67288 refs]
>>>

We could still have object.__format__ catch and re-throw the ValueError with a better message. I'd have to think it through if we could catch all ValueErrors, or if it's possible for another ValueError to be thrown and we'd only catch and rethrow this specific ValueError.

But since this is deprecated, I'm not sure it's worth the hassle. I'd advocate closing this issue as "won't fix".

bitdancer · 2012-01-21T14:20:12Z

So the error is going to be something about the source type not supporting '__format__'?

That change will also address the OP's concern about truncated reprs when a fixed string length is specified, so I agree that the title issue can be closed. Terry's patch with the ("{}") removed should be committed, though.

ericvsmith · 2012-01-21T17:35:19Z

The error message will be: "non-empty format string passed to object.__format__".

I agree with your comment about Terry's patch.

terryjreedy · 2012-01-22T02:16:15Z

Looking further, I noticed that 'string' needed to be changed to 'specification' in the following sentence also. Then I decided that the preceding sentence

"Most built-in types implement the following options for format specifications, although some of the formatting options are only supported by the numeric types."

should really follow the one about non-empty format specs. This positioning should make it more obvious that most of the options affect the string representation of the object after, not before, the string is produced, and are therefore applicable to all objects and not just string and number objects. I also propose to modify it so it is shorter and no longer contradictory, to read

"Most built-in types implement various options for such modifications, although some are only supported by the numeric types."

Further on, under "The available string presentation types are:"
I think "'s' String format. This is the default type for strings and may be omitted." should have 'and other non-numeric types ' inserted after strings. New patch i13790b.diff attached

The point of these additional changes is to make it clearer that the default formatting of non-number, non-string objects is to call str() and then apply the options to the resulting string. That makes something like
>>> format(range(5), '-^20s') # same with object.__format__(), 3.3.0a0
'----range(0, 5)-----'
predictable and comprehensible.

I agree with not making a temporary change (but see below ;-).

But it seems that the 3.4 message should at least be
"numeric format string passed to object.__format__" or
"format string with number-only options passed to object.__format__" or
"object.__format__ cannot handle number-only options"
as string formats work fine and, I presume, are not deprecated (?).

However, if the new ValueError message did not specify object.__format__ (which could still be confusing, even if more accurate), the change could be make now. For instance
'Numeric option 'd' for non-number object'.
It would not really matter if it is later raised in object.__format__ instead of str.__format__. I believe *all* of the format codes 'unknown' to str (and by extension, by default, to all other non-number types) *are* number codes.

serhiy-storchaka · 2012-06-29T18:17:31Z

>>> '%d' % ([],)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: %d format: a number is required, not list

ericvsmith · 2012-06-29T18:29:26Z

Serhiy: I'm not sure what you're saying. At the point that str.format() is producing its error message, it doesn't know as much as %-formatting does about the original arguments, so it can't produce a similar message.

serhiy-storchaka · 2012-06-30T07:01:09Z

Serhiy: I'm not sure what you're saying. At the point that str.format() is producing its error message, it doesn't know as much as %-formatting does about the original arguments, so it can't produce a similar message.

I'm surprised that the code of the classic and the modern formatting is
so different. Looking deeper, I saw that the issue will go away in 3.4.
I agree with you in msg151728.

BreamoreBoy · 2014-06-13T00:04:40Z

Python 3.4.1 (v3.4.1:c0e311e010fc, May 18 2014, 10:38:22) [MSC v.1600 32 bit (Intel)] on win32
Type "copyright", "credits" or "license()" for more information.
>>> '{0:d}'.format('a')
Traceback (most recent call last):
  File "<pyshell#0>", line 1, in <module>
    '{0:d}'.format('a')
ValueError: Unknown format code 'd' for object of type 'str'

Nothing appears to have changed despite "the issue will go away in 3.4" in msg164373. What should have happened here?

ericvsmith · 2014-06-13T00:32:11Z

I believe that comment was referring to the subject of this bug:

$ ./python
Python 3.4.1+ (3.4:bec6f18dd636, Jun 12 2014, 20:23:30)
[GCC 4.8.1] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> format([], 'd')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: non-empty format string passed to object.__format__
>>> format((), 'd')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: non-empty format string passed to object.__format__
>>> format({}, 'd')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: non-empty format string passed to object.__format__
>>> format(set(), 'd')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: non-empty format string passed to object.__format__

With the possible exception of listing the type in this error message, I think these are all fine.

I'm not sure what you'd expect format('a', 'd') to produce other than the error you're seeing. 'd' is in fact an unknown "format code" for str.

>>> format('a', 'd')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: Unknown format code 'd' for object of type 'str'

py-user · 2014-06-13T01:21:31Z

Python 2.7.7 is still printing.

>>> format([], 'd')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: Unknown format code 'd' for object of type 'str'
>>>

terryjreedy · 2014-06-13T01:24:04Z

Yes, the deprecation in 3.3 did not apply to 2.7.

csabella · 2018-02-25T17:05:08Z

From the examples in msg220401, bpo-28385 changed it to print the object type in the message.

>>> format([], 'd')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: unsupported format string passed to list.__format__
>>> format((), 'd')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: unsupported format string passed to tuple.__format__

Would the change left on this issue be to create a PR for Terry's documetation patch?

Thanks!

terryjreedy · 2018-02-25T19:14:53Z

In msg151730, R. David Murry said "Terry's [first] patch with the ("{}") removed should be committed, though."
In msg151738, Eric V. Smith said "I agree with your comment about Terry's patch."

My second patch removed "{}" but also made more text changes, explained in msg151757. Someone should re-review

terryjreedy · 2020-02-28T19:46:47Z

PR-18690 makes the approved change of 'string' to 'specification'. After merging, I will re-review the other changes in i13790b.diff and possibly make another PR for review.

terryjreedy · 2020-02-28T19:59:23Z

New changeset 916895f by Terry Jan Reedy in branch 'master':
bpo-13790: Change 'string' to 'specification' in format doc (GH-18690)
916895f

miss-islington · 2020-02-28T20:04:26Z

New changeset 5157506 by Miss Islington (bot) in branch '3.7':
bpo-13790: Change 'string' to 'specification' in format doc (GH-18690)
5157506

miss-islington · 2020-02-28T20:07:01Z

New changeset 445152e by Miss Islington (bot) in branch '3.8':
bpo-13790: Change 'string' to 'specification' in format doc (GH-18690)
445152e

iritkatriel · 2020-09-27T14:03:43Z

This seems complete.

py-user mannequin added interpreter-core (Objects, Python, Grammar, and Parser dirs) type-bug An unexpected behavior, bug, or error labels Jan 15, 2012

ericvsmith self-assigned this Jan 15, 2012

ericvsmith added easy docs Documentation in the Doc dir and removed interpreter-core (Objects, Python, Grammar, and Parser dirs) labels Jan 16, 2012

ericvsmith assigned docspython and unassigned ericvsmith Jan 16, 2012

terryjreedy assigned terryjreedy and unassigned docspython Jan 22, 2012

csabella added 3.7 (EOL) end of life 3.8 (EOL) end of life labels Feb 25, 2018

terryjreedy added the 3.9 only security fixes label Feb 28, 2020

ericvsmith closed this as completed Sep 27, 2020

ezio-melotti transferred this issue from another repository Apr 10, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

In str.format an incorrect error message for list, tuple, dict, set #57999

In str.format an incorrect error message for list, tuple, dict, set #57999

py-user mannequin commented Jan 15, 2012

py-user mannequin commented Jan 15, 2012

ericvsmith commented Jan 15, 2012

py-user mannequin commented Jan 15, 2012

bitdancer commented Jan 15, 2012

bitdancer commented Jan 15, 2012

py-user mannequin commented Jan 15, 2012

py-user mannequin commented Jan 15, 2012

bitdancer commented Jan 15, 2012

py-user mannequin commented Jan 16, 2012

bitdancer commented Jan 16, 2012

ericvsmith commented Jan 16, 2012

terryjreedy commented Jan 20, 2012

ericvsmith commented Jan 21, 2012

bitdancer commented Jan 21, 2012

bitdancer commented Jan 21, 2012

terryjreedy commented Jan 21, 2012

ericvsmith commented Jan 21, 2012

bitdancer commented Jan 21, 2012

ericvsmith commented Jan 21, 2012

terryjreedy commented Jan 22, 2012

serhiy-storchaka commented Jun 29, 2012

ericvsmith commented Jun 29, 2012

serhiy-storchaka commented Jun 30, 2012

BreamoreBoy mannequin commented Jun 13, 2014

ericvsmith commented Jun 13, 2014

py-user mannequin commented Jun 13, 2014

terryjreedy commented Jun 13, 2014

csabella commented Feb 25, 2018

terryjreedy commented Feb 25, 2018

terryjreedy commented Feb 28, 2020

terryjreedy commented Feb 28, 2020

miss-islington commented Feb 28, 2020

miss-islington commented Feb 28, 2020

iritkatriel commented Sep 27, 2020

In str.format an incorrect error message for list, tuple, dict, set #57999

In str.format an incorrect error message for list, tuple, dict, set #57999

Comments

py-user mannequin commented Jan 15, 2012

py-user mannequin commented Jan 15, 2012

ericvsmith commented Jan 15, 2012

py-user mannequin commented Jan 15, 2012

bitdancer commented Jan 15, 2012

bitdancer commented Jan 15, 2012

py-user mannequin commented Jan 15, 2012

py-user mannequin commented Jan 15, 2012

bitdancer commented Jan 15, 2012

py-user mannequin commented Jan 16, 2012

bitdancer commented Jan 16, 2012

ericvsmith commented Jan 16, 2012

terryjreedy commented Jan 20, 2012

ericvsmith commented Jan 21, 2012

bitdancer commented Jan 21, 2012

bitdancer commented Jan 21, 2012

terryjreedy commented Jan 21, 2012

ericvsmith commented Jan 21, 2012

bitdancer commented Jan 21, 2012

ericvsmith commented Jan 21, 2012

terryjreedy commented Jan 22, 2012

serhiy-storchaka commented Jun 29, 2012

ericvsmith commented Jun 29, 2012

serhiy-storchaka commented Jun 30, 2012

BreamoreBoy mannequin commented Jun 13, 2014

ericvsmith commented Jun 13, 2014

py-user mannequin commented Jun 13, 2014

terryjreedy commented Jun 13, 2014

csabella commented Feb 25, 2018

terryjreedy commented Feb 25, 2018

terryjreedy commented Feb 28, 2020

terryjreedy commented Feb 28, 2020

miss-islington commented Feb 28, 2020

miss-islington commented Feb 28, 2020

iritkatriel commented Sep 27, 2020