Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

In str.format an incorrect error message for list, tuple, dict, set #57999

Closed
py-user mannequin opened this issue Jan 15, 2012 · 34 comments
Closed

In str.format an incorrect error message for list, tuple, dict, set #57999

py-user mannequin opened this issue Jan 15, 2012 · 34 comments
Assignees
Labels
3.7 (EOL) end of life 3.8 only security fixes 3.9 only security fixes docs Documentation in the Doc dir easy type-bug An unexpected behavior, bug, or error

Comments

@py-user
Copy link
Mannequin

py-user mannequin commented Jan 15, 2012

BPO 13790
Nosy @terryjreedy, @ericvsmith, @ezio-melotti, @bitdancer, @py-user, @csabella, @miss-islington, @iritkatriel
PRs
  • bpo-13790: Change 'string' to 'specification' in format doc #18690
  • [3.7] bpo-13790: Change 'string' to 'specification' in format doc (GH-18690) #18692
  • [3.8] bpo-13790: Change 'string' to 'specification' in format doc (GH-18690) #18693
  • Files
  • i13790.diff
  • i13790b.diff
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/terryjreedy'
    closed_at = <Date 2020-09-27.14:25:01.174>
    created_at = <Date 2012-01-15.05:31:18.327>
    labels = ['easy', 'type-bug', '3.8', '3.9', '3.7', 'docs']
    title = 'In str.format an incorrect error message for list, tuple, dict, set'
    updated_at = <Date 2020-09-27.14:25:01.173>
    user = 'https://github.com/py-user'

    bugs.python.org fields:

    activity = <Date 2020-09-27.14:25:01.173>
    actor = 'eric.smith'
    assignee = 'terry.reedy'
    closed = True
    closed_date = <Date 2020-09-27.14:25:01.174>
    closer = 'eric.smith'
    components = ['Documentation']
    creation = <Date 2012-01-15.05:31:18.327>
    creator = 'py.user'
    dependencies = []
    files = ['24287', '24290']
    hgrepos = []
    issue_num = 13790
    keywords = ['patch', 'easy']
    message_count = 34.0
    messages = ['151277', '151279', '151300', '151301', '151302', '151306', '151307', '151308', '151316', '151355', '151378', '151709', '151711', '151720', '151722', '151725', '151728', '151730', '151738', '151757', '164343', '164345', '164373', '220399', '220401', '220411', '220412', '312821', '312841', '362910', '362914', '362916', '362917', '377570']
    nosy_count = 9.0
    nosy_names = ['terry.reedy', 'eric.smith', 'ezio.melotti', 'r.david.murray', 'docs@python', 'py.user', 'cheryl.sabella', 'miss-islington', 'iritkatriel']
    pr_nums = ['18690', '18692', '18693']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'behavior'
    url = 'https://bugs.python.org/issue13790'
    versions = ['Python 3.7', 'Python 3.8', 'Python 3.9']

    @py-user
    Copy link
    Mannequin Author

    py-user mannequin commented Jan 15, 2012

    >>> '{0:d}'.format('a')
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    ValueError: Unknown format code 'd' for object of type 'str'
    >>> '{0:d}'.format(1+1j)
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    ValueError: Unknown format code 'd' for object of type 'complex'
    >>> '{0:d}'.format([])
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    ValueError: Unknown format code 'd' for object of type 'str'
    >>>
    
    also strange behavior:
    >>> '{0:s}'.format((1, 2, 3))
    '(1, 2, 3)'
    >>> '{0:10.5s}'.format([1, 2, 3])
    '[1, 2     '
    >>>

    @py-user py-user mannequin added interpreter-core (Objects, Python, Grammar, and Parser dirs) type-bug An unexpected behavior, bug, or error labels Jan 15, 2012
    @ericvsmith
    Copy link
    Member

    I agree it's not the best error message. What's happening is that these types (list, tuple, etc.) do not implement __format__, so object.__format__ is used. It returns str(self). Then the resulting string is formatted with the given format_spec. Since str does not support the 'd' format type, the error you see is raised.

    I'm open to suggestions on how to improve this, but I don't see how it's possible given what str.__format__ knows when it generates the error.

    @ericvsmith ericvsmith self-assigned this Jan 15, 2012
    @py-user
    Copy link
    Mannequin Author

    py-user mannequin commented Jan 15, 2012

    also strange(unobvious) behavior:
    >>> '{0:.3s}'.format((i for i in (1, 2, 3)))
    '<ge'
    >>> '{0:.3s}'.format(range(10))
    'ran'
    >>> '{0:.3s}'.format(None)
    'Non'
    >>>

    it would be better to print an error:
    ValueError: Unknown format code 's' for object of type 'generator'

    like in this:
    >>> '{0:d}'.format(4.5)
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    ValueError: Unknown format code 'd' for object of type 'float'
    >>>

    in the documentation there is nothing about it

    @bitdancer
    Copy link
    Member

    No, it wouldn't. I expect

    "{}".format(x)

    to produce something for an arbitrary x. Breaking that would break a fundamental Python contract.

    Improving the error message for 'd' is more possible. Perhaps "the format code 'd' is not implemented by objects of type <type>"?

    @bitdancer
    Copy link
    Member

    Oh, and when you say there is nothing in the documentation about the 's' case for arbitrary objects, it is made clear in various places that every object has an str, which defaults to its repr if it has no specific __str__. Combine that with the description of what happens when you use a fixed field length for 's', and you get the results you see. There should be nothing surprising about this to anyone who has read the tutorial, I think. (But specific suggestions for improving the docs are always welcome.)

    @py-user
    Copy link
    Mannequin Author

    py-user mannequin commented Jan 15, 2012

    R. David Murray wrote:

    it is made clear in various places that every object has an str

    here:
    http://docs.python.org/py3k/library/string.html#format-specification-mini-language

    3rd paragraph:
    "A general convention is that an empty format string ("") produces the same result as if you had called str() on the value. A non-empty format string typically modifies the result."

    "an empty format string ("")" what does it mean ?

    "".format(value) or "{}".format(value) or "{0}".format(value) ?

    @py-user
    Copy link
    Mannequin Author

    py-user mannequin commented Jan 15, 2012

    also here:
    http://docs.python.org/py3k/library/string.html#format-examples

    there is no example with list or tuple to know exactly how they are formatted

    @bitdancer
    Copy link
    Member

    "an empty format string" is exactly what I was talking about. Putting nothing between the {}'s is an empty format string. I can't think of any way to make that wording clearer.

    The format docs should not contains examples of the repr of all possible python objects. The examples of what tuples and lists and dicts &c look like are shown in the docs for those objects.

    @py-user
    Copy link
    Mannequin Author

    py-user mannequin commented Jan 16, 2012

    R. David Murray wrote:

    Putting nothing between the {}'s is an empty format string.

    this is an empty replacement field

    here:
    http://docs.python.org/py3k/library/string.html#format-string-syntax

    the definition of format string:
    "Format strings contain “replacement fields” surrounded by curly braces {}. Anything that is not contained in braces is considered literal text, which is copied unchanged to the output."

    "The grammar for a replacement field is as follows:"
    replacement_field ::= "{" [field_name] ["!" conversion] [":" format_spec] "}"

    @bitdancer
    Copy link
    Member

    Good point. That should be fixed. It should be "empty format specification".

    @ericvsmith
    Copy link
    Member

    Changing to a documentation issue.

    @ericvsmith ericvsmith added easy docs Documentation in the Doc dir and removed interpreter-core (Objects, Python, Grammar, and Parser dirs) labels Jan 16, 2012
    @ericvsmith ericvsmith assigned docspython and unassigned ericvsmith Jan 16, 2012
    @terryjreedy
    Copy link
    Member

    Doc patch attached to make sure correct. Should {} be quoted?

    Eric, do you want to close off the idea of changing :d errors, or switch back after the doc fix?

    @ericvsmith
    Copy link
    Member

    I don't think "{}" is the correct way to document this. These all have an empty format specifier:

    "{}".format(foo)
    "{:}".format(foo)
    "{0}".format(foo)
    "{0:}".format(foo)
    "{name}".format(name=foo)
    format(foo, "")
    format(foo)

    That is, they all call foo.__format__(""). If foo.__format__ (well, really type(foo).__format__) doesn't exist, then object.__format__(foo, "") gets called. It's object.__format__ that's checking for the empty format string, and if so it returns str(foo).

    What would you suggest changing the ':d' error message to, for objects that don't support a format type of 'd'? This makes sense to me:

    >>> format('', 'd')
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    ValueError: Unknown format code 'd' for object of type 'str'
    
    The problem, if there is one, is:
    >>> format([], 'd')
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    ValueError: Unknown format code 'd' for object of type 'str'

    The problem is that the str that's producing this error doesn't know that it exists because object.__format__ returned str([]).

    @bitdancer
    Copy link
    Member

    Oh, I see. Yes, that is a problem.

    object.__format__ knows the type of the object it was called on, right? Couldn't it catch the error and re-raise it with the correct type? (If the type isn't str, of course, we don't want to get into an infinite recursion.)

    @bitdancer
    Copy link
    Member

    Oh, never mind that comment about recursion, I wasn't thinking it through.

    @terryjreedy
    Copy link
    Member

    OK, the example of an empty format spec should be dropped. Let people figure it out ;-).

    >>> format([], 'd')
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    ValueError: Unknown format code 'd' for object of type 'str'

    One possibility is to give (str of) the object instead of the type:

    ValueError: Unknown format code 'd' for object '[]'

    The downside is a long message for long strings. It would need to be limited (as is done in test error reports).

    @ericvsmith
    Copy link
    Member

    While looking at object.__format__, I recall that we've already addressed this, sort of. For a different reason, this is already deprecated in 3.3 and will become an error in 3.4. See issues 9856 and 7994.

    $ ./python -Wd
    Python 3.3.0a0 (default:40e1be1e0707, Jan 15 2012, 00:58:51) 
    [GCC 4.6.1] on linux
    Type "help", "copyright", "credits" or "license" for more information.
    >>> format([], 'd')
    __main__:1: DeprecationWarning: object.__format__ with a non-empty format string is deprecated
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    ValueError: Unknown format code 'd' for object of type 'str'
    [67288 refs]
    >>> 

    We could still have object.__format__ catch and re-throw the ValueError with a better message. I'd have to think it through if we could catch all ValueErrors, or if it's possible for another ValueError to be thrown and we'd only catch and rethrow this specific ValueError.

    But since this is deprecated, I'm not sure it's worth the hassle. I'd advocate closing this issue as "won't fix".

    @bitdancer
    Copy link
    Member

    So the error is going to be something about the source type not supporting '__format__'?

    That change will also address the OP's concern about truncated reprs when a fixed string length is specified, so I agree that the title issue can be closed. Terry's patch with the ("{}") removed should be committed, though.

    @ericvsmith
    Copy link
    Member

    The error message will be: "non-empty format string passed to object.__format__".

    I agree with your comment about Terry's patch.

    @terryjreedy
    Copy link
    Member

    Looking further, I noticed that 'string' needed to be changed to 'specification' in the following sentence also. Then I decided that the preceding sentence

    "Most built-in types implement the following options for format specifications, although some of the formatting options are only supported by the numeric types."

    should really follow the one about non-empty format specs. This positioning should make it more obvious that most of the options affect the string representation of the object after, not before, the string is produced, and are therefore applicable to all objects and not just string and number objects. I also propose to modify it so it is shorter and no longer contradictory, to read

    "Most built-in types implement various options for such modifications, although some are only supported by the numeric types."

    Further on, under "The available string presentation types are:"
    I think "'s' String format. This is the default type for strings and may be omitted." should have 'and other non-numeric types ' inserted after strings. New patch i13790b.diff attached

    The point of these additional changes is to make it clearer that the default formatting of non-number, non-string objects is to call str() and then apply the options to the resulting string. That makes something like
    >>> format(range(5), '-^20s') # same with object.__format__(), 3.3.0a0
    '----range(0, 5)-----'
    predictable and comprehensible.

    I agree with not making a temporary change (but see below ;-).

    But it seems that the 3.4 message should at least be
    "numeric format string passed to object.__format__" or
    "format string with number-only options passed to object.__format__" or
    "object.__format__ cannot handle number-only options"
    as string formats work fine and, I presume, are not deprecated (?).

    However, if the new ValueError message did not specify object.__format__ (which could still be confusing, even if more accurate), the change could be make now. For instance
    'Numeric option 'd' for non-number object'.
    It would not really matter if it is later raised in object.__format__ instead of str.__format__. I believe *all* of the format codes 'unknown' to str (and by extension, by default, to all other non-number types) *are* number codes.

    @serhiy-storchaka
    Copy link
    Member

    >>> '%d' % ([],)
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    TypeError: %d format: a number is required, not list

    @ericvsmith
    Copy link
    Member

    Serhiy: I'm not sure what you're saying. At the point that str.format() is producing its error message, it doesn't know as much as %-formatting does about the original arguments, so it can't produce a similar message.

    @serhiy-storchaka
    Copy link
    Member

    Serhiy: I'm not sure what you're saying. At the point that str.format() is producing its error message, it doesn't know as much as %-formatting does about the original arguments, so it can't produce a similar message.

    I'm surprised that the code of the classic and the modern formatting is
    so different. Looking deeper, I saw that the issue will go away in 3.4.
    I agree with you in msg151728.

    @BreamoreBoy
    Copy link
    Mannequin

    BreamoreBoy mannequin commented Jun 13, 2014

    Python 3.4.1 (v3.4.1:c0e311e010fc, May 18 2014, 10:38:22) [MSC v.1600 32 bit (Intel)] on win32
    Type "copyright", "credits" or "license()" for more information.
    >>> '{0:d}'.format('a')
    Traceback (most recent call last):
      File "<pyshell#0>", line 1, in <module>
        '{0:d}'.format('a')
    ValueError: Unknown format code 'd' for object of type 'str'

    Nothing appears to have changed despite "the issue will go away in 3.4" in msg164373. What should have happened here?

    @ericvsmith
    Copy link
    Member

    I believe that comment was referring to the subject of this bug:

    $ ./python
    Python 3.4.1+ (3.4:bec6f18dd636, Jun 12 2014, 20:23:30)
    [GCC 4.8.1] on linux
    Type "help", "copyright", "credits" or "license" for more information.
    >>> format([], 'd')
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    TypeError: non-empty format string passed to object.__format__
    >>> format((), 'd')
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    TypeError: non-empty format string passed to object.__format__
    >>> format({}, 'd')
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    TypeError: non-empty format string passed to object.__format__
    >>> format(set(), 'd')
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    TypeError: non-empty format string passed to object.__format__

    With the possible exception of listing the type in this error message, I think these are all fine.

    I'm not sure what you'd expect format('a', 'd') to produce other than the error you're seeing. 'd' is in fact an unknown "format code" for str.

    >>> format('a', 'd')
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    ValueError: Unknown format code 'd' for object of type 'str'

    @py-user
    Copy link
    Mannequin Author

    py-user mannequin commented Jun 13, 2014

    Python 2.7.7 is still printing.

    >>> format([], 'd')
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    ValueError: Unknown format code 'd' for object of type 'str'
    >>>

    @terryjreedy
    Copy link
    Member

    Yes, the deprecation in 3.3 did not apply to 2.7.

    @csabella
    Copy link
    Contributor

    From the examples in msg220401, bpo-28385 changed it to print the object type in the message.

    >>> format([], 'd')
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    TypeError: unsupported format string passed to list.__format__
    >>> format((), 'd')
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    TypeError: unsupported format string passed to tuple.__format__

    Would the change left on this issue be to create a PR for Terry's documetation patch?

    Thanks!

    @csabella csabella added 3.7 (EOL) end of life 3.8 only security fixes labels Feb 25, 2018
    @terryjreedy
    Copy link
    Member

    In msg151730, R. David Murry said "Terry's [first] patch with the ("{}") removed should be committed, though."
    In msg151738, Eric V. Smith said "I agree with your comment about Terry's patch."

    My second patch removed "{}" but also made more text changes, explained in msg151757. Someone should re-review

    @terryjreedy
    Copy link
    Member

    PR-18690 makes the approved change of 'string' to 'specification'. After merging, I will re-review the other changes in i13790b.diff and possibly make another PR for review.

    @terryjreedy terryjreedy added the 3.9 only security fixes label Feb 28, 2020
    @terryjreedy
    Copy link
    Member

    New changeset 916895f by Terry Jan Reedy in branch 'master':
    bpo-13790: Change 'string' to 'specification' in format doc (GH-18690)
    916895f

    @miss-islington
    Copy link
    Contributor

    New changeset 5157506 by Miss Islington (bot) in branch '3.7':
    bpo-13790: Change 'string' to 'specification' in format doc (GH-18690)
    5157506

    @miss-islington
    Copy link
    Contributor

    New changeset 445152e by Miss Islington (bot) in branch '3.8':
    bpo-13790: Change 'string' to 'specification' in format doc (GH-18690)
    445152e

    @iritkatriel
    Copy link
    Member

    This seems complete.

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.7 (EOL) end of life 3.8 only security fixes 3.9 only security fixes docs Documentation in the Doc dir easy type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Development

    No branches or pull requests

    7 participants