doc: Clarify error message when attempting to call function via str.format()

[mickey695]

BPO	31907
Nosy	@ericvsmith, @serhiy-storchaka, @miss-islington, @iritkatriel
PRs	bpo-31907: [doc] clarify that str.format() does not support arbitrary… #25053 [3.9] bpo-31907: [doc] clarify that str.format() does not support arbitrary expressions (GH-25053) #25055 [3.8] bpo-31907: [doc] clarify that str.format() does not support arbitrary expressions (GH-25053) #25056

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/ericvsmith'
closed_at = <Date 2021-03-28.20:58:54.085>
created_at = <Date 2017-10-31.00:35:20.151>
labels = ['interpreter-core', 'type-feature', '3.10', 'docs']
title = 'doc: Clarify error message when attempting to call function via str.format()'
updated_at = <Date 2021-03-28.21:05:16.373>
user = 'https://bugs.python.org/mickey695'

bugs.python.org fields:

activity = <Date 2021-03-28.21:05:16.373>
actor = 'eric.smith'
assignee = 'eric.smith'
closed = True
closed_date = <Date 2021-03-28.20:58:54.085>
closer = 'eric.smith'
components = ['Documentation', 'Interpreter Core']
creation = <Date 2017-10-31.00:35:20.151>
creator = 'mickey695'
dependencies = []
files = []
hgrepos = []
issue_num = 31907
keywords = ['patch']
message_count = 12.0
messages = ['305263', '305276', '305287', '305413', '305496', '309183', '389405', '389642', '389644', '389648', '389649', '389650']
nosy_count = 5.0
nosy_names = ['eric.smith', 'serhiy.storchaka', 'mickey695', 'miss-islington', 'iritkatriel']
pr_nums = ['25053', '25055', '25056']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue31907'
versions = ['Python 3.10']

[mickey695]

PEP-3101 states that format strings may only use the "."(getattr) or the "[]" (getitem) operators to address either attributes or items of parameters.
Should a programmer attempt to, for example, call a function of a parameter as follows:

>> d = datetime.datetime(2017, 10, 31)
>> "{0.ctime()}".format(d)

they will receive an error message such as:

AttributeError: 'datetime.datetime' object has no attribute 'ctime()'

Proposal:
Raise an error stating that cannot embed arbitrary expressions in str.format() format strings

[ericvsmith]

How would you distinguish this from the case where an actually missing attribute was given?

>>> "{0.ctimex}".format(d)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'datetime.datetime' object has no attribute 'ctimex'

I guess it would be possible to change the parser to only look for valid chars in an identifier and give an error if there were "extra" chars found, but I'm not sure it's worth the hassle. The error message seems clear enough to me. Also, I wouldn't want to change this to not be an AttributeError (for backward compatibility), and I'm not sure an AttributeError with a different message would be good.

Of course, this works with f-strings:

>>> f"{d.ctime()}"
'Tue Oct 31 00:00:00 2017'

And I just realized that your code actually is valid, with a custom __getattr__. And while I wouldn't recommend doing this, I also wouldn't want to break it. For all I know, people have written custom evaluators in __getattr__:

>>> class C:
...     def __getattr__(self, name):
...         if name == 'ctime()':
...             return 'okay'
...         raise AttributeError('not found')
...

>>> '{0.x}'.format(C())
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 5, in __getattr__
AttributeError: not found

>>> '{0.ctime()}'.format(C())
'okay'

I'm changing versions because this would be a new feature, since it breaks valid code.

[serhiy-storchaka]

I have wrote a patch that change the parser to only accept valid identifiers. But now I have the same doubts and want first to discuss this [1]. This change can break the existing code as Eric's example shows.

[1] https://mail.python.org/pipermail/python-dev/2017-October/150052.html

[ericvsmith]

Guido's suggestion in https://mail.python.org/pipermail/python-dev/2017-October/150055.html is to not change anything.

Assuming we do that, it's still an open issue as to if/how we should document and test the current behavior. I'd be surprised if it's tested, but I haven't looked.

[mickey695]

I think it should be documented properly.

In roughly two weeks I will have some time to look into it.
So I could probably document the current behaviour by the start of December(unless someone beats me to it)

[mickey695]

Well, it has been more than the 2 weeks I promised however now I finally have time to look into this.

I did some fuzzing to build a list quickly and the following ASCII characters can not be used as part of the argument name:
exclamation mark(!)
period(.)
colon(:)
opening square bracket([)
opening/closing curly brackets({})

also, the argument name may not start with digits from zero to nine(0-9)

Basically what needs to be changed is arg_name under the section "Format String Syntax" in \Doc\library\string.rst

Problem is, I really don't understand the syntax of the Python/sphinx formal language.

The following expression is what I came up with:
arg_name ::= [^`decinteger`] [<any character except "!" | "." | "[" | "{" | "}"> | integer ]+

If anyone from the documentation team could look into it I would be grateful.

Moreover, I think attribute_name should probably be changed as it does not have to be a valid identifier. In fact, it can be any text. Eric V. Smiths' example can be used with any text as the attribute name. A comment should be added to the aforementioned section explaining that normally only valid identifiers are accepted, however by implementing your own __getattr__ this can be changed.

[iritkatriel]

I would suggest the following change in https://docs.python.org/3/library/string.html#format-string-syntax

replace the sentence:

The syntax is related to that of formatted string literals, but there are differences.

by:

The syntax is related to that of formatted string literals, but it is less sophisticated and, in particular, does not support arbitrary expressions.

Is this accurate? Is it enough?