str.format and f-string divergence

[thatch]

BPO	37341
Nosy	@warsaw, @brettcannon, @rhettinger, @larryhastings, @ericvsmith, @thatch, @ambv, @serhiy-storchaka, @lisroach

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2019-06-19.19:01:44.738>
created_at = <Date 2019-06-19.17:36:46.767>
labels = ['type-feature', '3.9', 'docs']
title = 'str.format and f-string divergence'
updated_at = <Date 2019-06-20.08:24:56.943>
user = 'https://github.com/thatch'

bugs.python.org fields:

activity = <Date 2019-06-20.08:24:56.943>
actor = 'eric.smith'
assignee = 'docs@python'
closed = True
closed_date = <Date 2019-06-19.19:01:44.738>
closer = 'brett.cannon'
components = ['Documentation']
creation = <Date 2019-06-19.17:36:46.767>
creator = 'thatch'
dependencies = []
files = []
hgrepos = []
issue_num = 37341
keywords = []
message_count = 6.0
messages = ['346065', '346066', '346070', '346073', '346077', '346109']
nosy_count = 10.0
nosy_names = ['barry', 'brett.cannon', 'rhettinger', 'larry', 'eric.smith', 'thatch', 'docs@python', 'lukasz.langa', 'serhiy.storchaka', 'lisroach']
pr_nums = []
priority = 'normal'
resolution = 'rejected'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue37341'
versions = ['Python 3.9']

[thatch]

TL;DR

f"{x+2}" and f"{x=}" do something sensible.
"{x+2}".format(x=1) and "{x=}".format(x=1) raise KeyError.
f"{0.1}" and "{0.1}".format(...) are different.

Having had a feature request to be able to codemod f-strings[1] and just learning about bpo-36817[2] last night, I went digging into how these are parsed in the standard library to be able to reuse that.

str.format, string.Formatter, and the _string module can only parse literal keys, not expressions, despite appearing to take the same syntax as f-strings. I'm happy to contribute code to change this, but unsure if it's considered a bug or feature (now that we're past feature freeze for 3.8). I would love to see these converge to prevent confusion and let us document in just one place.

The {} and {1} style are still exceptional.

We already parse expressions and the "=" suffix acceptably,

>>> list(_string.formatter_parser("{1+1}"))
[('', '1+1', '', None)]
>>> list(_string.formatter_parser("{(x)}"))
[('', '(x)', '', None)]
>>> list(_string.formatter_parser("{x=!r:1}"))
[('', 'x=', '1', 'r')]
>>> list(_string.formatter_parser("{ x = }"))
[('', ' x = ', '', None)]

But the consumers would need to check for /=\s*$/ and call eval on the items in formatter_field_name_split. (I lack a good heuristic, maybe we eval every time unless it's strictly numeric?)

It would also break unusual uses like these in the name of unification

>>> "{1+1}".format(**{"1+1": "zzz"})
'zzz'

and

>>> class T:
...   pass
... 
>>> setattr(T, "0", "zero")
>>> f"{T.0}"
  File "<fstring>", line 1
    (T.0)
       ^
SyntaxError: invalid syntax
>>> "{0.0}".format(T)
'zero'

[1] facebookincubator/Bowler#87
[2] incorrectly listed in Misc/NEWS.d/3.8.0b1.rst as bpo-36774 btw

[ericvsmith]

str.format, string.Formatter, and the _string module can only
parse literal keys, not expressions, despite appearing to take the
same syntax as f-strings. I'm happy to contribute code to change
this, but unsure if it's considered a bug or feature (now that we're
past feature freeze for 3.8). I would love to see these converge to
prevent confusion and let us document in just one place.

It's a feature that str.format does not accept expressions. If it did, it would:
1: require compiler support (eval isn't good enough)
2: be a security hole magnet

I think both of these are sufficiently strong arguments that I won't support making expressions work in str.format().

For the security hole, you don't want:

user_provided_string.format()

to be able to execute arbitrary code. You're basically eval-ing (parts of) strings, and everyone would suddenly have to audit all of their code to make sure there are no security holes exposed.

[rhettinger]

I concur with Eric. This differences are there by design. The technical and security reasons are both compelling.

[brettcannon]

I also agree with Eric's assessment.

Thanks for the idea, Tim, but I'm closing this as rejected.

[thatch]

ok, I suppose it's just documentation then.

[ericvsmith]

I've fixed the bpo number in Misc/NEWS.d/3.8.0b1.rst. Thanks for reporting that.